From eddbdfe929afdde75c6ebdb808b4892dff507e67 Mon Sep 17 00:00:00 2001 From: Hendrik Cannoodt Date: Fri, 13 Oct 2023 10:05:48 +0200 Subject: [PATCH 1/4] Feature/strict json schema (#564) * Basic implementation of splitting platforms into executors and containers For now: - all docker platforms are containers - all native and nextflow platforms are executors the debug platform is just a platform should we add a docker executor? * define Executor and Container as a standalone trait * look for containerDirective in NextflowPlatform using getContainers * add id field to executor * add executorresources * add placeholder objects * Provide temporary wrapper to fill in new generateExecutor * make modifyFunctionality dependent on generateExecutor Move logic to new & desired function. Moving step by step. * Remove BashWrapper.inputs It has been unused for 3 years and somewhat complicated the code. * Make DebugPlatform an Executor as well so it gets the same interface * Remove modifyFunctionality methods and use generateExecutor method Add a temporary Executor.get() method to get an executor from a platform * Get executor during readConfig Centralizes logic and is the desired functionality anyway * Create AppliedConfig case class Replaces tuple with Config, Option[Executor], Option[Platform] Already makes code more user friendly but will be more so if e.g. some matches are rewritten Will also ease replacing Option[Platform] with List[Container] * refactor passing through appliedConfig as case class More to do * Refactor to pass AppliedConfig more instead of config & executor & platform Add some more lenses to clean up the code * Convert handleSingleConfigDependency to AppliedConfig Piggy back some minor improvements & documentation * Add BuildStatus in AppliedConfig * remove unused helper function * rename static helper functions * add initial implementation for dockercontainer * add wip implementation of executable executor * Push AppliedConfig down all the way down until Config.readConfig Further reduces logic paths Provide implicit conversion from Config to AppliedConfig Piggy back some more Console.xxx.prints * Improve printing of messages only when there is something to print. This reduces printing of empty lines during a `ns build`, while still using the new logger methods instead of resorting to `Console.(out|err).print` * Add executors & containers in Viash Config Add encoders & decoders Container decoders is currently fixed to DockerContainers since we don't have a `type` field. * Rename container to engine Add `type` field in engine allowing for Circe encoders en decoders to be completed further * rework executables * refactor main, cli, and Viash* classes * strip platforms * Convert platforms to engines and executors Only native & docker so far. Nextflow still to do. Adapted testbenches from `-p x` to `--engine x --executor x` Failed tests from 210 to ~78 * Fix binary || and && operations in ExecutableExecutor * Fix TestAllComponentsSuite `-p x` -> `--engine x --executor x` * change testbenches to use engine & executor instead of platforms * Map NextflowPlatform to NextflowExecutor and optionally NativeEngine * apply engines & executors and remove platforms, allow namespace configs to fail applying executor and engine without failing completely Fix some more testbenches * change testbench checking docker tag names now always appending the image name to the tag * Mark missing variable local before assigning it declaring the variable as local and getting the exit code together erases the exit code * Add a line of comment for the missing variable Otherwise, next iteration of the code might undo the change as it is not obvious why the change is made * Add BuildStatus for when no executor or engine could be applied Previously used BuildError, which is not really true and set error true while now we can differentiate and set error false * Remove test covering no longer available functionality Functionality wasn't used in reality so wasn't converted to engine/executor * Remove testbenches for disabled docker chown Verify that disabling chown throws an error instead of silently ignoring the parameter * Rename executor to runner * Update test temporary folder name * Fix --apply_platform -> --apply_engine & --apply_runner and fix ns exec functionality ns exec didn't work when no engines were allocated * Fix running executables Also fix argument shifts with ---engine and ---setup parameters. whoops. * Tweak viash test cache testbench Add a command to run in the container so we can better check the behaviour * Move DockerPlatform.chown == true assert to DockerPlatform While it's nice to have no implementation in the platforms, moving the assert cleans up things quite a bit. * Convert platforms to runners and engines in the Circe decode step Basically perform pre-processing instead of post-processing Added deprecated and removed annotations Remove `getRunners` & `getEngines` methods, now use `runners` & `engines` directly. * Fix deprecation and removed version tags * Add platforms deprecation warning test * Add a validation step for platforms before converting them to engines and runners We can't auto-infer the type so we have to do it manually instead of calling `validator`. * Ignore StdErr warnings about platforms in some tests tests check either start of the output or a match, which will always give issues if there are some deprecation warnings * Remove more references to platforms and adapt testbash to runners/engines Add basic documentation for engines and runners. Should be extended further. * Remove more references to platforms * move platform subclasses to engines and runners * Update where the DockerPlatform.chown assertion happens Was moved to the validation step, so no need to do it again in the prepare step Improve testbench to check the error output too. * Write a changelog entry. * Strip unused imports from platforms * fix test namespaces and update extra viash underscore components Add -r and -e short arguments for --apply_runner and --apply-engine respectively * Move nextflow resources from io/viash/platforms to io/viash/runners * Fix testbench failing only when running locally and while running sbt test When running only that test locally it was successfully too. Turns out to be a collision in the docker image name. Rewritten the test to use the ConfigDeriver functionality. Lots of the custom code for this test was separately implemented in ConfigDeriver. Piggy back a minor improvement in the config_mods AppendTest testbench. * Add backwards compatibility for `platfroms/...` exports Add testbenches to check legacy paths Add extra testbench to check messing resource error during test * Fix some more references to platforms Somehow these didn't show up earlier * Add runner and engine types in config documentation Fix future reference link in static page * Provide platform backwards compatibility in dependencies We want to support already built components without having them first switch to runners & engines * Use 'executable' as output folder for dependency builds Also provide backwards compatibility with platforms (executable <-> native) when matching runnerIds * Fix dependency example indenting * Add default annotations for platforms, runners, engines This allows skipping the values in the json schema * Create files for local dependency tests from scratch Use minimal config Also run output and check output * Add a remote dependency test Tweak the local dependency a bit too * Add test for dependency with nested dependencies * Add a testbench for config mod paths & filters * Fix dependency code still using console prints directly * Remove unused code Use `orElse` instead * Add parameter validation test for long arguments * WIP merge Nextflow tests fail big time added txt files with snippets of significant sections of code that get removed during the merge * move helper functions from platforms/nextflow to runners/nextflow * merge nextflowplatform changes into nextflowrunner * remove txt * Add changelog entry * Reapply code fixes from develop that didn't get merged properly * Remove WorkflowHelper.nf from git and add it into the .gitignore * Allow dependency resolution to use components that were disabled by a query When rebuilding a pipeline with previously built components, one wants to use `-q`, but this resulted in missing dependencies. Now pick up DisabledByQuery components for local dependencies too. * differentiate disabled and disabled by query status in readConfigs * Add options to export json schema in strict and minimal versions 'Minimal' removes 'description' fields. 'Strict' removes alternative field definitions. An exception needs to be made for class subtypes, e.g. arguments, otherwise all arguments need to be StringArguments etc. * Fix Json schema export for docker.port Make string arrays the preferred type * Clean up enums in strict mode, move DockerResolveVolume back to platforms * Skip deprecated classes and fields in strict mode Change platform deprecation from 0.8.0 to 0.9.0 * Add exception for NextflowDirectives strict json schema * Don't output `internal functionality` fields during `config view` Same goes for `ns list` Add a custom `deriveConfiguredEncoderStrict` which checks annotations and remove any field annotated with `@internalFunctionality`. Add a `hasInternalFunctionality` and `hasUndocumented` in the `Parameters` class. This means we now always have to output the parameters fields, even if previously skipped, so add an extra filter on the `data` field. Change the `.info` field from `internalFunctionality` to `undocumented` * apply config_mods before applying parent folders to resources applying config_mods uses serialization which now erases parent folder values Adapt some test benches that specifically were testing serialization * Add 'undocumented' values in the json_schema * Use CollectedSchemas.getParameters to perform the reflection This allows for getting annotations from parent classes too Add test that no internal functionality is specified in the config yaml * Restructure deprecation checking code * Add a changelog entry * Use strict encoder for dependencies and repositories * Fix naming of the testResourcesLens * attempt to relax codecov PR thresholds a little bit Allow upto 1% of coverage decrease * Add test checking json decode validator for internal functionality --------- Co-authored-by: Robrecht Cannoodt --- CHANGELOG.md | 6 + codecov.yml | 12 ++ src/main/scala/io/viash/Main.scala | 4 +- src/main/scala/io/viash/ViashExport.scala | 4 +- src/main/scala/io/viash/cli/CLIConf.scala | 10 ++ src/main/scala/io/viash/config/Config.scala | 36 ++--- src/main/scala/io/viash/config/package.scala | 3 +- .../functionality/arguments/package.scala | 17 +-- .../functionality/dependencies/package.scala | 11 +- .../io/viash/functionality/package.scala | 3 +- .../functionality/resources/package.scala | 19 +-- ...onfiguredDecoderWithDeprecationCheck.scala | 61 ++++---- .../circe/DeriveConfiguredEncoderStrict.scala | 42 ++++++ .../scala/io/viash/lenses/ConfigLenses.scala | 2 + .../io/viash/lenses/FunctionalityLenses.scala | 2 + .../io/viash/platforms/DockerPlatform.scala | 2 +- .../scala/io/viash/platforms/Platform.scala | 1 + .../io/viash/schemas/CollectedSchemas.scala | 11 +- .../scala/io/viash/schemas/JsonSchema.scala | 134 +++++++++++++----- .../io/viash/schemas/ParameterSchema.scala | 14 +- src/main/scala/io/viash/schemas/package.scala | 2 + .../io/viash/TestingAllComponentsSuite.scala | 6 +- .../io/viash/e2e/build/NativeSuite.scala | 12 ++ .../arguments/StringArgumentTest.scala | 9 +- 24 files changed, 299 insertions(+), 124 deletions(-) create mode 100644 codecov.yml create mode 100644 src/main/scala/io/viash/helpers/circe/DeriveConfiguredEncoderStrict.scala diff --git a/CHANGELOG.md b/CHANGELOG.md index eef7ba54b..983bad0af 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,12 @@ TODO add summary In brief, the `native platform` became a `native engine` and `docker platform` became a `docker engine`. Additionally, the `native platform` and `docker platform` became a `executable runner`, `nextflow platform` became a `nextflow runner`. The fields of `docker platform` is split between `docker engine` and `docker runner`: `port`, `workdir`, `setup_strategy`, and `run_args` are captured by the `runner` as they define how the component is run. The other fields are captured by the `engine` as they define the environment in which the component is run. One exception is `chown` which is rarely set to false and is now always enabled. + +## NEW FUNCTIONALITY + +* `export json_schema`: Add a `--strict` option to output a subset of the schema representing the internal structure of the Viash config (PR #564). + +* `config view` and `ns list`: Do not output internal functionality fields (#564). Additionally, add a validation that no internal fields are present when reading a Viash config file. # Viash 0.8.0-RC5 (2023-10-11): Fix run workflow diff --git a/codecov.yml b/codecov.yml new file mode 100644 index 000000000..ad90768f7 --- /dev/null +++ b/codecov.yml @@ -0,0 +1,12 @@ +coverage: + precision: 2 + round: down + status: + project: + default: + target: auto + threshold: 1% + patch: + default: + enabled: yes + threshold: 5% \ No newline at end of file diff --git a/src/main/scala/io/viash/Main.scala b/src/main/scala/io/viash/Main.scala index 34d5c92b4..5434d7462 100644 --- a/src/main/scala/io/viash/Main.scala +++ b/src/main/scala/io/viash/Main.scala @@ -368,7 +368,9 @@ object Main extends Logging { val output = cli.export.json_schema.output.toOption.map(Paths.get(_)) ViashExport.exportJsonSchema( output, - format = cli.export.json_schema.format() + format = cli.export.json_schema.format(), + strict = cli.export.json_schema.strict(), + minimal = cli.export.json_schema.minimal() ) 0 case List(cli.export, cli.export.resource) => diff --git a/src/main/scala/io/viash/ViashExport.scala b/src/main/scala/io/viash/ViashExport.scala index c142cb812..801aa38d5 100644 --- a/src/main/scala/io/viash/ViashExport.scala +++ b/src/main/scala/io/viash/ViashExport.scala @@ -65,8 +65,8 @@ object ViashExport extends Logging { } - def exportJsonSchema(output: Option[Path], format: String): Unit = { - val data = JsonSchema.getJsonSchema + def exportJsonSchema(output: Option[Path], format: String, strict: Boolean, minimal: Boolean): Unit = { + val data = JsonSchema.getJsonSchema(strict, minimal) val str = data.toFormattedString(format) if (output.isDefined) { Files.write(output.get, str.getBytes()) diff --git a/src/main/scala/io/viash/cli/CLIConf.scala b/src/main/scala/io/viash/cli/CLIConf.scala index 1dba098b8..eeeacbfda 100644 --- a/src/main/scala/io/viash/cli/CLIConf.scala +++ b/src/main/scala/io/viash/cli/CLIConf.scala @@ -542,6 +542,16 @@ class CLIConf(arguments: Seq[String]) extends ScallopConf(arguments) with Loggin choices = List("yaml", "json"), descr = "Which output format to use." ) + val strict = registerOpt[Boolean]( + name = "strict", + default = Some(false), + descr = "Whether or not to use export the strict schema variant." + ) + val minimal = registerOpt[Boolean]( + name = "minimal", + default = Some(false), + descr = "Whether or not to output extra schema annotations." + ) } addSubcommand(resource) diff --git a/src/main/scala/io/viash/config/Config.scala b/src/main/scala/io/viash/config/Config.scala index f6aea4d56..8db18d274 100644 --- a/src/main/scala/io/viash/config/Config.scala +++ b/src/main/scala/io/viash/config/Config.scala @@ -85,7 +85,7 @@ case class Config( @default("Empty") engines: List[Engine] = Nil, - @internalFunctionality + @undocumented info: Option[Info] = None ) { @@ -280,33 +280,33 @@ object Config extends Logging { // convert Json into Config val conf0 = Convert.jsonToClass[Config](json2, uri.toString()) - /* CONFIG 1: store parent path in resource to be able to access them in the future */ - val parentURI = uri.resolve("") - val resources = conf0.functionality.resources.map(_.copyWithAbsolutePath(parentURI, projectDir)) - val tests = conf0.functionality.test_resources.map(_.copyWithAbsolutePath(parentURI, projectDir)) - - // copy resources with updated paths into config and return - val conf1 = conf0.copy( - functionality = conf0.functionality.copy( - resources = resources, - test_resources = tests - ) - ) - - /* CONFIG 2: apply post-parse config mods */ + /* CONFIG 1: apply post-parse config mods */ // apply config mods only if need be - val conf2 = + val conf1 = if (confMods.postparseCommands.nonEmpty) { // turn config back into json - val js = encodeConfig(conf1) + val js = encodeConfig(conf0) // apply config mods val modifiedJs = confMods(js, preparse = false) // turn json back into a config modifiedJs.as[Config].fold(throw _, identity) } else { - conf1 + conf0 } + /* CONFIG 1: store parent path in resource to be able to access them in the future */ + val parentURI = uri.resolve("") + val resources = conf1.functionality.resources.map(_.copyWithAbsolutePath(parentURI, projectDir)) + val tests = conf1.functionality.test_resources.map(_.copyWithAbsolutePath(parentURI, projectDir)) + + // copy resources with updated paths into config and return + val conf2 = conf1.copy( + functionality = conf0.functionality.copy( + resources = resources, + test_resources = tests + ) + ) + /* CONFIG 3: add info */ // gather git info // todo: resolve git in project? diff --git a/src/main/scala/io/viash/config/package.scala b/src/main/scala/io/viash/config/package.scala index 3c293fd8a..f12d23857 100644 --- a/src/main/scala/io/viash/config/package.scala +++ b/src/main/scala/io/viash/config/package.scala @@ -27,11 +27,12 @@ package object config { import io.viash.helpers.circe.DeriveConfiguredDecoderFullChecks._ import io.viash.helpers.circe.DeriveConfiguredDecoderWithDeprecationCheck._ import io.viash.helpers.circe.DeriveConfiguredDecoderWithValidationCheck._ + import io.viash.helpers.circe.DeriveConfiguredEncoderStrict._ implicit val customConfig: Configuration = Configuration.default.withDefaults // encoders and decoders for Config - implicit val encodeConfig: Encoder.AsObject[Config] = deriveConfiguredEncoder + implicit val encodeConfig: Encoder.AsObject[Config] = deriveConfiguredEncoderStrict[Config] implicit val decodeConfig: Decoder[Config] = deriveConfiguredDecoderWithValidationCheck[Config].prepare{ checkDeprecation[Config](_) // map platforms to runners and engines diff --git a/src/main/scala/io/viash/functionality/arguments/package.scala b/src/main/scala/io/viash/functionality/arguments/package.scala index 48738c042..553cb9dcf 100644 --- a/src/main/scala/io/viash/functionality/arguments/package.scala +++ b/src/main/scala/io/viash/functionality/arguments/package.scala @@ -22,6 +22,7 @@ import io.circe.generic.extras.semiauto.{deriveConfiguredDecoder, deriveConfigur import cats.syntax.functor._ // for .widen import io.viash.helpers.circe.DeriveConfiguredDecoderFullChecks._ import io.viash.helpers.circe.DeriveConfiguredDecoderWithValidationCheck._ +import io.viash.helpers.circe.DeriveConfiguredEncoderStrict._ import io.viash.exceptions.ConfigParserSubTypeException package object arguments { @@ -62,14 +63,14 @@ package object arguments { } // encoders and decoders for Argument - implicit val encodeStringArgument: Encoder.AsObject[StringArgument] = deriveConfiguredEncoder - implicit val encodeIntegerArgument: Encoder.AsObject[IntegerArgument] = deriveConfiguredEncoder - implicit val encodeLongArgument: Encoder.AsObject[LongArgument] = deriveConfiguredEncoder - implicit val encodeDoubleArgument: Encoder.AsObject[DoubleArgument] = deriveConfiguredEncoder - implicit val encodeBooleanArgumentR: Encoder.AsObject[BooleanArgument] = deriveConfiguredEncoder - implicit val encodeBooleanArgumentT: Encoder.AsObject[BooleanTrueArgument] = deriveConfiguredEncoder - implicit val encodeBooleanArgumentF: Encoder.AsObject[BooleanFalseArgument] = deriveConfiguredEncoder - implicit val encodeFileArgument: Encoder.AsObject[FileArgument] = deriveConfiguredEncoder + implicit val encodeStringArgument: Encoder.AsObject[StringArgument] = deriveConfiguredEncoderStrict[StringArgument] + implicit val encodeIntegerArgument: Encoder.AsObject[IntegerArgument] = deriveConfiguredEncoderStrict[IntegerArgument] + implicit val encodeLongArgument: Encoder.AsObject[LongArgument] = deriveConfiguredEncoderStrict[LongArgument] + implicit val encodeDoubleArgument: Encoder.AsObject[DoubleArgument] = deriveConfiguredEncoderStrict[DoubleArgument] + implicit val encodeBooleanArgumentR: Encoder.AsObject[BooleanArgument] = deriveConfiguredEncoderStrict[BooleanArgument] + implicit val encodeBooleanArgumentT: Encoder.AsObject[BooleanTrueArgument] = deriveConfiguredEncoderStrict[BooleanTrueArgument] + implicit val encodeBooleanArgumentF: Encoder.AsObject[BooleanFalseArgument] = deriveConfiguredEncoderStrict[BooleanFalseArgument] + implicit val encodeFileArgument: Encoder.AsObject[FileArgument] = deriveConfiguredEncoderStrict[FileArgument] implicit def encodeArgument[A <: Argument[_]]: Encoder[A] = Encoder.instance { par => diff --git a/src/main/scala/io/viash/functionality/dependencies/package.scala b/src/main/scala/io/viash/functionality/dependencies/package.scala index 6d0e4838f..22ee2e436 100644 --- a/src/main/scala/io/viash/functionality/dependencies/package.scala +++ b/src/main/scala/io/viash/functionality/dependencies/package.scala @@ -25,13 +25,14 @@ import cats.syntax.functor._ package object dependencies { import io.viash.helpers.circe._ + import io.viash.helpers.circe.DeriveConfiguredEncoderStrict._ // encoders and decoders for Argument - implicit val encodeDependency: Encoder.AsObject[Dependency] = deriveConfiguredEncoder - implicit val encodeGitRepository: Encoder.AsObject[GitRepository] = deriveConfiguredEncoder - implicit val encodeGithubRepository: Encoder.AsObject[GithubRepository] = deriveConfiguredEncoder - implicit val encodeViashhubRepository: Encoder.AsObject[ViashhubRepository] = deriveConfiguredEncoder - implicit val encodeLocalRepository: Encoder.AsObject[LocalRepository] = deriveConfiguredEncoder + implicit val encodeDependency: Encoder.AsObject[Dependency] = deriveConfiguredEncoderStrict + implicit val encodeGitRepository: Encoder.AsObject[GitRepository] = deriveConfiguredEncoderStrict + implicit val encodeGithubRepository: Encoder.AsObject[GithubRepository] = deriveConfiguredEncoderStrict + implicit val encodeViashhubRepository: Encoder.AsObject[ViashhubRepository] = deriveConfiguredEncoderStrict + implicit val encodeLocalRepository: Encoder.AsObject[LocalRepository] = deriveConfiguredEncoderStrict implicit def encodeRepository[A <: Repository]: Encoder[A] = Encoder.instance { par => val typeJson = Json.obj("type" -> Json.fromString(par.`type`)) diff --git a/src/main/scala/io/viash/functionality/package.scala b/src/main/scala/io/viash/functionality/package.scala index 00bddc148..de8b8baad 100644 --- a/src/main/scala/io/viash/functionality/package.scala +++ b/src/main/scala/io/viash/functionality/package.scala @@ -34,9 +34,10 @@ package object functionality extends Logging { import io.viash.helpers.circe.DeriveConfiguredDecoderFullChecks._ import io.viash.helpers.circe.DeriveConfiguredDecoderWithDeprecationCheck._ import io.viash.helpers.circe.DeriveConfiguredDecoderWithValidationCheck._ + import io.viash.helpers.circe.DeriveConfiguredEncoderStrict._ // encoder and decoder for Functionality - implicit val encodeFunctionality: Encoder.AsObject[Functionality] = deriveConfiguredEncoder + implicit val encodeFunctionality: Encoder.AsObject[Functionality] = deriveConfiguredEncoderStrict[Functionality] // add file & direction defaults for inputs & outputs implicit val decodeFunctionality: Decoder[Functionality] = deriveConfiguredDecoderFullChecks diff --git a/src/main/scala/io/viash/functionality/resources/package.scala b/src/main/scala/io/viash/functionality/resources/package.scala index 67292e1ad..613f5c4dc 100644 --- a/src/main/scala/io/viash/functionality/resources/package.scala +++ b/src/main/scala/io/viash/functionality/resources/package.scala @@ -28,6 +28,7 @@ package object resources { import io.viash.helpers.circe.DeriveConfiguredDecoderFullChecks._ import io.viash.helpers.circe._ + import io.viash.helpers.circe.DeriveConfiguredEncoderStrict._ implicit val encodeURI: Encoder[URI] = Encoder.instance { uri => Json.fromString(uri.toString) @@ -37,15 +38,15 @@ package object resources { } // encoders and decoders for Object - implicit val encodeBashScript: Encoder.AsObject[BashScript] = deriveConfiguredEncoder - implicit val encodePythonScript: Encoder.AsObject[PythonScript] = deriveConfiguredEncoder - implicit val encodeRScript: Encoder.AsObject[RScript] = deriveConfiguredEncoder - implicit val encodeJavaScriptScript: Encoder.AsObject[JavaScriptScript] = deriveConfiguredEncoder - implicit val encodeNextflowScript: Encoder.AsObject[NextflowScript] = deriveConfiguredEncoder - implicit val encodeScalaScript: Encoder.AsObject[ScalaScript] = deriveConfiguredEncoder - implicit val encodeCSharpScript: Encoder.AsObject[CSharpScript] = deriveConfiguredEncoder - implicit val encodeExecutable: Encoder.AsObject[Executable] = deriveConfiguredEncoder - implicit val encodePlainFile: Encoder.AsObject[PlainFile] = deriveConfiguredEncoder + implicit val encodeBashScript: Encoder.AsObject[BashScript] = deriveConfiguredEncoderStrict[BashScript] + implicit val encodePythonScript: Encoder.AsObject[PythonScript] = deriveConfiguredEncoderStrict[PythonScript] + implicit val encodeRScript: Encoder.AsObject[RScript] = deriveConfiguredEncoderStrict[RScript] + implicit val encodeJavaScriptScript: Encoder.AsObject[JavaScriptScript] = deriveConfiguredEncoderStrict[JavaScriptScript] + implicit val encodeNextflowScript: Encoder.AsObject[NextflowScript] = deriveConfiguredEncoderStrict[NextflowScript] + implicit val encodeScalaScript: Encoder.AsObject[ScalaScript] = deriveConfiguredEncoderStrict[ScalaScript] + implicit val encodeCSharpScript: Encoder.AsObject[CSharpScript] = deriveConfiguredEncoderStrict[CSharpScript] + implicit val encodeExecutable: Encoder.AsObject[Executable] = deriveConfiguredEncoderStrict[Executable] + implicit val encodePlainFile: Encoder.AsObject[PlainFile] = deriveConfiguredEncoderStrict[PlainFile] implicit def encodeResource[A <: Resource]: Encoder[A] = Encoder.instance { par => diff --git a/src/main/scala/io/viash/helpers/circe/DeriveConfiguredDecoderWithDeprecationCheck.scala b/src/main/scala/io/viash/helpers/circe/DeriveConfiguredDecoderWithDeprecationCheck.scala index da5e58157..a61e2339d 100644 --- a/src/main/scala/io/viash/helpers/circe/DeriveConfiguredDecoderWithDeprecationCheck.scala +++ b/src/main/scala/io/viash/helpers/circe/DeriveConfiguredDecoderWithDeprecationCheck.scala @@ -28,45 +28,52 @@ import io.viash.schemas.ParameterSchema import io.circe.ACursor import io.viash.helpers.Logging +import io.viash.schemas.CollectedSchemas object DeriveConfiguredDecoderWithDeprecationCheck extends Logging { - private def memberDeprecationCheck(name: String, history: List[CursorOp], T: Type): Unit = { - val m = T.member(TermName(name)) - val schema = ParameterSchema(name, "", List.empty, m.annotations) - val deprecated = schema.flatMap(_.deprecated) - val removed = schema.flatMap(_.removed) - if (deprecated.isDefined) { - val d = deprecated.get - val historyString = history.collect{ case df: CursorOp.DownField => df.k }.reverse.mkString(".") - info(s"Warning: .$historyString.$name is deprecated: ${d.message} Deprecated since ${d.deprecation}, planned removal ${d.removal}.") + private def memberDeprecationCheck(name: String, history: List[CursorOp], parameters: List[ParameterSchema]): Unit = { + val schema = parameters.find(p => p.name == name).getOrElse(ParameterSchema("", "", "", None, None, None, None, None, None, None, None, false, false)) + + lazy val historyString = history.collect{ case df: CursorOp.DownField => df.k }.reverse.mkString(".") + + schema.deprecated match { + case Some(d) => + info(s"Warning: .$historyString.$name is deprecated: ${d.message} Deprecated since ${d.deprecation}, planned removal ${d.removal}.") + case _ => + } + schema.removed match { + case Some(r) => + info(s"Error: .$historyString.$name was removed: ${r.message} Initially deprecated ${r.deprecation}, removed ${r.removal}.") + case _ => } - if (removed.isDefined) { - val r = removed.get - val historyString = history.collect{ case df: CursorOp.DownField => df.k }.reverse.mkString(".") - info(s"Error: .$historyString.$name was removed: ${r.message} Initially deprecated ${r.deprecation}, removed ${r.removal}.") + if (schema.hasInternalFunctionality) { + error(s"Error: .$historyString.$name is internal functionality.") + throw new RuntimeException(s"Internal functionality used: .$historyString.$name") } } - private def selfDeprecationCheck(T: Type): Unit = { - val baseClass = T.baseClasses.head - val name = baseClass.fullName.split('.').last - val schema = ParameterSchema("", "", List.empty, baseClass.annotations) - val deprecated = schema.flatMap(_.deprecated) - val removed = schema.flatMap(_.removed) - if (deprecated.isDefined) { - val d = deprecated.get - info(s"Warning: $name is deprecated: ${d.message} Deprecated since ${d.deprecation}, planned removal ${d.removal}.") + private def selfDeprecationCheck(parameters: List[ParameterSchema]): Unit = { + val schema = parameters.find(p => p.name == "__this__").get + + schema.deprecated match { + case Some(d) => + info(s"Warning: ${schema.`type`} is deprecated: ${d.message} Deprecated since ${d.deprecation}, planned removal ${d.removal}.") + case _ => } - if (removed.isDefined) { - val r = removed.get - info(s"Error: $name was removed: ${r.message} Initially deprecated ${r.deprecation}, removed ${r.removal}.") + schema.removed match { + case Some(r) => + info(s"Error: ${schema.`type`} was removed: ${r.message} Initially deprecated ${r.deprecation}, removed ${r.removal}.") + case _ => } } // def checkDeprecation[A](cursor: ACursor)(implicit tag: TypeTag[A]) : ACursor = { - selfDeprecationCheck(typeOf[A]) + val parameters = CollectedSchemas.getParameters[A]() + + selfDeprecationCheck(parameters) + // check each defined 'key' value for (key <- cursor.keys.getOrElse(Nil)) { val isEmpty = @@ -77,7 +84,7 @@ object DeriveConfiguredDecoderWithDeprecationCheck extends Logging { case _ => false } if (!isEmpty) { - memberDeprecationCheck(key, cursor.history, typeOf[A]) + memberDeprecationCheck(key, cursor.history, parameters) } } cursor // return unchanged json info diff --git a/src/main/scala/io/viash/helpers/circe/DeriveConfiguredEncoderStrict.scala b/src/main/scala/io/viash/helpers/circe/DeriveConfiguredEncoderStrict.scala new file mode 100644 index 000000000..78e78f585 --- /dev/null +++ b/src/main/scala/io/viash/helpers/circe/DeriveConfiguredEncoderStrict.scala @@ -0,0 +1,42 @@ +/* + * Copyright (C) 2020 Data Intuitive + * + * This program is free software: you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation, either version 3 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License + * along with this program. If not, see . + */ + +package io.viash.helpers.circe + +import io.circe.{Encoder, Json, HCursor} +// import io.circe.generic.extras.Configuration +import io.circe.generic.extras.semiauto.deriveConfiguredEncoder +import io.circe.generic.extras.encoding.ConfiguredAsObjectEncoder + +import scala.reflect.runtime.universe._ +import shapeless.Lazy +import io.viash.schemas.ParameterSchema +import io.viash.schemas.CollectedSchemas + +object DeriveConfiguredEncoderStrict { + + final def deriveConfiguredEncoderStrict[T](implicit encode: Lazy[ConfiguredAsObjectEncoder[T]], tag: TypeTag[T]) = deriveConfiguredEncoder[T] + .mapJsonObject{ jsonObject => + val parameters = CollectedSchemas.getParameters[T]() + jsonObject.filterKeys( k => + parameters + .find(_.name == k) // find the correct parameter + .map(!_.hasInternalFunctionality) // check if it has the 'internalFunctionality' annotation + .getOrElse(true) // fallback, shouldn't really happen + ) + } +} diff --git a/src/main/scala/io/viash/lenses/ConfigLenses.scala b/src/main/scala/io/viash/lenses/ConfigLenses.scala index 6f904e3f0..4a73ef259 100644 --- a/src/main/scala/io/viash/lenses/ConfigLenses.scala +++ b/src/main/scala/io/viash/lenses/ConfigLenses.scala @@ -31,6 +31,8 @@ object ConfigLenses { val composedNameLens = functionalityLens ^|-> nameLens val composedVersionLens = functionalityLens ^|-> versionLens val composedRequirementsLens = functionalityLens ^|-> requirementsLens + val composedResourcesLens = functionalityLens ^|-> resourcesLens + val composedTestResourcesLens = functionalityLens ^|-> testResourcesLens val composedDependenciesLens = functionalityLens ^|-> dependenciesLens val composedRepositoriesLens = functionalityLens ^|-> repositoriesLens } diff --git a/src/main/scala/io/viash/lenses/FunctionalityLenses.scala b/src/main/scala/io/viash/lenses/FunctionalityLenses.scala index f3d36fb4f..f4fb53f00 100644 --- a/src/main/scala/io/viash/lenses/FunctionalityLenses.scala +++ b/src/main/scala/io/viash/lenses/FunctionalityLenses.scala @@ -26,6 +26,8 @@ object FunctionalityLenses { val nameLens = GenLens[Functionality](_.name) val versionLens = GenLens[Functionality](_.version) val requirementsLens = GenLens[Functionality](_.requirements) + val resourcesLens = GenLens[Functionality](_.resources) + val testResourcesLens = GenLens[Functionality](_.test_resources) val dependenciesLens = GenLens[Functionality](_.dependencies) val repositoriesLens = GenLens[Functionality](_.repositories) } diff --git a/src/main/scala/io/viash/platforms/DockerPlatform.scala b/src/main/scala/io/viash/platforms/DockerPlatform.scala index 585e24b93..1b218f58d 100644 --- a/src/main/scala/io/viash/platforms/DockerPlatform.scala +++ b/src/main/scala/io/viash/platforms/DockerPlatform.scala @@ -175,4 +175,4 @@ case class DockerPlatform( @since("Viash 0.7.4") cmd: Option[Either[String, List[String]]] = None -) extends Platform \ No newline at end of file +) extends Platform diff --git a/src/main/scala/io/viash/platforms/Platform.scala b/src/main/scala/io/viash/platforms/Platform.scala index ebc241559..196a96d4f 100644 --- a/src/main/scala/io/viash/platforms/Platform.scala +++ b/src/main/scala/io/viash/platforms/Platform.scala @@ -41,6 +41,7 @@ import io.viash.engines.requirements.Requirements @subclass("NativePlatform") @subclass("DockerPlatform") @subclass("NextflowPlatform") +@deprecated("Use 'engines' and 'runners' instead.", "0.9.0", "0.10.0") trait Platform { @description("Specifies the type of the platform.") val `type`: String diff --git a/src/main/scala/io/viash/schemas/CollectedSchemas.scala b/src/main/scala/io/viash/schemas/CollectedSchemas.scala index ff1853080..20b25851d 100644 --- a/src/main/scala/io/viash/schemas/CollectedSchemas.scala +++ b/src/main/scala/io/viash/schemas/CollectedSchemas.scala @@ -71,9 +71,10 @@ object CollectedSchemas { private val jsonPrinter = JsonPrinter.spaces2.copy(dropNullValues = true) import io.viash.helpers.circe._ + import io.viash.helpers.circe.DeriveConfiguredEncoderStrict._ private implicit val encodeConfigSchema: Encoder.AsObject[CollectedSchemas] = deriveConfiguredEncoder - private implicit val encodeParameterSchema: Encoder.AsObject[ParameterSchema] = deriveConfiguredEncoder + private implicit val encodeParameterSchema: Encoder.AsObject[ParameterSchema] = deriveConfiguredEncoderStrict private implicit val encodeDeprecatedOrRemoved: Encoder.AsObject[DeprecatedOrRemovedSchema] = deriveConfiguredEncoder private implicit val encodeExample: Encoder.AsObject[ExampleSchema] = deriveConfiguredEncoder @@ -207,12 +208,16 @@ object CollectedSchemas { private val getSchema = (t: (Map[String,List[MemberInfo]], List[Symbol])) => t match { case (members, classes) => { - annotationsOf(members, classes).flatMap{ case (name, tpe, hierarchy, annotations) => ParameterSchema(name, tpe, hierarchy, annotations) } + annotationsOf(members, classes).map{ case (name, tpe, hierarchy, annotations) => ParameterSchema(name, tpe, hierarchy, annotations) } } } + // get all parameters for a given type, including parent class annotations + def getParameters[T: TypeTag]() = getSchema(getMembers[T]()) + // Main call for documentation output - lazy val data: List[List[ParameterSchema]] = schemaClasses.map{ v => getSchema(v)} + lazy val fullData: List[List[ParameterSchema]] = schemaClasses.map{ v => getSchema(v)} + lazy val data: List[List[ParameterSchema]] = fullData.map(_.filter(p => !p.hasUndocumented && !p.hasInternalFunctionality)) def getKeyFromParamList(data: List[ParameterSchema]): String = data.find(p => p.name == "__this__").get.`type` diff --git a/src/main/scala/io/viash/schemas/JsonSchema.scala b/src/main/scala/io/viash/schemas/JsonSchema.scala index 04657ebc6..acb403de4 100644 --- a/src/main/scala/io/viash/schemas/JsonSchema.scala +++ b/src/main/scala/io/viash/schemas/JsonSchema.scala @@ -22,7 +22,12 @@ import io.viash.runners.executable.DockerSetupStrategy object JsonSchema { - lazy val data = CollectedSchemas.data + case class SchemaConfig( + strict: Boolean = false, + minimal: Boolean = false + ) + + lazy val data = CollectedSchemas.fullData.map(_.filter(!_.hasInternalFunctionality)) def typeOrRefJson(`type`: String): (String, Json) = { `type` match { @@ -45,28 +50,36 @@ object JsonSchema { } } - def valueType(`type`: String, description: Option[String] = None): Json = { + def valueType(`type`: String, description: Option[String] = None)(implicit config: SchemaConfig): Json = { + val descr = config.minimal match { + case true => None + case false => description + } Json.obj( - description.map(s => Seq("description" -> Json.fromString(s))).getOrElse(Nil) ++ + descr.map(s => Seq("description" -> Json.fromString(s))).getOrElse(Nil) ++ Seq(typeOrRefJson(`type`)): _* ) } - def arrayType(`type`: String, description: Option[String] = None): Json = { + def arrayType(`type`: String, description: Option[String] = None)(implicit config: SchemaConfig): Json = { arrayJson(valueType(`type`), description) } - def mapType(`type`: String, description: Option[String] = None): Json = { + def mapType(`type`: String, description: Option[String] = None)(implicit config: SchemaConfig): Json = { mapJson(valueType(`type`), description) } - def oneOrMoreType(`type`: String, description: Option[String] = None): Json = { + def oneOrMoreType(`type`: String, description: Option[String] = None)(implicit config: SchemaConfig): Json = { oneOrMoreJson(valueType(`type`, description)) } - def arrayJson(json: Json, description: Option[String] = None): Json = { + def arrayJson(json: Json, description: Option[String] = None)(implicit config: SchemaConfig): Json = { + val descr = config.minimal match { + case true => None + case false => description + } Json.obj( - description.map(s => Seq("description" -> Json.fromString(s))).getOrElse(Nil) ++ + descr.map(s => Seq("description" -> Json.fromString(s))).getOrElse(Nil) ++ Seq( "type" -> Json.fromString("array"), "items" -> json @@ -74,9 +87,13 @@ object JsonSchema { ) } - def mapJson(json: Json, description: Option[String] = None) = { + def mapJson(json: Json, description: Option[String] = None)(implicit config: SchemaConfig) = { + val descr = config.minimal match { + case true => None + case false => description + } Json.obj( - description.map(s => Seq("description" -> Json.fromString(s))).getOrElse(Nil) ++ + descr.map(s => Seq("description" -> Json.fromString(s))).getOrElse(Nil) ++ Seq( "type" -> Json.fromString("object"), "additionalProperties" -> json @@ -84,21 +101,24 @@ object JsonSchema { ) } - def oneOrMoreJson(json: Json): Json = { + def oneOrMoreJson(json: Json)(implicit config: SchemaConfig): Json = { eitherJson( json, arrayJson(json) ) } - def eitherJson(jsons: Json*): Json = { - Json.obj("oneOf" -> Json.arr(jsons: _*)) + def eitherJson(jsons: Json*)(implicit config: SchemaConfig, mustIncludeAll: Boolean = false): Json = { + if (config.strict && !mustIncludeAll) { + jsons.last + } else { + Json.obj("anyOf" -> Json.arr(jsons: _*)) + } } - def getThisParameter(data: List[ParameterSchema]): ParameterSchema = data.find(_.name == "__this__").get - def createSchema(info: List[ParameterSchema]): (String, Json) = { + def createSchema(info: List[ParameterSchema])(implicit config: SchemaConfig): (String, Json) = { def removeMarkup(text: String): String = { val markupRegex = raw"@\[(.*?)\]\(.*?\)".r @@ -110,7 +130,7 @@ object JsonSchema { val thisParameter = getThisParameter(info) val description = removeMarkup(thisParameter.description.get) val subclass = thisParameter.subclass.map(l => l.head) - val properties = info.filter(p => !p.name.startsWith("__")).filter(p => !p.removed.isDefined) + val properties = info.filter(p => !p.name.startsWith("__")).filter(p => !p.removed.isDefined && (!config.strict || !p.deprecated.isDefined)) val propertiesJson = properties.map(p => { val pDescription = p.description.map(s => removeMarkup(s)) val trimmedType = p.`type` match { @@ -120,6 +140,8 @@ object JsonSchema { val mapRegex = "(List)?Map\\[String,(\\w*)\\]".r + implicit val useAllInEither = thisParameter.`type` == "NextflowDirectives" + trimmedType match { case s"List[$s]" => (p.name, arrayType(s, pDescription)) @@ -156,15 +178,15 @@ object JsonSchema { )) case s"OneOrMore[$s]" => - if (s == "String" && p.name == "port" && subclass == Some("docker")) { + if (s == "String" && p.name == "port" && subclass == Some("executable")) { // Custom exception // This is the port field for a excutable runner. // We want to allow a Strings or Ints. (p.name, eitherJson( - valueType("String", pDescription), valueType("Int", pDescription), - arrayType("String", pDescription), - arrayType("Int", pDescription) + valueType("String", pDescription), + arrayType("Int", pDescription), + arrayType("String", pDescription) )) } else { (p.name, oneOrMoreType(s, pDescription)) @@ -174,10 +196,16 @@ object JsonSchema { (p.name, mapType(s, pDescription)) case s if p.name == "type" && subclass.isDefined => - ("type", Json.obj( - "description" -> Json.fromString(description), // not pDescription! We want to show the description of the main class - "const" -> Json.fromString(subclass.get) - )) + if (config.minimal) { + ("type", Json.obj( + "const" -> Json.fromString(subclass.get) + )) + } else { + ("type", Json.obj( + "description" -> Json.fromString(description), // not pDescription! We want to show the description of the main class + "const" -> Json.fromString(subclass.get) + )) + } case s => (p.name, valueType(s, pDescription)) @@ -194,52 +222,82 @@ object JsonSchema { val requiredJson = required.map(p => Json.fromString(p.name)) val k = thisParameter.`type` + val descr = config.minimal match { + case true => None + case false => Some(description) + } val v = Json.obj( - "description" -> Json.fromString(description), - "type" -> Json.fromString("object"), + descr.map(s => Seq("description" -> Json.fromString(s))).getOrElse(Nil) ++ + Seq("type" -> Json.fromString("object"), "properties" -> Json.obj(propertiesJson: _*), "required" -> Json.arr(requiredJson: _*), - "additionalProperties" -> Json.False + "additionalProperties" -> Json.False): _* ) k -> v } - def createSuperClassSchema(info: List[ParameterSchema]): (String, Json) = { + def createSuperClassSchema(info: List[ParameterSchema])(implicit config: SchemaConfig): (String, Json) = { val thisParameter = getThisParameter(info) val k = thisParameter.`type` + implicit val mustIncludeAll = true val v = eitherJson( thisParameter.subclass.get.map(s => Json.obj("$ref" -> Json.fromString(s"#/definitions/$s"))): _* ) k -> v } - def createSchemas(data: List[List[ParameterSchema]]) : Seq[(String, Json)] = { + def createSchemas(data: List[List[ParameterSchema]])(implicit config: SchemaConfig) : Seq[(String, Json)] = { data.flatMap{ - case v if getThisParameter(v).removed.isDefined => None + case v if getThisParameter(v).`type` == "EnvironmentVariables" => None + case v if getThisParameter(v).removed.isDefined || (getThisParameter(v).deprecated.isDefined && config.strict) => None case v if getThisParameter(v).subclass.map(_.length).getOrElse(0) > 1 => Some(createSuperClassSchema(v)) case v => Some(createSchema(v)) } } - def createEnum(values: Seq[String], description: Option[String], comment: Option[String]): Json = { + def createEnum(values: Seq[String], description: Option[String], comment: Option[String])(implicit config: SchemaConfig): Json = { + val descr = config.minimal match { + case true => None + case false => description + } + val comm = config.minimal match { + case true => None + case false => comment + } Json.obj( Seq("enum" -> Json.arr(values.map(s => Json.fromString(s)): _*)) ++ - comment.map(s => Seq("$comment" -> Json.fromString(s))).getOrElse(Nil) ++ - description.map(s => Seq("description" -> Json.fromString(s))).getOrElse(Nil): _* + comm.map(s => Seq("$comment" -> Json.fromString(s))).getOrElse(Nil) ++ + descr.map(s => Seq("description" -> Json.fromString(s))).getOrElse(Nil): _* ) } - def getJsonSchema: Json = { - val definitions = - createSchemas(data) ++ + def getJsonSchema(strict: Boolean, minimal: Boolean): Json = { + implicit val configSchema = SchemaConfig(strict, minimal) + + val enumDefinitions = if (strict) { + Seq( + "DockerSetupStrategy" -> createEnum(DockerSetupStrategy.objs.map(obj => obj.id).toSeq, Some("The Docker setup strategy to use when building a container."), Some("TODO add descriptions to different strategies")), + "Direction" -> createEnum(Seq("input", "output"), Some("Makes this argument an `input` or an `output`, as in does the file/folder needs to be read or written. `input` by default."), None), + "Status" -> createEnum(Seq("enabled", "disabled", "deprecated"), Some("Allows setting a component to active, deprecated or disabled."), None), + "DoubleStrings" -> createEnum(Seq("+infinity", "-infinity", "nan"), None, None) + ) + } else { Seq( "DockerSetupStrategy" -> createEnum(DockerSetupStrategy.map.keys.toSeq, Some("The Docker setup strategy to use when building a container."), Some("TODO add descriptions to different strategies")), "Direction" -> createEnum(Seq("input", "output"), Some("Makes this argument an `input` or an `output`, as in does the file/folder needs to be read or written. `input` by default."), None), "Status" -> createEnum(Seq("enabled", "disabled", "deprecated"), Some("Allows setting a component to active, deprecated or disabled."), None), "DockerResolveVolume" -> createEnum(Seq("manual", "automatic", "auto", "Manual", "Automatic", "Auto"), Some("Enables or disables automatic volume mapping. Enabled when set to `Automatic` or disabled when set to `Manual`. Default: `Automatic`"), Some("TODO make fully case insensitive")), "DoubleStrings" -> createEnum(Seq("+.inf", "+inf", "+infinity", "positiveinfinity", "positiveinf", "-.inf", "-inf", "-infinity", "negativeinfinity", "negativeinf", ".nan", "nan"), None, None) - ) ++ - Seq("DoubleWithInf" -> eitherJson(valueType("Double_"), valueType("DoubleStrings"))) + ) + } + + val definitions = + createSchemas(data) ++ + enumDefinitions ++ + { + implicit val mustIncludeAll = true + Seq("DoubleWithInf" -> eitherJson(valueType("Double_"), valueType("DoubleStrings"))) + } Json.obj( "$schema" -> Json.fromString("https://json-schema.org/draft-07/schema#"), diff --git a/src/main/scala/io/viash/schemas/ParameterSchema.scala b/src/main/scala/io/viash/schemas/ParameterSchema.scala index 3a1dbf398..f0919a7c3 100644 --- a/src/main/scala/io/viash/schemas/ParameterSchema.scala +++ b/src/main/scala/io/viash/schemas/ParameterSchema.scala @@ -18,6 +18,7 @@ package io.viash.schemas import scala.reflect.runtime.universe._ +import io.viash.schemas.internalFunctionality final case class ParameterSchema( name: String, @@ -31,6 +32,10 @@ final case class ParameterSchema( removed: Option[DeprecatedOrRemovedSchema], default: Option[String], subclass: Option[List[String]], + @internalFunctionality + hasUndocumented: Boolean, + @internalFunctionality + hasInternalFunctionality: Boolean, ) object ParameterSchema { @@ -75,7 +80,7 @@ object ParameterSchema { (name, values) } - def apply(name: String, `type`: String, hierarchy: List[String], annotations: List[Annotation]): Option[ParameterSchema] = { + def apply(name: String, `type`: String, hierarchy: List[String], annotations: List[Annotation]): ParameterSchema = { def beautifyTypeName(s: String): String = { @@ -158,11 +163,8 @@ object ParameterSchema { val undocumented = annStrings.exists{ case (name, value) => name.endsWith("undocumented")} val internalFunctionality = annStrings.exists{ case (name, value) => name.endsWith("internalFunctionality")} - internalFunctionality || undocumented match { - case true => None - case _ => Some(ParameterSchema(name_, typeName, beautifyTypeName(typeName), hierarchyOption, description, examples, since, deprecated, removed, default, subclass)) - } - + + ParameterSchema(name_, typeName, beautifyTypeName(typeName), hierarchyOption, description, examples, since, deprecated, removed, default, subclass, undocumented, internalFunctionality) } } diff --git a/src/main/scala/io/viash/schemas/package.scala b/src/main/scala/io/viash/schemas/package.scala index 87469aaaf..eb6210f5f 100644 --- a/src/main/scala/io/viash/schemas/package.scala +++ b/src/main/scala/io/viash/schemas/package.scala @@ -41,9 +41,11 @@ package object schemas { @getter @setter @beanGetter @beanSetter @field class default(default: String) extends scala.annotation.StaticAnnotation + // Do not generate documentation for this field and don't output it during serialization @getter @setter @beanGetter @beanSetter @field class internalFunctionality() extends scala.annotation.StaticAnnotation + // Do not generate documentation for this field but do output it during serialization @getter @setter @beanGetter @beanSetter @field class undocumented() extends scala.annotation.StaticAnnotation diff --git a/src/test/scala/io/viash/TestingAllComponentsSuite.scala b/src/test/scala/io/viash/TestingAllComponentsSuite.scala index fa917c805..0e0a9f064 100644 --- a/src/test/scala/io/viash/TestingAllComponentsSuite.scala +++ b/src/test/scala/io/viash/TestingAllComponentsSuite.scala @@ -4,6 +4,7 @@ import io.viash.config.Config import org.scalatest.funsuite.AnyFunSuite import io.viash.helpers.Logger import org.scalatest.ParallelTestExecution +import io.viash.lenses.ConfigLenses class TestingAllComponentsSuite extends AnyFunSuite with ParallelTestExecution { Logger.UseColorOverride.value = Some(false) @@ -82,8 +83,11 @@ class TestingAllComponentsSuite extends AnyFunSuite with ParallelTestExecution { // convert back to config val conf2 = confJson.as[Config].toOption.get + // strip parent parameters as those are internal functionality and are not serialized + val strippedConf1 = ConfigLenses.composedResourcesLens.modify(_.map(_.copyResource(parent = None)))(conf) + val strippedConf2 = ConfigLenses.composedTestResourcesLens.modify(_.map(_.copyResource(parent = None)))(strippedConf1) // check if equal - assert(conf == conf2) + assert(strippedConf2 == conf2) } } } \ No newline at end of file diff --git a/src/test/scala/io/viash/e2e/build/NativeSuite.scala b/src/test/scala/io/viash/e2e/build/NativeSuite.scala index 01ad22e7f..c0521919b 100644 --- a/src/test/scala/io/viash/e2e/build/NativeSuite.scala +++ b/src/test/scala/io/viash/e2e/build/NativeSuite.scala @@ -244,6 +244,18 @@ class NativeSuite extends AnyFunSuite with BeforeAndAfterAll { assert(testRegex.findFirstIn(testOutput.error).isDefined, testOutput.error) } + test("Test whether setting an internalFunctionality field throws an error") { + val newConfigFilePath = configDeriver.derive(""".functionality.argument_groups[.name == "First group"].arguments[.name == "input"].dest := "foo"""", "set_internal_functionality") + + val testOutput = TestHelper.testMainException2[RuntimeException]( + "build", + "-o", tempFolStr, + newConfigFilePath + ) + + assert(testOutput.error.contains("Error: .functionality.argument_groups.arguments.dest is internal functionality.")) + } + test("Test config without a main script") { val testOutput = TestHelper.testMain( "build", diff --git a/src/test/scala/io/viash/functionality/arguments/StringArgumentTest.scala b/src/test/scala/io/viash/functionality/arguments/StringArgumentTest.scala index baef4aa14..6f989dcdc 100644 --- a/src/test/scala/io/viash/functionality/arguments/StringArgumentTest.scala +++ b/src/test/scala/io/viash/functionality/arguments/StringArgumentTest.scala @@ -83,7 +83,8 @@ class StringArgumentTest extends AnyFunSuite with BeforeAndAfterAll { assert(arg.dest == "meta") val argParsed = arg.asJson.as[StringArgument].fold(throw _, a => a) - assert(argParsed == arg) + // override dest parameter as that is internal functionality and is not serialized + assert(argParsed == arg.copyArg(dest = "par")) } test("copyArg helper function") { @@ -104,7 +105,8 @@ class StringArgumentTest extends AnyFunSuite with BeforeAndAfterAll { ) val arg2GParsed = arg2generic.asJson.as[Argument[_]].fold(throw _, a => a) - assert(arg2GParsed == arg2generic) + // override dest parameter as that is internal functionality and is not serialized + assert(arg2GParsed == arg2generic.copyArg(dest = "par")) assert(arg2generic.isInstanceOf[StringArgument]) val arg2 = arg2generic.asInstanceOf[StringArgument] @@ -123,6 +125,7 @@ class StringArgumentTest extends AnyFunSuite with BeforeAndAfterAll { assert(arg2.dest == "meta") val arg2Parsed = arg2.asJson.as[StringArgument].fold(throw _, a => a) - assert(arg2Parsed == arg2) + // override dest parameter as that is internal functionality and is not serialized + assert(arg2Parsed == arg2.copyArg(dest = "par")) } } From 6afbc302d99e50650daef71c47d4e491ae7310f7 Mon Sep 17 00:00:00 2001 From: Hendrik Cannoodt Date: Fri, 13 Oct 2023 11:25:51 +0200 Subject: [PATCH 2/4] Feature/local dependency test2 (#565) * Test local repositories Test with both absolute and relative paths. * Refactor TestHelper class Extend ExceptionOutput with exitCode and use it in all functions Remove testMain and replace it with testMainWithStdErr functionality and rename it testMain as the sole function Combine functionality of testMainException and testMainException2 * Fix modified testbench checking the wrong output for text * Fix cherry picking * Add changelog entry * Fix merge --- CHANGELOG.md | 6 + src/test/scala/io/viash/TestHelper.scala | 79 ++---- .../MainBuildAuxiliaryDockerChown.scala | 4 +- ...MainBuildAuxiliaryDockerRequirements.scala | 18 +- ...nBuildAuxiliaryDockerResourceCopying.scala | 4 +- .../MainTestAuxiliaryDockerResourceCopy.scala | 24 +- .../io/viash/e2e/build/DockerMoreSuite.scala | 22 +- .../io/viash/e2e/build/NativeSuite.scala | 13 +- .../e2e/config_view/MainConfigViewSuite.scala | 12 +- .../io/viash/e2e/export/MainExportSuite.scala | 78 +++--- .../io/viash/e2e/help/MainHelpSuite.scala | 38 +-- .../e2e/ns_build/MainNSBuildNativeSuite.scala | 6 +- .../e2e/ns_exec/MainNSExecNativeSuite.scala | 13 +- .../e2e/ns_list/MainNSListNativeSuite.scala | 146 ++++++------ .../e2e/ns_test/MainNSTestNativeSuite.scala | 26 +- .../run/RunComputationalRequirements.scala | 54 ++--- .../viash/e2e/test/MainTestDockerSuite.scala | 84 +++---- .../viash/e2e/test/MainTestNativeSuite.scala | 224 +++++++++--------- .../test/TestComputationalRequirements.scala | 54 ++--- .../dependencies/Dependency.scala | 122 +++++++++- .../runners/nextflow/NextflowScriptTest.scala | 4 +- .../runners/nextflow/Vdsl3ModuleTest.scala | 8 +- .../nextflow/Vdsl3StandaloneTest.scala | 2 +- .../runners/nextflow/WorkflowHelperTest.scala | 2 +- 24 files changed, 565 insertions(+), 478 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 983bad0af..2e89bd247 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,12 @@ TODO add summary * `export json_schema`: Add a `--strict` option to output a subset of the schema representing the internal structure of the Viash config (PR #564). * `config view` and `ns list`: Do not output internal functionality fields (#564). Additionally, add a validation that no internal fields are present when reading a Viash config file. + +## MINOR CHANGES + +* `testbenches`: Add testbenches for local dependencies (PR #565). + +* `testbenches`: Refactor testbenches helper functions to uniformize them (PR #565). # Viash 0.8.0-RC5 (2023-10-11): Fix run workflow diff --git a/src/test/scala/io/viash/TestHelper.scala b/src/test/scala/io/viash/TestHelper.scala index b92d11cd5..4224a6669 100644 --- a/src/test/scala/io/viash/TestHelper.scala +++ b/src/test/scala/io/viash/TestHelper.scala @@ -10,94 +10,65 @@ import scala.reflect.ClassTag object TestHelper { - case class ExceptionOutput( - exceptionText: String, - output: String, - error: String, + case class TestMainOutput( + stdout: String, + stderr: String, + exitCode: Option[Int], + exceptionText: Option[String], ) /** - * Method to capture the console stdout generated by Main.main() so we can analyse what's being outputted to the console - * As the capture prevents the stdout being printed to the console, we print it after the Main.main() is finished. + * Method to capture the console stdout and stderr generated by Main.main() so we can analyse what's being outputted to the console * @param args all the arguments typically passed to Main.main() - * @return a string of all the output + * @return TestMainOutput containing the console output text and exit code */ - def testMain(args: String*) : String = { - val os = new ByteArrayOutputStream() - Console.withErr(os) { - Console.withOut(os) { - Main.mainCLI(args.toArray) - } - } - - val stdout = os.toString - // Console.print(stdout) - stdout - } + def testMain(args: String*): TestMainOutput = testMain(None, args: _*) /** * Method to capture the console stdout and stderr generated by Main.main() so we can analyse what's being outputted to the console - * As the capture prevents the stdout and stderr being printed to the console, we print it after the Main.main() is finished. + * @param workingDir the working directory to run the command in * @param args all the arguments typically passed to Main.main() - * @return a tuple of stdout and stderr strings of all the output + * @return TestMainOutput containing the console output text and exit code */ - def testMainWithStdErr(args: String*) : (String, String, Int) = { + def testMain(workingDir: Option[Path], args: String*): TestMainOutput = { val outStream = new ByteArrayOutputStream() val errStream = new ByteArrayOutputStream() val exitCode = Console.withOut(outStream) { Console.withErr(errStream) { - Main.mainCLI(args.toArray) + Main.mainCLI(args.toArray, workingDir) } } - val stdout = outStream.toString - val stderr = errStream.toString - // Console.print(stdout) - (stdout, stderr, exitCode) + TestMainOutput(outStream.toString, errStream.toString, Some(exitCode), None) } /** - * Method to capture the console stdout generated by Main.main() so we can analyse what's being outputted to the console - * As the capture prevents the stdout being printed to the console, we print it after the Main.main() is finished. - * Additionally it handles a thrown RuntimeException using assertThrows + * Method to capture the console stdout and stderr generated by Main.main() so we can analyse what's being outputted to the console. + * Additionally it handles a thrown Exception/Throwable and returns the exception text. * @param args all the arguments typically passed to Main.main() - * @return a string of all the output + * @return TestMainOutput containing the exception text and the console output text */ - def testMainException[T <: AnyRef: ClassTag](args: String*) : String = { - val os = new ByteArrayOutputStream() - assertThrows[T] { - Console.withOut(os) { - Main.mainCLI(args.toArray) - } - } - - val stdout = os.toString - // Console.print(stdout) - stdout - } + def testMainException[T <: Throwable](args: String*)(implicit classTag: ClassTag[T]) : TestMainOutput = testMainException(None, args: _*) /** - * Method to capture the console stdout generated by Main.main() so we can analyse what's being outputted to the console - * As the capture prevents the stdout being printed to the console, we print it after the Main.main() is finished. - * Additionally it handles a thrown RuntimeException using assertThrows + * Method to capture the console stdout and stderr generated by Main.main() so we can analyse what's being outputted to the console. + * Additionally it handles a thrown Exception/Throwable and returns the exception text. + * @param workingDir the working directory to run the command in * @param args all the arguments typically passed to Main.main() - * @return ExceptionOutput containing the exception text and the console output text + * @return TestMainOutput containing the exception text and the console output text */ - def testMainException2[T <: Exception](args: String*) : ExceptionOutput = { + def testMainException[T <: Throwable](workingDir: Option[Path], args: String*)(implicit classTag: ClassTag[T]) : TestMainOutput = { val outStream = new ByteArrayOutputStream() val errStream = new ByteArrayOutputStream() - val caught = intercept[Exception] { + val caught = intercept[T] { Console.withOut(outStream) { Console.withErr(errStream) { - Main.mainCLI(args.toArray) + Main.mainCLI(args.toArray, workingDir) } } } - val stdout = outStream.toString - val stderr = errStream.toString - // Console.print(stdout) - ExceptionOutput(caught.getMessage, stdout, stderr) + TestMainOutput(outStream.toString, errStream.toString, None, Some(caught.getMessage)) } diff --git a/src/test/scala/io/viash/auxiliary/MainBuildAuxiliaryDockerChown.scala b/src/test/scala/io/viash/auxiliary/MainBuildAuxiliaryDockerChown.scala index 426206efc..73749d4f8 100644 --- a/src/test/scala/io/viash/auxiliary/MainBuildAuxiliaryDockerChown.scala +++ b/src/test/scala/io/viash/auxiliary/MainBuildAuxiliaryDockerChown.scala @@ -116,13 +116,13 @@ class MainBuildAuxiliaryDockerChown extends AnyFunSuite with BeforeAndAfterAll w test("Test with platform and chown is set to false", DockerTest) { val newConfigFile = configDeriver.derive(""".platforms := [{type: "docker", chown: false }]""", "docker_chown_false") // functionality not provided in runner, should throw exception - val output = TestHelper.testMainException2[ConfigParserException]( + val output = TestHelper.testMainException[ConfigParserException]( "build", "--engine", "docker_chown", "-o", tempFolStr, newConfigFile ) - assert(output.error.contains("Error: ..chown was removed: Compability not provided with the Runners functionality.")) + assert(output.stderr.contains("Error: ..chown was removed: Compability not provided with the Runners functionality.")) } override def afterAll(): Unit = { diff --git a/src/test/scala/io/viash/auxiliary/MainBuildAuxiliaryDockerRequirements.scala b/src/test/scala/io/viash/auxiliary/MainBuildAuxiliaryDockerRequirements.scala index c7d16867d..3242a4d4d 100644 --- a/src/test/scala/io/viash/auxiliary/MainBuildAuxiliaryDockerRequirements.scala +++ b/src/test/scala/io/viash/auxiliary/MainBuildAuxiliaryDockerRequirements.scala @@ -568,29 +568,29 @@ class MainBuildAuxiliaryDockerRequirementsApkTest extends AbstractMainBuildAuxil test("test_setup; check the fortune package is added for the test option", DockerTest) { f => val newConfigFilePath = deriveEngineConfig(None, Some("""[{ "type": "apk", "packages": ["fortune"] }]"""), "apk_test_fortune_test") - val testText = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", newConfigFilePath ) - assert(testText.contains("Running tests in temporary directory: ")) - assert(testText.contains("SUCCESS! All 1 out of 1 test scripts succeeded!")) - assert(testText.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("SUCCESS! All 1 out of 1 test scripts succeeded!")) + assert(testOutput.stdout.contains("Cleaning up temporary directory")) } test("test_setup; check the fortune package is not added for the test option when not specified", DockerTest) { f => val newConfigFilePath = deriveEngineConfig(None, None, "apk_base_test") - val testOutput = TestHelper.testMainException2[RuntimeException]( + val testOutput = TestHelper.testMainException[RuntimeException]( "test", "-k", "false", newConfigFilePath ) - assert(testOutput.exceptionText == "Only 0 out of 1 test scripts succeeded!") + assert(testOutput.exceptionText.get == "Only 0 out of 1 test scripts succeeded!") - assert(testOutput.output.contains("Running tests in temporary directory: ")) - assert(testOutput.output.contains("ERROR! Only 0 out of 1 test scripts succeeded!")) - assert(testOutput.output.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("ERROR! Only 0 out of 1 test scripts succeeded!")) + assert(testOutput.stdout.contains("Cleaning up temporary directory")) } } diff --git a/src/test/scala/io/viash/auxiliary/MainBuildAuxiliaryDockerResourceCopying.scala b/src/test/scala/io/viash/auxiliary/MainBuildAuxiliaryDockerResourceCopying.scala index ff17f663e..98690930c 100644 --- a/src/test/scala/io/viash/auxiliary/MainBuildAuxiliaryDockerResourceCopying.scala +++ b/src/test/scala/io/viash/auxiliary/MainBuildAuxiliaryDockerResourceCopying.scala @@ -77,14 +77,14 @@ class MainBuildAuxiliaryDockerResourceCopying extends AnyFunSuite with BeforeAnd test("Check resources with unsupported format") { val configResourcesUnsupportedProtocolFile = configDeriver.derive(""".functionality.resources := [{type: "bash_script", path: "./check_bash_version.sh"}, {path: "ftp://ftp.ubuntu.com/releases/robots.txt"}]""", "config_resource_unsupported_protocol").toString // generate viash script - val testOutput = TestHelper.testMainException2[RuntimeException]( + val testOutput = TestHelper.testMainException[RuntimeException]( "build", "--engine", "docker", "-o", tempFolStr, configResourcesUnsupportedProtocolFile ) - assert(testOutput.exceptionText == "Unsupported scheme: ftp") + assert(testOutput.exceptionText.get == "Unsupported scheme: ftp") } override def afterAll(): Unit = { diff --git a/src/test/scala/io/viash/auxiliary/MainTestAuxiliaryDockerResourceCopy.scala b/src/test/scala/io/viash/auxiliary/MainTestAuxiliaryDockerResourceCopy.scala index d968c251a..e9e6228ef 100644 --- a/src/test/scala/io/viash/auxiliary/MainTestAuxiliaryDockerResourceCopy.scala +++ b/src/test/scala/io/viash/auxiliary/MainTestAuxiliaryDockerResourceCopy.scala @@ -29,7 +29,7 @@ class MainTestAuxiliaryDockerResourceCopy extends AnyFunSuite with BeforeAndAfte Files.copy(tmpFolderResourceSourceFile, tmpFolderResourceDestinationFile, StandardCopyOption.REPLACE_EXISTING) // generate viash script - val testText = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", "--engine", "docker", "-k", "true", @@ -37,13 +37,13 @@ class MainTestAuxiliaryDockerResourceCopy extends AnyFunSuite with BeforeAndAfte ) // basic checks to see if standard test/build was correct - assert(testText.contains("Running tests in temporary directory: ")) - assert(testText.contains("WARNING! No tests found!")) - assert(!testText.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("WARNING! No tests found!")) + assert(!testOutput.stdout.contains("Cleaning up temporary directory")) val FolderRegex = ".*Running tests in temporary directory: '([^']*)'.*".r - val tempPath = testText.replaceAll("\n", "") match { + val tempPath = testOutput.stdout.replaceAll("\n", "") match { case FolderRegex(path) => path case _ => "" } @@ -77,27 +77,27 @@ class MainTestAuxiliaryDockerResourceCopy extends AnyFunSuite with BeforeAndAfte } Directory(tmpFolderResourceDestinationFolder).deleteRecursively() - checkTempDirAndRemove(testText, true, "viash_test_auxiliary_resources") + checkTempDirAndRemove(testOutput.stdout, true, "viash_test_auxiliary_resources") } test("Check resources with unsupported format", DockerTest) { val configResourcesUnsupportedProtocolFile = configDeriver.derive(""".functionality.resources := [{type: "bash_script", path: "./check_bash_version.sh"}, {path: "ftp://ftp.ubuntu.com/releases/robots.txt"}]""", "config_resource_unsupported_protocol").toString // generate viash script - val testOutput = TestHelper.testMainException2[RuntimeException]( + val testOutput = TestHelper.testMainException[RuntimeException]( "test", "--engine", "docker", "-k", "true", configResourcesUnsupportedProtocolFile ) - assert(testOutput.exceptionText == "Unsupported scheme: ftp") + assert(testOutput.exceptionText.get == "Unsupported scheme: ftp") // basic checks to see if standard test/build was correct - assert(testOutput.output.contains("Running tests in temporary directory: ")) - assert(!testOutput.output.contains("WARNING! No tests found!")) - assert(!testOutput.output.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(!testOutput.stdout.contains("WARNING! No tests found!")) + assert(!testOutput.stdout.contains("Cleaning up temporary directory")) - checkTempDirAndRemove(testOutput.output, true, "viash_test_auxiliary_resources") + checkTempDirAndRemove(testOutput.stdout, true, "viash_test_auxiliary_resources") } /** diff --git a/src/test/scala/io/viash/e2e/build/DockerMoreSuite.scala b/src/test/scala/io/viash/e2e/build/DockerMoreSuite.scala index ff9e505cd..de40089fa 100644 --- a/src/test/scala/io/viash/e2e/build/DockerMoreSuite.scala +++ b/src/test/scala/io/viash/e2e/build/DockerMoreSuite.scala @@ -36,7 +36,7 @@ class DockerMoreSuite extends AnyFunSuite with BeforeAndAfterAll { "commands_default" ) - val (stdout, _, _) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "build", "--engine", "docker", "--runner", "docker", @@ -45,7 +45,7 @@ class DockerMoreSuite extends AnyFunSuite with BeforeAndAfterAll { "--setup", "alwaysbuild" ) - assert(stdout.matches("\\[notice\\] Building container 'testbash:0\\.1' with Dockerfile\\s*"), stdout) + assert(testOutput.stdout.matches("\\[notice\\] Building container 'testbash:0\\.1' with Dockerfile\\s*"), testOutput.stdout) } test("Verify adding extra commands to verify", DockerTest) { @@ -54,7 +54,7 @@ class DockerMoreSuite extends AnyFunSuite with BeforeAndAfterAll { "commands_extra" ) - val (stdout, _, _) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "build", "--engine", "docker", "--runner", "docker", @@ -63,7 +63,7 @@ class DockerMoreSuite extends AnyFunSuite with BeforeAndAfterAll { "--setup", "alwaysbuild" ) - assert(stdout.matches("\\[notice\\] Building container 'testbash:0\\.1' with Dockerfile\\s*"), stdout) + assert(testOutput.stdout.matches("\\[notice\\] Building container 'testbash:0\\.1' with Dockerfile\\s*"), testOutput.stdout) } test("Verify base adding an extra required command that doesn't exist", DockerTest) { @@ -72,7 +72,7 @@ class DockerMoreSuite extends AnyFunSuite with BeforeAndAfterAll { "non_existing_command" ) - val stdout = TestHelper.testMain( + val testOutput = TestHelper.testMain( "build", "--engine", "docker", "--runner", "docker", @@ -81,14 +81,14 @@ class DockerMoreSuite extends AnyFunSuite with BeforeAndAfterAll { "--setup", "alwaysbuild" ) - assert(stdout.contains("[notice] Building container 'testbash:0.1' with Dockerfile")) - assert(stdout.contains("[error] Docker container 'testbash:0.1' does not contain command 'non_existing_command'.")) + assert(testOutput.stdout.contains("[notice] Building container 'testbash:0.1' with Dockerfile")) + assert(testOutput.stdout.contains("[error] Docker container 'testbash:0.1' does not contain command 'non_existing_command'.")) } test("Check deprecated warning", DockerTest) { val newConfigFilePath = configDeriver.derive(""".functionality.status := "deprecated"""", "deprecated") - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "build", "--engine", "docker", "--runner", "docker", @@ -97,8 +97,8 @@ class DockerMoreSuite extends AnyFunSuite with BeforeAndAfterAll { "--setup", "alwaysbuild" ) - assert(stderr.contains("The status of the component 'testbash' is set to deprecated.")) - assert(exitCode == 0) + assert(testOutput.stderr.contains("The status of the component 'testbash' is set to deprecated.")) + assert(testOutput.exitCode == Some(0)) } test("Check component works when multiple_sep is set to ;", DockerTest) { @@ -107,7 +107,7 @@ class DockerMoreSuite extends AnyFunSuite with BeforeAndAfterAll { "multiple_sep" ) - val _ = TestHelper.testMainWithStdErr( + val _ = TestHelper.testMain( "build", "--engine", "docker", "--runner", "docker", diff --git a/src/test/scala/io/viash/e2e/build/NativeSuite.scala b/src/test/scala/io/viash/e2e/build/NativeSuite.scala index c0521919b..bc0e34d9a 100644 --- a/src/test/scala/io/viash/e2e/build/NativeSuite.scala +++ b/src/test/scala/io/viash/e2e/build/NativeSuite.scala @@ -10,6 +10,7 @@ import io.viash.config.Config import scala.io.Source import io.viash.helpers.{IO, Exec, Logger} +import io.viash.exceptions.ConfigParserException class NativeSuite extends AnyFunSuite with BeforeAndAfterAll { Logger.UseColorOverride.value = Some(false) @@ -222,11 +223,11 @@ class NativeSuite extends AnyFunSuite with BeforeAndAfterAll { ) val testRegex = "Warning: ..platforms is deprecated: Use 'engines' and 'runners' instead.".r - assert(testRegex.findFirstIn(testOutput).isDefined, testOutput) + assert(testRegex.findFirstIn(testOutput.stderr).isDefined, testOutput) } test("Test whether defining strings as arguments in argument groups throws a removed error") { - val testOutput = TestHelper.testMainException2[Exception]( + val testOutput = TestHelper.testMainException[Exception]( "build", "-o", tempFolStr, configDeprecatedArgumentGroups @@ -241,19 +242,19 @@ class NativeSuite extends AnyFunSuite with BeforeAndAfterAll { assert(out.exitValue == 0) val testRegex = "Error: specifying strings in the .argument field of argument group 'First group' was removed.".r - assert(testRegex.findFirstIn(testOutput.error).isDefined, testOutput.error) + assert(testRegex.findFirstIn(testOutput.stderr).isDefined, testOutput.stderr) } test("Test whether setting an internalFunctionality field throws an error") { val newConfigFilePath = configDeriver.derive(""".functionality.argument_groups[.name == "First group"].arguments[.name == "input"].dest := "foo"""", "set_internal_functionality") - val testOutput = TestHelper.testMainException2[RuntimeException]( + val testOutput = TestHelper.testMainException[ConfigParserException]( "build", "-o", tempFolStr, newConfigFilePath ) - assert(testOutput.error.contains("Error: .functionality.argument_groups.arguments.dest is internal functionality.")) + assert(testOutput.stderr.contains("Error: .functionality.argument_groups.arguments.dest is internal functionality.")) } test("Test config without a main script") { @@ -264,7 +265,7 @@ class NativeSuite extends AnyFunSuite with BeforeAndAfterAll { "-c", ".functionality.resources := []" ) - assert(testOutput.contains("Warning: no resources specified!")) + assert(testOutput.stderr.contains("Warning: no resources specified!")) } override def afterAll(): Unit = { diff --git a/src/test/scala/io/viash/e2e/config_view/MainConfigViewSuite.scala b/src/test/scala/io/viash/e2e/config_view/MainConfigViewSuite.scala index ba31035c0..65a1d4a83 100644 --- a/src/test/scala/io/viash/e2e/config_view/MainConfigViewSuite.scala +++ b/src/test/scala/io/viash/e2e/config_view/MainConfigViewSuite.scala @@ -13,23 +13,23 @@ class MainConfigViewSuite extends AnyFunSuite{ test("viash config view local") { - val (stdout, _, _) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "config", "view", configFile ) - assert(stdout.startsWith("functionality:")) - assert(stdout.contains("testbash")) + assert(testOutput.stdout.startsWith("functionality:")) + assert(testOutput.stdout.contains("testbash")) } test("viash config view remote") { - val (stdout, _, _) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "config", "view", "https://raw.githubusercontent.com/viash-io/viash/develop/src/test/resources/testbash/config.vsh.yaml" ) - assert(stdout.startsWith("functionality:")) - assert(stdout.contains("testbash")) + assert(testOutput.stdout.startsWith("functionality:")) + assert(testOutput.stdout.contains("testbash")) } } diff --git a/src/test/scala/io/viash/e2e/export/MainExportSuite.scala b/src/test/scala/io/viash/e2e/export/MainExportSuite.scala index 04f073df8..fb1ffedca 100644 --- a/src/test/scala/io/viash/e2e/export/MainExportSuite.scala +++ b/src/test/scala/io/viash/e2e/export/MainExportSuite.scala @@ -23,15 +23,15 @@ class MainExportSuite extends AnyFunSuite with BeforeAndAfter { // These are all *very* basic tests. Practicly no validation whatsoever to check whether the output is correct or not. test("viash export resource") { - val stdout = TestHelper.testMain( + val testOutput = TestHelper.testMain( "export", "resource", "runners/nextflow/WorkflowHelper.nf" ) - assert(stdout.contains("def readConfig(")) + assert(testOutput.stdout.contains("def readConfig(")) } test("viash export resource to file") { - val stdout = TestHelper.testMain( + TestHelper.testMain( "export", "resource", "runners/nextflow/WorkflowHelper.nf", "--output", tempFile.toString ) @@ -41,38 +41,38 @@ class MainExportSuite extends AnyFunSuite with BeforeAndAfter { } test("viash export resource legacy") { - val (stdout, stderr, code) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "export", "resource", "platforms/nextflow/WorkflowHelper.nf" ) - assert(stderr.contains("WARNING: The 'platforms/' prefix is deprecated. Please use 'runners/' instead.")) + assert(testOutput.stderr.contains("WARNING: The 'platforms/' prefix is deprecated. Please use 'runners/' instead.")) - assert(stdout.contains("def readConfig(")) + assert(testOutput.stdout.contains("def readConfig(")) } test("viash export resource to file legacy") { - val (stdout, stderr, code) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "export", "resource", "platforms/nextflow/WorkflowHelper.nf", "--output", tempFile.toString ) - assert(stderr.contains("WARNING: The 'platforms/' prefix is deprecated. Please use 'runners/' instead.")) + assert(testOutput.stderr.contains("WARNING: The 'platforms/' prefix is deprecated. Please use 'runners/' instead.")) val lines = helpers.IO.read(tempFile.toUri()) assert(lines.contains("def readConfig(")) } test("viash export cli_schema") { - val stdout = TestHelper.testMain( + val testOutput = TestHelper.testMain( "export", "cli_schema" ) - assert(stdout.startsWith("""- name: "run"""")) - assert(stdout.contains("viash config inject")) + assert(testOutput.stdout.startsWith("""- name: "run"""")) + assert(testOutput.stdout.contains("viash config inject")) } test("viash export cli_schema to file") { - val stdout = TestHelper.testMain( + TestHelper.testMain( "export", "cli_schema", "--output", tempFile.toString ) @@ -83,16 +83,16 @@ class MainExportSuite extends AnyFunSuite with BeforeAndAfter { } test("viash export cli_autocomplete without format") { - val stdout = TestHelper.testMain( + val testOutput = TestHelper.testMain( "export", "cli_autocomplete" ) - assert(stdout.startsWith("""# bash completion for viash""")) - assert(stdout.contains("COMPREPLY=($(compgen -W 'run build test ns config' -- \"$cur\"))")) + assert(testOutput.stdout.startsWith("""# bash completion for viash""")) + assert(testOutput.stdout.contains("COMPREPLY=($(compgen -W 'run build test ns config' -- \"$cur\"))")) } test("viash export cli_autocomplete without format to file") { - val stdout = TestHelper.testMain( + TestHelper.testMain( "export", "cli_autocomplete", "--output", tempFile.toString ) @@ -103,17 +103,17 @@ class MainExportSuite extends AnyFunSuite with BeforeAndAfter { } test("viash export cli_autocomplete Bash") { - val stdout = TestHelper.testMain( + val testOutput = TestHelper.testMain( "export", "cli_autocomplete", "--format", "bash" ) - assert(stdout.startsWith("""# bash completion for viash""")) - assert(stdout.contains("COMPREPLY=($(compgen -W 'run build test ns config' -- \"$cur\"))")) + assert(testOutput.stdout.startsWith("""# bash completion for viash""")) + assert(testOutput.stdout.contains("COMPREPLY=($(compgen -W 'run build test ns config' -- \"$cur\"))")) } test("viash export cli_autocomplete Bash to file") { - val stdout = TestHelper.testMain( + TestHelper.testMain( "export", "cli_autocomplete", "--format", "bash", "--output", tempFile.toString @@ -125,17 +125,17 @@ class MainExportSuite extends AnyFunSuite with BeforeAndAfter { } test("viash export cli_autocomplete Zsh") { - val stdout = TestHelper.testMain( + val testOutput = TestHelper.testMain( "export", "cli_autocomplete", "--format", "zsh" ) - assert(stdout.startsWith("""#compdef viash""")) - assert(stdout.contains("_viash_export_commands")) + assert(testOutput.stdout.startsWith("""#compdef viash""")) + assert(testOutput.stdout.contains("_viash_export_commands")) } test("viash export cli_autocomplete Zsh to file") { - val stdout = TestHelper.testMain( + TestHelper.testMain( "export", "cli_autocomplete", "--format", "zsh", "--output", tempFile.toString @@ -147,16 +147,16 @@ class MainExportSuite extends AnyFunSuite with BeforeAndAfter { } test("viash export config_schema") { - val stdout = TestHelper.testMain( + val testOutput = TestHelper.testMain( "export", "config_schema" ) - assert(stdout.startsWith("""- - name: "__this__"""")) - assert(stdout.contains("""type: "OneOrMore[String]"""")) + assert(testOutput.stdout.startsWith("""- - name: "__this__"""")) + assert(testOutput.stdout.contains("""type: "OneOrMore[String]"""")) } test("viash export config_schema to file") { - val stdout = TestHelper.testMain( + TestHelper.testMain( "export", "config_schema", "--output", tempFile.toString ) @@ -167,25 +167,25 @@ class MainExportSuite extends AnyFunSuite with BeforeAndAfter { } test("viash export json_schema") { - val stdout = TestHelper.testMain( + val testOutput = TestHelper.testMain( "export", "json_schema" ) - assert(stdout.startsWith("""$schema: "https://json-schema.org/draft-07/schema#"""")) - assert(stdout.contains("""- $ref: "#/definitions/Config"""")) + assert(testOutput.stdout.startsWith("""$schema: "https://json-schema.org/draft-07/schema#"""")) + assert(testOutput.stdout.contains("""- $ref: "#/definitions/Config"""")) } test("viash export json_schema, explicit yaml format") { - val stdout = TestHelper.testMain( + val testOutput = TestHelper.testMain( "export", "json_schema", "--format", "yaml" ) - assert(stdout.startsWith("""$schema: "https://json-schema.org/draft-07/schema#"""")) - assert(stdout.contains("""- $ref: "#/definitions/Config"""")) + assert(testOutput.stdout.startsWith("""$schema: "https://json-schema.org/draft-07/schema#"""")) + assert(testOutput.stdout.contains("""- $ref: "#/definitions/Config"""")) } test("viash export json_schema to file, explicit yaml format") { - val stdout = TestHelper.testMain( + TestHelper.testMain( "export", "json_schema", "--format", "yaml", "--output", tempFile.toString ) @@ -196,20 +196,20 @@ class MainExportSuite extends AnyFunSuite with BeforeAndAfter { } test("viash export json_schema, json format") { - val stdout = TestHelper.testMain( + val testOutput = TestHelper.testMain( "export", "json_schema", "--format", "json" ) - assert(stdout.startsWith( + assert(testOutput.stdout.startsWith( """{ | "$schema" : "https://json-schema.org/draft-07/schema#", | "definitions" : { |""".stripMargin)) - assert(stdout.contains(""""$ref" : "#/definitions/Config"""")) + assert(testOutput.stdout.contains(""""$ref" : "#/definitions/Config"""")) } test("viash export json_schema to file, json format") { - val stdout = TestHelper.testMain( + TestHelper.testMain( "export", "json_schema", "--format", "json", "--output", tempFile.toString ) diff --git a/src/test/scala/io/viash/e2e/help/MainHelpSuite.scala b/src/test/scala/io/viash/e2e/help/MainHelpSuite.scala index a02c36f0a..d68012429 100644 --- a/src/test/scala/io/viash/e2e/help/MainHelpSuite.scala +++ b/src/test/scala/io/viash/e2e/help/MainHelpSuite.scala @@ -11,69 +11,69 @@ class MainHelpSuite extends AnyFunSuite{ private val configFile = getClass.getResource(s"/testbash/config.vsh.yaml").getPath test("viash config view default functionality without help") { - val (stdout, _, _) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "config", "view", configFile ) - assert(stdout.startsWith("functionality:")) - assert(stdout.contains("testbash")) + assert(testOutput.stdout.startsWith("functionality:")) + assert(testOutput.stdout.contains("testbash")) } test("viash config view default functionality leading help") { - val output = TestHelper.testMainException[ExitException]( + val testOutput = TestHelper.testMainException[ExitException]( "config", "view", "--help" ) - assert(output.startsWith("viash config view")) - assert(!output.contains("testbash")) + assert(testOutput.stdout.startsWith("viash config view")) + assert(!testOutput.stdout.contains("testbash")) } test("viash config view default functionality trailing help") { - val output = TestHelper.testMainException[ExitException]( + val testOutput = TestHelper.testMainException[ExitException]( "config", "view", configFile, "--help" ) - assert(output.startsWith("viash config view")) - assert(!output.contains("testbash")) + assert(testOutput.stdout.startsWith("viash config view")) + assert(!testOutput.stdout.contains("testbash")) } test("viash config view default functionality trailing help after platform argument") { - val output = TestHelper.testMainException[ExitException]( + val testOutput = TestHelper.testMainException[ExitException]( "config", "view", configFile, - "--platform", "native", + "--runner", "native", "--help" ) - assert(output.startsWith("viash config view")) - assert(!output.contains("testbash")) + assert(testOutput.stdout.startsWith("viash config view")) + assert(!testOutput.stdout.contains("testbash")) } test("viash config view default functionality trailing help before platform argument") { - val output = TestHelper.testMainException[ExitException]( + val testOutput = TestHelper.testMainException[ExitException]( "config", "view", configFile, "--help", - "--platform", "native" + "--runner", "native" ) - assert(output.startsWith("viash config view")) - assert(!output.contains("testbash")) + assert(testOutput.stdout.startsWith("viash config view")) + assert(!testOutput.stdout.contains("testbash")) } test("viash config view default functionality with --help as runner argument") { - val output = TestHelper.testMainException[RuntimeException]( + val testOutput = TestHelper.testMainException[RuntimeException]( "config", "view", configFile, "--runner", "--help" ) - assert(!output.contains("viash config view")) + assert(!testOutput.stdout.contains("viash config view")) } } diff --git a/src/test/scala/io/viash/e2e/ns_build/MainNSBuildNativeSuite.scala b/src/test/scala/io/viash/e2e/ns_build/MainNSBuildNativeSuite.scala index 17a50ba01..84911f228 100644 --- a/src/test/scala/io/viash/e2e/ns_build/MainNSBuildNativeSuite.scala +++ b/src/test/scala/io/viash/e2e/ns_build/MainNSBuildNativeSuite.scala @@ -34,7 +34,7 @@ class MainNSBuildNativeSuite extends AnyFunSuite with BeforeAndAfterAll{ // convert testbash test("viash ns can build") { - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "build", "-s", nsPath, "-t", tempFolStr @@ -42,7 +42,7 @@ class MainNSBuildNativeSuite extends AnyFunSuite with BeforeAndAfterAll{ assert(nsFolder.exists) assert(nsFolder.isDirectory) - assert(exitCode == 1) + assert(testOutput.exitCode == Some(1)) for ((component, _, _, _) <- components) { val executable = componentExecutableFile(component) @@ -51,7 +51,7 @@ class MainNSBuildNativeSuite extends AnyFunSuite with BeforeAndAfterAll{ } val regexBuildError = raw"Reading file \'.*/src/ns_error/config\.vsh\.yaml\' failed".r - assert(regexBuildError.findFirstIn(stderr).isDefined, "Expecting to get an error because of an invalid yaml in ns_error") + assert(regexBuildError.findFirstIn(testOutput.stderr).isDefined, "Expecting to get an error because of an invalid yaml in ns_error") } test("Check whether the executable can run") { diff --git a/src/test/scala/io/viash/e2e/ns_exec/MainNSExecNativeSuite.scala b/src/test/scala/io/viash/e2e/ns_exec/MainNSExecNativeSuite.scala index 946606b6c..21f0263ea 100644 --- a/src/test/scala/io/viash/e2e/ns_exec/MainNSExecNativeSuite.scala +++ b/src/test/scala/io/viash/e2e/ns_exec/MainNSExecNativeSuite.scala @@ -27,16 +27,15 @@ class MainNSExecNativeSuite extends AnyFunSuite with BeforeAndAfterAll { private val tempFolStr = temporaryFolder.toString test("Check whether ns exec \\; works") { - val (stdoutRaw, stderrRaw, _) = - TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "exec", "--src", nsPath, "--apply_runner", "--apply_engine", "echo _{functionality-name}_ -{dir}- !{path}! ~{engine}~ ={namespace}=+\\;" ) - val stdout = stdoutRaw.replaceAll(nsPath, "src/") - val stderr = stderrRaw.replaceAll(nsPath, "src/") + val stdout = testOutput.stdout.replaceAll(nsPath, "src/") + val stderr = testOutput.stderr.replaceAll(nsPath, "src/") for (component <- components) { val regexCommand = s"""\\+ echo _${component}_ -src/$component/?- !src/$component/config.vsh.yaml! ~native~ =testns=""".r @@ -47,13 +46,13 @@ class MainNSExecNativeSuite extends AnyFunSuite with BeforeAndAfterAll { } test("Check whether ns exec + works") { - val (stdoutRaw, stderrRaw, _) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "exec", "--src", nsPath, "echo {path} +" ) - val stdout = stdoutRaw.replaceAll(nsPath, "src/") - val stderr = stderrRaw.replaceAll(nsPath, "src/") + val stdout = testOutput.stdout.replaceAll(nsPath, "src/") + val stderr = testOutput.stderr.replaceAll(nsPath, "src/") // can't guarantee order of components val regexCommand = s"""\\+ echo src/[^/]*/config.vsh.yaml src/[^/]*/config.vsh.yaml src/[^/]*/config.vsh.yaml src/[^/]*/config.vsh.yaml""".r diff --git a/src/test/scala/io/viash/e2e/ns_list/MainNSListNativeSuite.scala b/src/test/scala/io/viash/e2e/ns_list/MainNSListNativeSuite.scala index 5afbeeb7b..47a1b49cd 100644 --- a/src/test/scala/io/viash/e2e/ns_list/MainNSListNativeSuite.scala +++ b/src/test/scala/io/viash/e2e/ns_list/MainNSListNativeSuite.scala @@ -29,22 +29,22 @@ class MainNSListNativeSuite extends AnyFunSuite{ // convert testbash test("viash ns list") { - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "list", "-s", nsPath, ) - assert(exitCode == 1) + assert(testOutput.exitCode == Some(1)) for (component <- components) { val regexName = raw"""name:\s+"$component"""" - assert(regexName.r.findFirstIn(stdout).isDefined, s"\nRegex: ${regexName}; text: \n$stdout") + assert(regexName.r.findFirstIn(testOutput.stdout).isDefined, s"\nRegex: ${regexName}; text: \n${testOutput.stdout}") } val regexBuildError = raw"Reading file \'.*/src/ns_error/config\.vsh\.yaml\' failed" - assert(regexBuildError.r.findFirstIn(stderr).isDefined, "Expecting to get an error because of an invalid yaml in ns_error") + assert(regexBuildError.r.findFirstIn(testOutput.stderr).isDefined, "Expecting to get an error because of an invalid yaml in ns_error") - val stdout2 = s"(?s)(\u001b.{4})?((Not all configs parsed successfully)|(All \\d+ configs parsed successfully)).*$$".r.replaceAllIn(stdout, "") + val stdout2 = s"(?s)(\u001b.{4})?((Not all configs parsed successfully)|(All \\d+ configs parsed successfully)).*$$".r.replaceAllIn(testOutput.stdout, "") val config = parser.parse(stdout2) .fold(throw _, _.as[Array[Config]]) @@ -59,7 +59,7 @@ class MainNSListNativeSuite extends AnyFunSuite{ ) for ((regex, count) <- samples) { - assert(regex.r.findAllMatchIn(stdout).size == count, s"Expecting $count hits on stdout of regex [$regex]") + assert(regex.r.findAllMatchIn(testOutput.stdout).size == count, s"Expecting $count hits on stdout of regex [$regex]") } } @@ -67,44 +67,44 @@ class MainNSListNativeSuite extends AnyFunSuite{ // convert testbash test("viash ns list filter by engine and runner") { - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "list", "-s", nsPath, "--engine", "docker", "--runner", "docker" ) - assert(exitCode == 1) - val configs = parser.parse(stdout) + assert(testOutput.exitCode == Some(1)) + val configs = parser.parse(testOutput.stdout) .fold(throw _, _.as[Array[Config]]) .fold(throw _, identity) assert(configs.length == 0) } test("viash ns list filter by engine #2") { - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "list", "-s", scalaPath, "--engine", "docker", "--runner", "executable" ) - assert(exitCode == 0) + assert(testOutput.exitCode == Some(0)) - val configs = parser.parse(stdout) + val configs = parser.parse(testOutput.stdout) .fold(throw _, _.as[Array[Config]]) .fold(throw _, identity) assert(configs.length == 1) } test("viash ns list filter by engine #3") { - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "list", "-s", scalaPath, "--engine", "not_exists", "--runner", "not_exists" ) - assert(exitCode == 0) - val configs = parser.parse(stdout) + assert(testOutput.exitCode == Some(0)) + val configs = parser.parse(testOutput.stdout) .fold(throw _, _.as[Array[Config]]) .fold(throw _, identity) assert(configs.length == 0) @@ -112,200 +112,200 @@ class MainNSListNativeSuite extends AnyFunSuite{ // test query_name test("viash ns list query_name") { - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "list", "-s", nsPath, "--query_name", "ns_add" ) - assert(exitCode == 1) - val configs = parser.parse(stdout) + assert(testOutput.exitCode == Some(1)) + val configs = parser.parse(testOutput.stdout) .fold(throw _, _.as[Array[Config]]) .fold(throw _, identity) assert(configs.length == 1) - assert(stdout.contains("name: \"ns_add\"")) + assert(testOutput.stdout.contains("name: \"ns_add\"")) } test("viash ns list query_name full match") { - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "list", "-s", nsPath, "--query_name", "^ns_add$" ) - assert(exitCode == 1) - val configs = parser.parse(stdout) + assert(testOutput.exitCode == Some(1)) + val configs = parser.parse(testOutput.stdout) .fold(throw _, _.as[Array[Config]]) .fold(throw _, identity) assert(configs.length == 1) - assert(stdout.contains("name: \"ns_add\"")) + assert(testOutput.stdout.contains("name: \"ns_add\"")) } test("viash ns list query_name partial match") { - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "list", "-s", nsPath, "--query_name", "add" ) - assert(exitCode == 1) - val configs = parser.parse(stdout) + assert(testOutput.exitCode == Some(1)) + val configs = parser.parse(testOutput.stdout) .fold(throw _, _.as[Array[Config]]) .fold(throw _, identity) assert(configs.length == 1) - assert(stdout.contains("name: \"ns_add\"")) + assert(testOutput.stdout.contains("name: \"ns_add\"")) } test("viash ns list query_name no match") { - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "list", "-s", nsPath, "--query_name", "foo" ) - assert(exitCode == 1) - assert(stdout.trim() == "[]") + assert(testOutput.exitCode == Some(1)) + assert(testOutput.stdout.trim() == "[]") } // test query test("viash ns list query") { - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "list", "-s", nsPath, "-q", "testns/ns_add" ) - assert(exitCode == 1) - val configs = parser.parse(stdout) + assert(testOutput.exitCode == Some(1)) + val configs = parser.parse(testOutput.stdout) .fold(throw _, _.as[Array[Config]]) .fold(throw _, identity) assert(configs.length == 1) - assert(stdout.contains("name: \"ns_add\"")) + assert(testOutput.stdout.contains("name: \"ns_add\"")) } test("viash ns list query full match") { - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "list", "-s", nsPath, "-q", "^testns/ns_add$" ) - assert(exitCode == 1) - val configs = parser.parse(stdout) + assert(testOutput.exitCode == Some(1)) + val configs = parser.parse(testOutput.stdout) .fold(throw _, _.as[Array[Config]]) .fold(throw _, identity) assert(configs.length == 1) - assert(stdout.contains("name: \"ns_add\"")) + assert(testOutput.stdout.contains("name: \"ns_add\"")) } test("viash ns list query partial match") { - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "list", "-s", nsPath, "-q", "test.*/.*add" ) - assert(exitCode == 1) - val configs = parser.parse(stdout) + assert(testOutput.exitCode == Some(1)) + val configs = parser.parse(testOutput.stdout) .fold(throw _, _.as[Array[Config]]) .fold(throw _, identity) assert(configs.length == 1) - assert(stdout.contains("name: \"ns_add\"")) + assert(testOutput.stdout.contains("name: \"ns_add\"")) } test("viash ns list query only partial name") { - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "list", "-s", nsPath, "-q", "add" ) - assert(exitCode == 1) - val configs = parser.parse(stdout) + assert(testOutput.exitCode == Some(1)) + val configs = parser.parse(testOutput.stdout) .fold(throw _, _.as[Array[Config]]) .fold(throw _, identity) assert(configs.length == 1) - assert(stdout.contains("name: \"ns_add\"")) + assert(testOutput.stdout.contains("name: \"ns_add\"")) } test("viash ns list query no match") { - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "list", "-s", nsPath, "-q", "foo" ) - assert(exitCode == 1) - assert(stdout.trim() == "[]") + assert(testOutput.exitCode == Some(1)) + assert(testOutput.stdout.trim() == "[]") } // test query_namespace test("viash ns list query_namespace") { - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "list", "-s", nsPath, "--query_namespace", "testns" ) - assert(exitCode == 1) - val configs = parser.parse(stdout) + assert(testOutput.exitCode == Some(1)) + val configs = parser.parse(testOutput.stdout) .fold(throw _, _.as[Array[Config]]) .fold(throw _, identity) assert(configs.length == components.length) - assert(stdout.contains("name: \"ns_add\"")) - assert(stdout.contains("name: \"ns_subtract\"")) - assert(!stdout.contains("name: \"ns_error\"")) - assert(!stdout.contains("name: \"ns_disabled\"")) + assert(testOutput.stdout.contains("name: \"ns_add\"")) + assert(testOutput.stdout.contains("name: \"ns_subtract\"")) + assert(!testOutput.stdout.contains("name: \"ns_error\"")) + assert(!testOutput.stdout.contains("name: \"ns_disabled\"")) } test("viash ns list query_namespace full match") { - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "list", "-s", nsPath, "--query_namespace", "^testns$" ) - assert(exitCode == 1) - val configs = parser.parse(stdout) + assert(testOutput.exitCode == Some(1)) + val configs = parser.parse(testOutput.stdout) .fold(throw _, _.as[Array[Config]]) .fold(throw _, identity) assert(configs.length == components.length) - assert(stdout.contains("name: \"ns_add\"")) - assert(stdout.contains("name: \"ns_subtract\"")) - assert(!stdout.contains("name: \"ns_error\"")) - assert(!stdout.contains("name: \"ns_disabled\"")) + assert(testOutput.stdout.contains("name: \"ns_add\"")) + assert(testOutput.stdout.contains("name: \"ns_subtract\"")) + assert(!testOutput.stdout.contains("name: \"ns_error\"")) + assert(!testOutput.stdout.contains("name: \"ns_disabled\"")) } test("viash ns list query_namespace partial match") { - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "list", "-s", nsPath, "--query_namespace", "test" ) - assert(exitCode == 1) - val configs = parser.parse(stdout) + assert(testOutput.exitCode == Some(1)) + val configs = parser.parse(testOutput.stdout) .fold(throw _, _.as[Array[Config]]) .fold(throw _, identity) assert(configs.length == components.length) - assert(stdout.contains("name: \"ns_add\"")) - assert(stdout.contains("name: \"ns_subtract\"")) - assert(!stdout.contains("name: \"ns_error\"")) - assert(!stdout.contains("name: \"ns_disabled\"")) + assert(testOutput.stdout.contains("name: \"ns_add\"")) + assert(testOutput.stdout.contains("name: \"ns_subtract\"")) + assert(!testOutput.stdout.contains("name: \"ns_error\"")) + assert(!testOutput.stdout.contains("name: \"ns_disabled\"")) } test("viash ns list query_namespace no match") { - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "list", "-s", nsPath, "--query_namespace", "foo" ) - assert(exitCode == 1) - assert(stdout.trim() == "[]") + assert(testOutput.exitCode == Some(1)) + assert(testOutput.stdout.trim() == "[]") } } diff --git a/src/test/scala/io/viash/e2e/ns_test/MainNSTestNativeSuite.scala b/src/test/scala/io/viash/e2e/ns_test/MainNSTestNativeSuite.scala index dee8976ea..add2216bc 100644 --- a/src/test/scala/io/viash/e2e/ns_test/MainNSTestNativeSuite.scala +++ b/src/test/scala/io/viash/e2e/ns_test/MainNSTestNativeSuite.scala @@ -47,7 +47,7 @@ class MainNSTestNativeSuite extends AnyFunSuite with BeforeAndAfterAll { test("Check namespace test output without working dir message") { - val (stdout, stderr, _) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "test", "--src", nsPath, "--keep", "false" @@ -55,24 +55,24 @@ class MainNSTestNativeSuite extends AnyFunSuite with BeforeAndAfterAll { // Test inclusion of a header val regexHeader = raw"^\s*namespace\s*functionality\s*runner\s*engine\s*test_name\s*exit_code\s*duration\s*result".r - assert(regexHeader.findFirstIn(stdout).isDefined, s"\nRegex: ${regexHeader.toString}; text: \n$stdout") + assert(regexHeader.findFirstIn(testOutput.stdout).isDefined, s"\nRegex: ${regexHeader.toString}; text: \n${testOutput.stdout}") for ( (component, steps) <- components; (step, resultPattern) <- steps ) { val regex = s"""testns\\s*$component\\s*executable\\s*native\\s*$step$resultPattern""".r - assert(regex.findFirstIn(stdout).isDefined, s"\nRegex: '${regex.toString}'; text: \n$stdout") + assert(regex.findFirstIn(testOutput.stdout).isDefined, s"\nRegex: '${regex.toString}'; text: \n${testOutput.stdout}") } val regexBuildError = raw"Reading file \'.*/src/ns_error/config\.vsh\.yaml\' failed".r - assert(regexBuildError.findFirstIn(stderr).isDefined, "Expecting to get an error because of an invalid yaml in ns_error") + assert(regexBuildError.findFirstIn(testOutput.stderr).isDefined, "Expecting to get an error because of an invalid yaml in ns_error") - assert(stderr.contains("The status of the component 'ns_power' is set to deprecated.")) + assert(testOutput.stderr.contains("The status of the component 'ns_power' is set to deprecated.")) } test("Check namespace test output with working dir message") { - val (stdout, stderr, _) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "test", "--src", nsPath, "--keep", "true" @@ -80,27 +80,27 @@ class MainNSTestNativeSuite extends AnyFunSuite with BeforeAndAfterAll { // Test inclusion of a header val regexHeader = raw"^\s*namespace\s*functionality\s*runner\s*engine\s*test_name\s*exit_code\s*duration\s*result".r - assert(regexHeader.findFirstIn(stdout).isDefined, s"\nRegex: ${regexHeader.toString}; text: \n$stdout") + assert(regexHeader.findFirstIn(testOutput.stdout).isDefined, s"\nRegex: ${regexHeader.toString}; text: \n${testOutput.stdout}") val regexWdir = raw"The working directory for the namespace tests is [\w/]+[\r\n]{1,2}".r - assert(regexWdir.findFirstIn(stderr).isDefined, s"\nRegex: ${regexHeader.toString}; text: \n$stderr") + assert(regexWdir.findFirstIn(testOutput.stderr).isDefined, s"\nRegex: ${regexHeader.toString}; text: \n${testOutput.stderr}") for ( (component, steps) <- components; (step, resultPattern) <- steps ) { val regex = s"""testns\\s*$component\\s*executable\\s*native\\s*$step$resultPattern""".r - assert(regex.findFirstIn(stdout).isDefined, s"\nRegex: '${regex.toString}'; text: \n$stdout") + assert(regex.findFirstIn(testOutput.stdout).isDefined, s"\nRegex: '${regex.toString}'; text: \n${testOutput.stdout}") } val regexBuildError = raw"Reading file \'.*/src/ns_error/config\.vsh\.yaml\' failed".r - assert(regexBuildError.findFirstIn(stderr).isDefined, "Expecting to get an error because of an invalid yaml in ns_error") + assert(regexBuildError.findFirstIn(testOutput.stderr).isDefined, "Expecting to get an error because of an invalid yaml in ns_error") } test("Check namespace test output with tsv option") { val log = Paths.get(tempFolStr, "log.tsv").toFile - val (testText, _, _) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "test", "--tsv", log.toString, "--src", nsPath @@ -133,7 +133,7 @@ class MainNSTestNativeSuite extends AnyFunSuite with BeforeAndAfterAll { val fileHeader = "Test header" + sys.props("line.separator") Files.write(log.toPath, fileHeader.getBytes(StandardCharsets.UTF_8)) - val (testText, _, _) = TestHelper.testMainWithStdErr( + TestHelper.testMain( "ns", "test", "--tsv", log.toString, "--append", @@ -166,7 +166,7 @@ class MainNSTestNativeSuite extends AnyFunSuite with BeforeAndAfterAll { test("Check namespace test output with tsv and append options without the output file exists") { val log = Paths.get(tempFolStr, "log_append_new.tsv").toFile - val (testText, _, _) = TestHelper.testMainWithStdErr( + TestHelper.testMain( "ns", "test", "--tsv", log.toString, "--append", diff --git a/src/test/scala/io/viash/e2e/run/RunComputationalRequirements.scala b/src/test/scala/io/viash/e2e/run/RunComputationalRequirements.scala index 768ca2dbf..6e01480f8 100644 --- a/src/test/scala/io/viash/e2e/run/RunComputationalRequirements.scala +++ b/src/test/scala/io/viash/e2e/run/RunComputationalRequirements.scala @@ -15,105 +15,105 @@ class RunComputationalRequirements extends AnyFunSuite with BeforeAndAfterAll { private val configDeriver = ConfigDeriver(Paths.get(configFile), temporaryFolder) test("Check without computational requirements") { - val output = TestHelper.testMain( + val testOutput = TestHelper.testMain( "run", configFile ) - assert(output.contains("cpus unset")) - assert(output.contains("memory unset")) + assert(testOutput.stdout.contains("cpus unset")) + assert(testOutput.stdout.contains("memory unset")) } test("Check set cpus in CLI") { - val output = TestHelper.testMain( + val testOutput = TestHelper.testMain( "run", "--cpus", "2", configFile ) - assert(output.contains("cpus: 2")) - assert(output.contains("memory unset")) + assert(testOutput.stdout.contains("cpus: 2")) + assert(testOutput.stdout.contains("memory unset")) } test("Check set memory in CLI") { - val output = TestHelper.testMain( + val testOutput = TestHelper.testMain( "run", "--memory", "2mb", configFile ) - assert(output.contains("cpus unset")) - assert(output.contains("memory: 2")) + assert(testOutput.stdout.contains("cpus unset")) + assert(testOutput.stdout.contains("memory: 2")) } test("Check set cpus in config") { val newConfigFilePath = configDeriver.derive(""".functionality.requirements := {cpus: 3}""", "cpus_set") - val output = TestHelper.testMain( + val testOutput = TestHelper.testMain( "run", newConfigFilePath ) - assert(output.contains("cpus: 3")) - assert(output.contains("memory unset")) + assert(testOutput.stdout.contains("cpus: 3")) + assert(testOutput.stdout.contains("memory unset")) } test("Check set memory in config") { val newConfigFilePath = configDeriver.derive(""".functionality.requirements := {memory: "3 mb"}""", "memory_set") - val output = TestHelper.testMain( + val testOutput = TestHelper.testMain( "run", newConfigFilePath ) - assert(output.contains("cpus unset")) - assert(output.contains("memory: 3")) + assert(testOutput.stdout.contains("cpus unset")) + assert(testOutput.stdout.contains("memory: 3")) } test("Check set cpus and memory in config") { val newConfigFilePath = configDeriver.derive(""".functionality.requirements := {cpus: 3, memory: "3 mb"}""", "cpus_memory_set") - val output = TestHelper.testMain( + val testOutput = TestHelper.testMain( "run", newConfigFilePath ) - assert(output.contains("cpus: 3")) - assert(output.contains("memory: 3")) + assert(testOutput.stdout.contains("cpus: 3")) + assert(testOutput.stdout.contains("memory: 3")) } test("Check set cpus in config and CLI") { val newConfigFilePath = configDeriver.derive(""".functionality.requirements := {cpus: 3}""", "cpus_set2") - val output = TestHelper.testMain( + val testOutput = TestHelper.testMain( "run", "--cpus", "2", newConfigFilePath ) - assert(output.contains("cpus: 2")) - assert(output.contains("memory unset")) + assert(testOutput.stdout.contains("cpus: 2")) + assert(testOutput.stdout.contains("memory unset")) } test("Check set memory in config and CLI") { val newConfigFilePath = configDeriver.derive(""".functionality.requirements := {memory: "3 mb"}""", "memory_set2") - val output = TestHelper.testMain( + val testOutput = TestHelper.testMain( "run", "--memory", "2mb", newConfigFilePath ) - assert(output.contains("cpus unset")) - assert(output.contains("memory: 2")) + assert(testOutput.stdout.contains("cpus unset")) + assert(testOutput.stdout.contains("memory: 2")) } test("Check set cpus and memory in config and CLI") { val newConfigFilePath = configDeriver.derive(""".functionality.requirements := {cpus: 3, memory: "3 mb"}""", "cpus_memory_set2") - val output = TestHelper.testMain( + val testOutput = TestHelper.testMain( "run", "--cpus", "2", "--memory", "2mb", newConfigFilePath ) - assert(output.contains("cpus: 2")) - assert(output.contains("memory: 2")) + assert(testOutput.stdout.contains("cpus: 2")) + assert(testOutput.stdout.contains("memory: 2")) } override def afterAll(): Unit = { diff --git a/src/test/scala/io/viash/e2e/test/MainTestDockerSuite.scala b/src/test/scala/io/viash/e2e/test/MainTestDockerSuite.scala index 92c85ec00..e575be536 100644 --- a/src/test/scala/io/viash/e2e/test/MainTestDockerSuite.scala +++ b/src/test/scala/io/viash/e2e/test/MainTestDockerSuite.scala @@ -22,37 +22,37 @@ class MainTestDockerSuite extends AnyFunSuite with BeforeAndAfterAll with Parall private val configDeriver = ConfigDeriver(Paths.get(configFile), temporaryFolder) test("Check standard test output for typical outputs", DockerTest) { - val testText = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", "--engine", "docker", "--runner", "docker", configFile ) - assert(testText.contains("Running tests in temporary directory: ")) - assert(testText.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) - assert(testText.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) + assert(testOutput.stdout.contains("Cleaning up temporary directory")) - checkTempDirAndRemove(testText, false) + checkTempDirAndRemove(testOutput.stdout, false) } test("Check standard test output with trailing arguments", DockerTest) { - val testText = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", configFile, "--engine", "docker", "--runner", "docker" ) - assert(testText.contains("Running tests in temporary directory: ")) - assert(testText.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) - assert(testText.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) + assert(testOutput.stdout.contains("Cleaning up temporary directory")) - checkTempDirAndRemove(testText, false) + checkTempDirAndRemove(testOutput.stdout, false) } test("Check standard test output with leading and trailing arguments", DockerTest) { - val testText = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", "--engine", "docker", "--runner", "docker", @@ -60,17 +60,17 @@ class MainTestDockerSuite extends AnyFunSuite with BeforeAndAfterAll with Parall "-k", "false" ) - assert(testText.contains("Running tests in temporary directory: ")) - assert(testText.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) - assert(testText.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) + assert(testOutput.stdout.contains("Cleaning up temporary directory")) - checkTempDirAndRemove(testText, false) + checkTempDirAndRemove(testOutput.stdout, false) } test("Check setup strategy", DockerTest) { val newConfigFilePath = configDeriver.derive(""".engines[.type == "docker" && !has(.id) ].setup := [{ type: "docker", run: "echo 'Hello world!'" }]""", "cache_config") // first run to create cache entries - val testText = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", "--engine", "docker", "--runner", "docker", @@ -79,7 +79,7 @@ class MainTestDockerSuite extends AnyFunSuite with BeforeAndAfterAll with Parall ) // Do a second run to check if forcing a docker build using setup works - val testTextNoCaching = TestHelper.testMain( + val testOutputNoCaching = TestHelper.testMain( "test", "--engine", "docker", "--runner", "docker", @@ -89,10 +89,10 @@ class MainTestDockerSuite extends AnyFunSuite with BeforeAndAfterAll with Parall ) val regexBuildCache = raw"\n#\d \[\d/\d\] RUN echo 'Hello world!'\n#\d CACHED\n".r - assert(!regexBuildCache.findFirstIn(testTextNoCaching).isDefined, "Expected to not find caching.") + assert(!regexBuildCache.findFirstIn(testOutputNoCaching.stdout).isDefined, "Expected to not find caching.") // Do a third run to check caching - val testTextCaching = TestHelper.testMain( + val testOutputCaching = TestHelper.testMain( "test", "--engine", "docker", "--runner", "docker", @@ -103,10 +103,10 @@ class MainTestDockerSuite extends AnyFunSuite with BeforeAndAfterAll with Parall // retry once if it failed val testTextCachingWithRetry = - if (regexBuildCache.findFirstIn(testTextCaching).isDefined) { - testTextCaching + if (regexBuildCache.findFirstIn(testOutputCaching.stdout).isDefined) { + testOutputCaching } else { - checkTempDirAndRemove(testTextCaching, false) + checkTempDirAndRemove(testOutputCaching.stdout, false) TestHelper.testMain( "test", @@ -118,62 +118,62 @@ class MainTestDockerSuite extends AnyFunSuite with BeforeAndAfterAll with Parall ) } - assert(regexBuildCache.findFirstIn(testTextCachingWithRetry).isDefined, "Expected to find caching.") + assert(regexBuildCache.findFirstIn(testTextCachingWithRetry.stdout).isDefined, "Expected to find caching.") - checkTempDirAndRemove(testText, false) - checkTempDirAndRemove(testTextNoCaching, false) - checkTempDirAndRemove(testTextCachingWithRetry, false) + checkTempDirAndRemove(testOutput.stdout, false) + checkTempDirAndRemove(testOutputNoCaching.stdout, false) + checkTempDirAndRemove(testTextCachingWithRetry.stdout, false) } test("Verify base config derivation", NativeTest) { val newConfigFilePath = configDeriver.derive(Nil, "default_config") - val testText = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", "--engine", "native", "--runner", "native", newConfigFilePath ) - assert(testText.contains("Running tests in temporary directory: ")) - assert(testText.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) - assert(testText.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) + assert(testOutput.stdout.contains("Cleaning up temporary directory")) - checkTempDirAndRemove(testText, false) + checkTempDirAndRemove(testOutput.stdout, false) } test("Check failing build", DockerTest) { val newConfigFilePath = configDeriver.derive(""".engines[.type == "docker" && !has(.id) ].setup := [{ type: "apt", packages: ["get_the_machine_that_goes_ping"] }]""", "failed_build") - val testOutput = TestHelper.testMainException2[RuntimeException]( + val testOutput = TestHelper.testMainException[RuntimeException]( "test", "--engine", "docker", "--runner", "docker", newConfigFilePath ) - assert(testOutput.exceptionText == "Setup failed!") + assert(testOutput.exceptionText.get == "Setup failed!") - assert(testOutput.output.contains("Running tests in temporary directory: ")) - assert(testOutput.output.contains("ERROR! Setup failed!")) - assert(!testOutput.output.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("ERROR! Setup failed!")) + assert(!testOutput.stdout.contains("Cleaning up temporary directory")) - checkTempDirAndRemove(testOutput.output, true) + checkTempDirAndRemove(testOutput.stdout, true) } test("Check config and resource files with spaces in the filename", DockerTest) { val newConfigFilePath = Paths.get(tempFolStr, "config with spaces.vsh.yaml") Files.copy(Paths.get(configFile), newConfigFilePath) - val testText = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", "--engine", "docker", "--runner", "docker", newConfigFilePath.toString() ) - assert(testText.contains("Running tests in temporary directory: ")) - assert(testText.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) - assert(testText.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) + assert(testOutput.stdout.contains("Cleaning up temporary directory")) - checkTempDirAndRemove(testText, false) + checkTempDirAndRemove(testOutput.stdout, false) } /** diff --git a/src/test/scala/io/viash/e2e/test/MainTestNativeSuite.scala b/src/test/scala/io/viash/e2e/test/MainTestNativeSuite.scala index 075d8a507..423456ad3 100644 --- a/src/test/scala/io/viash/e2e/test/MainTestNativeSuite.scala +++ b/src/test/scala/io/viash/e2e/test/MainTestNativeSuite.scala @@ -10,6 +10,8 @@ import org.scalatest.funsuite.AnyFunSuite import scala.reflect.io.Directory import sys.process._ +import io.viash.exceptions.ConfigParserException +import io.viash.exceptions.MissingResourceFileException class MainTestNativeSuite extends AnyFunSuite with BeforeAndAfterAll { Logger.UseColorOverride.value = Some(false) @@ -23,37 +25,37 @@ class MainTestNativeSuite extends AnyFunSuite with BeforeAndAfterAll { private val configInvalidYamlFile = getClass.getResource("/testbash/invalid_configs/config_invalid_yaml.vsh.yaml").getPath test("Check standard test output for typical outputs") { - val testText = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", "--engine", "native", "--runner", "native", configFile ) - assert(testText.contains("Running tests in temporary directory: ")) - assert(testText.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) - assert(testText.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) + assert(testOutput.stdout.contains("Cleaning up temporary directory")) - checkTempDirAndRemove(testText, false) + checkTempDirAndRemove(testOutput.stdout, false) } test("Check standard test output with trailing arguments") { - val testText = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", configFile, "--engine", "native", "--runner", "native" ) - assert(testText.contains("Running tests in temporary directory: ")) - assert(testText.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) - assert(testText.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) + assert(testOutput.stdout.contains("Cleaning up temporary directory")) - checkTempDirAndRemove(testText, false) + checkTempDirAndRemove(testOutput.stdout, false) } test("Check standard test output with leading and trailing arguments") { - val testText = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", "--engine", "native", "--runner", "native", @@ -61,243 +63,243 @@ class MainTestNativeSuite extends AnyFunSuite with BeforeAndAfterAll { "-k", "false" ) - assert(testText.contains("Running tests in temporary directory: ")) - assert(testText.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) - assert(testText.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) + assert(testOutput.stdout.contains("Cleaning up temporary directory")) - checkTempDirAndRemove(testText, false) + checkTempDirAndRemove(testOutput.stdout, false) } test("Verify base config derivation") { val newConfigFilePath = configDeriver.derive(Nil, "default_config") - val testText = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", "--engine", "native", "--runner", "native", newConfigFilePath ) - assert(testText.contains("Running tests in temporary directory: ")) - assert(testText.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) - assert(testText.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) + assert(testOutput.stdout.contains("Cleaning up temporary directory")) - checkTempDirAndRemove(testText, false) + checkTempDirAndRemove(testOutput.stdout, false) } test("Check test output when no tests are specified in the functionality file") { val newConfigFilePath = configDeriver.derive("del(.functionality.test_resources)", "no_tests") - val testText = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", "--engine", "native", "--runner", "native", newConfigFilePath ) - assert(testText.contains("Running tests in temporary directory: ")) - assert(testText.contains("WARNING! No tests found!")) - assert(testText.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("WARNING! No tests found!")) + assert(testOutput.stdout.contains("Cleaning up temporary directory")) - checkTempDirAndRemove(testText, false) + checkTempDirAndRemove(testOutput.stdout, false) } test("Check test output when a test fails") { val newConfigFilePath = configDeriver.derive(""".functionality.test_resources[.path == "tests/check_outputs.sh"].path := "tests/fail_failed_test.sh"""", "failed_test") - val testText = TestHelper.testMainException[RuntimeException]( + val testOutput = TestHelper.testMainException[RuntimeException]( "test", "--engine", "native", "--runner", "native", newConfigFilePath ) - assert(testText.contains("Running tests in temporary directory: ")) - assert(testText.contains("ERROR! Only 1 out of 2 test scripts succeeded!")) - assert(!testText.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("ERROR! Only 1 out of 2 test scripts succeeded!")) + assert(!testOutput.stdout.contains("Cleaning up temporary directory")) - checkTempDirAndRemove(testText, true) + checkTempDirAndRemove(testOutput.stdout, true) } test("Check test output when test doesn't exist") { val newConfigFilePath = configDeriver.derive(""".functionality.test_resources[.path == "tests/check_outputs.sh"].path := "tests/nonexistent_test.sh"""", "nonexisting_test") - val testOutput = TestHelper.testMainException2[RuntimeException]( + val testOutput = TestHelper.testMainException[RuntimeException]( "test", "--engine", "native", "--runner", "native", newConfigFilePath ) - assert(testOutput.exceptionText == "Only 1 out of 2 test scripts succeeded!") + assert(testOutput.exceptionText.get == "Only 1 out of 2 test scripts succeeded!") - assert(testOutput.output.contains("Running tests in temporary directory: ")) - assert(testOutput.output.contains("ERROR! Only 1 out of 2 test scripts succeeded!")) - assert(!testOutput.output.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("ERROR! Only 1 out of 2 test scripts succeeded!")) + assert(!testOutput.stdout.contains("Cleaning up temporary directory")) - checkTempDirAndRemove(testOutput.output, true) + checkTempDirAndRemove(testOutput.stdout, true) } test("Check config and resource files with spaces in the filename") { val newConfigFilePath = Paths.get(tempFolStr, "config with spaces.vsh.yaml") Files.copy(Paths.get(configFile), newConfigFilePath) - val testText = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", "--engine", "native", "--runner", "native", newConfigFilePath.toString() ) - assert(testText.contains("Running tests in temporary directory: ")) - assert(testText.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) - assert(testText.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) + assert(testOutput.stdout.contains("Cleaning up temporary directory")) - checkTempDirAndRemove(testText, false) + checkTempDirAndRemove(testOutput.stdout, false) } test("Check config file without 'functionality' specified") { val newConfigFilePath = configDeriver.derive("""del(.functionality)""", "missing_functionality") - val testOutput = TestHelper.testMainException2[RuntimeException]( + val testOutput = TestHelper.testMainException[RuntimeException]( "test", "--engine", "native", "--runner", "native", newConfigFilePath ) - assert(testOutput.exceptionText.contains("must be a yaml file containing a viash config.")) - assert(testOutput.output.isEmpty) + assert(testOutput.exceptionText.get.contains("must be a yaml file containing a viash config.")) + assert(testOutput.stdout.isEmpty) } test("Check invalid runner type") { val newConfigFilePath = configDeriver.derive(""".runners += { type: "foo" }""", "invalid_runner_type") - val testOutput = TestHelper.testMainException2[RuntimeException]( + val testOutput = TestHelper.testMainException[ConfigParserException]( "test", "--engine", "native", "--runner", "native", newConfigFilePath ) - assert(testOutput.exceptionText.contains("Type 'foo' is not recognised. Valid types are 'executable', and 'nextflow'.")) - assert(testOutput.exceptionText.contains( + assert(testOutput.exceptionText.get.contains("Type 'foo' is not recognised. Valid types are 'executable', and 'nextflow'.")) + assert(testOutput.exceptionText.get.contains( """{ | "type" : "foo" |}""".stripMargin)) - assert(testOutput.output.isEmpty) + assert(testOutput.stdout.isEmpty) } test("Check invalid engine type") { val newConfigFilePath = configDeriver.derive(""".engines += { type: "foo" }""", "invalid_engine_type") - val testOutput = TestHelper.testMainException2[RuntimeException]( + val testOutput = TestHelper.testMainException[ConfigParserException]( "test", "--engine", "native", "--runner", "native", newConfigFilePath ) - assert(testOutput.exceptionText.contains("Type 'foo' is not recognised. Valid types are 'docker', and 'native'.")) - assert(testOutput.exceptionText.contains( + assert(testOutput.exceptionText.get.contains("Type 'foo' is not recognised. Valid types are 'docker', and 'native'.")) + assert(testOutput.exceptionText.get.contains( """{ | "type" : "foo" |}""".stripMargin)) - assert(testOutput.output.isEmpty) + assert(testOutput.stdout.isEmpty) } test("Check invalid platform type") { val newConfigFilePath = configDeriver.derive(""".platforms := [{ type: "foo" }]""", "invalid_platform_type") - val testOutput = TestHelper.testMainException2[RuntimeException]( + val testOutput = TestHelper.testMainException[ConfigParserException]( "test", "--engine", "native", "--runner", "native", newConfigFilePath ) - assert(testOutput.exceptionText.contains("Type 'foo' is not recognised. Valid types are 'docker', 'native', and 'nextflow'.")) - assert(testOutput.exceptionText.contains( + assert(testOutput.exceptionText.get.contains("Type 'foo' is not recognised. Valid types are 'docker', 'native', and 'nextflow'.")) + assert(testOutput.exceptionText.get.contains( """{ | "type" : "foo" |}""".stripMargin)) - assert(testOutput.output.isEmpty) + assert(testOutput.stdout.isEmpty) } test("Check invalid field in runner") { val newConfigFilePath = configDeriver.derive(""".runners += { type: "executable", foo: "bar" }""", "invalid_runner_field") - val testOutput = TestHelper.testMainException2[RuntimeException]( + val testOutput = TestHelper.testMainException[ConfigParserException]( "test", "--engine", "native", "--runner", "native", newConfigFilePath ) - assert(testOutput.exceptionText.contains("Invalid data fields for ExecutableRunner.")) - assert(testOutput.exceptionText.contains( + assert(testOutput.exceptionText.get.contains("Invalid data fields for ExecutableRunner.")) + assert(testOutput.exceptionText.get.contains( """{ | "type" : "executable", | "foo" : "bar" |}""".stripMargin)) - assert(testOutput.output.isEmpty) + assert(testOutput.stdout.isEmpty) } test("Check invalid field in engine") { val newConfigFilePath = configDeriver.derive(""".engines += { type: "native", foo: "bar" }""", "invalid_engine_field") - val testOutput = TestHelper.testMainException2[RuntimeException]( + val testOutput = TestHelper.testMainException[ConfigParserException]( "test", "--engine", "native", "--runner", "native", newConfigFilePath ) - assert(testOutput.exceptionText.contains("Invalid data fields for NativeEngine.")) - assert(testOutput.exceptionText.contains( + assert(testOutput.exceptionText.get.contains("Invalid data fields for NativeEngine.")) + assert(testOutput.exceptionText.get.contains( """{ | "type" : "native", | "foo" : "bar" |}""".stripMargin)) - assert(testOutput.output.isEmpty) + assert(testOutput.stdout.isEmpty) } test("Check invalid field in platform") { val newConfigFilePath = configDeriver.derive(""".platforms := [{ type: "native", foo: "bar" }]""", "invalid_platform_field") - val testOutput = TestHelper.testMainException2[RuntimeException]( + val testOutput = TestHelper.testMainException[ConfigParserException]( "test", "--engine", "native", "--runner", "native", newConfigFilePath ) - assert(testOutput.exceptionText.contains("Invalid data fields for NativePlatform.")) - assert(testOutput.exceptionText.contains( + assert(testOutput.exceptionText.get.contains("Invalid data fields for NativePlatform.")) + assert(testOutput.exceptionText.get.contains( """{ | "type" : "native", | "foo" : "bar" |}""".stripMargin)) - assert(testOutput.output.isEmpty) + assert(testOutput.stdout.isEmpty) } test("Check valid viash config yaml but with wrong file extension") { val newConfigFilePath = Paths.get(tempFolStr, "config.txt") Files.copy(Paths.get(configFile), newConfigFilePath) - val testOutput = TestHelper.testMainException2[RuntimeException]( + val testOutput = TestHelper.testMainException[RuntimeException]( "test", "--engine", "native", "--runner", "native", newConfigFilePath.toString() ) - assert(testOutput.exceptionText.contains("must be a yaml file containing a viash config.")) - assert(testOutput.output.isEmpty) + assert(testOutput.exceptionText.get.contains("must be a yaml file containing a viash config.")) + assert(testOutput.stdout.isEmpty) } test("Check invalid viash config yaml") { - val testOutput = TestHelper.testMainException2[io.circe.ParsingFailure]( + val testOutput = TestHelper.testMainException[org.yaml.snakeyaml.parser.ParserException]( "test", "--engine", "native", "--runner", "native", configInvalidYamlFile ) - assert(testOutput.exceptionText.contains("while parsing a flow mapping")) - assert(testOutput.output.isEmpty) + assert(testOutput.exceptionText.get.contains("while parsing a flow mapping")) + assert(testOutput.stdout.isEmpty) } test("Check output in case --keep true is specified") { - val testText = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", "--engine", "native", "--runner", "native", @@ -305,15 +307,15 @@ class MainTestNativeSuite extends AnyFunSuite with BeforeAndAfterAll { configFile ) - assert(testText.contains("Running tests in temporary directory: ")) - assert(testText.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) - assert(!testText.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) + assert(!testOutput.stdout.contains("Cleaning up temporary directory")) - checkTempDirAndRemove(testText, true) + checkTempDirAndRemove(testOutput.stdout, true) } test("Check output in case --keep false is specified") { - val testText = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", "--engine", "native", "--runner", "native", @@ -321,16 +323,16 @@ class MainTestNativeSuite extends AnyFunSuite with BeforeAndAfterAll { configFile ) - assert(testText.contains("Running tests in temporary directory: ")) - assert(testText.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) - assert(testText.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) + assert(testOutput.stdout.contains("Cleaning up temporary directory")) - checkTempDirAndRemove(testText, false) + checkTempDirAndRemove(testOutput.stdout, false) } test("Check test output when a test fails and --keep true is specified") { val newConfigFilePath = configDeriver.derive(""".functionality.test_resources[.path == "tests/check_outputs.sh"].path := "tests/fail_failed_test.sh"""", "failed_test_keep_true") - val testOutput = TestHelper.testMainException2[RuntimeException]( + val testOutput = TestHelper.testMainException[RuntimeException]( "test", "--engine", "native", "--runner", "native", @@ -338,13 +340,13 @@ class MainTestNativeSuite extends AnyFunSuite with BeforeAndAfterAll { newConfigFilePath ) - assert(testOutput.exceptionText == "Only 1 out of 2 test scripts succeeded!") + assert(testOutput.exceptionText.get == "Only 1 out of 2 test scripts succeeded!") - assert(testOutput.output.contains("Running tests in temporary directory: ")) - assert(testOutput.output.contains("ERROR! Only 1 out of 2 test scripts succeeded!")) - assert(!testOutput.output.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("ERROR! Only 1 out of 2 test scripts succeeded!")) + assert(!testOutput.stdout.contains("Cleaning up temporary directory")) - checkTempDirAndRemove(testOutput.output, true) + checkTempDirAndRemove(testOutput.stdout, true) } test("Check test output when a test fails and --keep false is specified") { @@ -352,7 +354,7 @@ class MainTestNativeSuite extends AnyFunSuite with BeforeAndAfterAll { """.functionality.test_resources[.path == "tests/check_outputs.sh"].path := "tests/fail_failed_test.sh"""", "failed_test_keep_false" ) - val testOutput = TestHelper.testMainException2[RuntimeException]( + val testOutput = TestHelper.testMainException[RuntimeException]( "test", "--engine", "native", "--runner", "native", @@ -360,69 +362,69 @@ class MainTestNativeSuite extends AnyFunSuite with BeforeAndAfterAll { newConfigFilePath ) - assert(testOutput.exceptionText == "Only 1 out of 2 test scripts succeeded!") + assert(testOutput.exceptionText.get == "Only 1 out of 2 test scripts succeeded!") - assert(testOutput.output.contains("Running tests in temporary directory: ")) - assert(testOutput.output.contains("ERROR! Only 1 out of 2 test scripts succeeded!")) - assert(testOutput.output.contains("Cleaning up temporary directory")) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("ERROR! Only 1 out of 2 test scripts succeeded!")) + assert(testOutput.stdout.contains("Cleaning up temporary directory")) - checkTempDirAndRemove(testOutput.output, false) + checkTempDirAndRemove(testOutput.stdout, false) } test("Check deprecation warning") { val newConfigFilePath = configDeriver.derive(""".functionality.status := "deprecated"""", "deprecated") - val (testText, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "test", "--engine", "native", "--runner", "native", newConfigFilePath ) - assert(exitCode == 0) - assert(testText.contains("Running tests in temporary directory: ")) - assert(testText.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) - assert(testText.contains("Cleaning up temporary directory")) + assert(testOutput.exitCode == Some(0)) + assert(testOutput.stdout.contains("Running tests in temporary directory: ")) + assert(testOutput.stdout.contains("SUCCESS! All 2 out of 2 test scripts succeeded!")) + assert(testOutput.stdout.contains("Cleaning up temporary directory")) - assert(stderr.contains("The status of the component 'testbash' is set to deprecated.")) + assert(testOutput.stderr.contains("The status of the component 'testbash' is set to deprecated.")) - checkTempDirAndRemove(testText, false) + checkTempDirAndRemove(testOutput.stdout, false) } test("Check standard test output with bad engine name") { - val testOutput = TestHelper.testMainException2[RuntimeException]( + val testOutput = TestHelper.testMainException[RuntimeException]( "test", "--engine", "non_existing_engine", "--runner", "native", configFile ) - assert(testOutput.exceptionText == "no engine id matching regex 'non_existing_engine' could not be found in the config.") - assert(testOutput.output.isEmpty) + assert(testOutput.exceptionText.get == "no engine id matching regex 'non_existing_engine' could not be found in the config.") + assert(testOutput.stdout.isEmpty) } test("Check standard test output with bad runner name") { - val testOutput = TestHelper.testMainException2[RuntimeException]( + val testOutput = TestHelper.testMainException[RuntimeException]( "test", "--engine", "native", "--runner", "non_existing_runner", configFile ) - assert(testOutput.exceptionText == "no runner id matching regex 'non_existing_runner' could not be found in the config.") - assert(testOutput.output.isEmpty) + assert(testOutput.exceptionText.get == "no runner id matching regex 'non_existing_runner' could not be found in the config.") + assert(testOutput.stdout.isEmpty) } test("Check standard test output with missing test resource") { val newConfigFilePath = configDeriver.derive(""".functionality.test_resources += { type: 'file', path: 'foobar.txt' }""", "deprecated") - val testOutput = TestHelper.testMainException2[RuntimeException]( + val testOutput = TestHelper.testMainException[MissingResourceFileException]( "test", "--engine", "native", "--runner", "native", newConfigFilePath ) - assert(testOutput.exceptionText.matches("Missing resource .*foobar\\.txt as specified in .*")) + assert(testOutput.exceptionText.get.matches("Missing resource .*foobar\\.txt as specified in .*")) } /** diff --git a/src/test/scala/io/viash/e2e/test/TestComputationalRequirements.scala b/src/test/scala/io/viash/e2e/test/TestComputationalRequirements.scala index b8a7f48cc..85c25d6fb 100644 --- a/src/test/scala/io/viash/e2e/test/TestComputationalRequirements.scala +++ b/src/test/scala/io/viash/e2e/test/TestComputationalRequirements.scala @@ -15,105 +15,105 @@ class TestComputationalRequirements extends AnyFunSuite with BeforeAndAfterAll { private val configDeriver = ConfigDeriver(Paths.get(configFile), temporaryFolder) test("Check without computational requirements") { - val output = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", configFile ) - assert(output.contains("cpus unset")) - assert(output.contains("memory unset")) + assert(testOutput.stdout.contains("cpus unset")) + assert(testOutput.stdout.contains("memory unset")) } test("Check set cpus in CLI") { - val output = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", "--cpus", "2", configFile ) - assert(output.contains("cpus: 2")) - assert(output.contains("memory unset")) + assert(testOutput.stdout.contains("cpus: 2")) + assert(testOutput.stdout.contains("memory unset")) } test("Check set memory in CLI") { - val output = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", "--memory", "2mb", configFile ) - assert(output.contains("cpus unset")) - assert(output.contains("memory: 2")) + assert(testOutput.stdout.contains("cpus unset")) + assert(testOutput.stdout.contains("memory: 2")) } test("Check set cpus in config") { val newConfigFilePath = configDeriver.derive(""".functionality.requirements := {cpus: 3}""", "cpus_set") - val output = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", newConfigFilePath ) - assert(output.contains("cpus: 3")) - assert(output.contains("memory unset")) + assert(testOutput.stdout.contains("cpus: 3")) + assert(testOutput.stdout.contains("memory unset")) } test("Check set memory in config") { val newConfigFilePath = configDeriver.derive(""".functionality.requirements := {memory: "3 mb"}""", "memory_set") - val output = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", newConfigFilePath ) - assert(output.contains("cpus unset")) - assert(output.contains("memory: 3")) + assert(testOutput.stdout.contains("cpus unset")) + assert(testOutput.stdout.contains("memory: 3")) } test("Check set cpus and memory in config") { val newConfigFilePath = configDeriver.derive(""".functionality.requirements := {cpus: 3, memory: "3 mb"}""", "cpus_memory_set") - val output = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", newConfigFilePath ) - assert(output.contains("cpus: 3")) - assert(output.contains("memory: 3")) + assert(testOutput.stdout.contains("cpus: 3")) + assert(testOutput.stdout.contains("memory: 3")) } test("Check set cpus in config and CLI") { val newConfigFilePath = configDeriver.derive(""".functionality.requirements := {cpus: 3}""", "cpus_set2") - val output = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", "--cpus", "2", newConfigFilePath ) - assert(output.contains("cpus: 2")) - assert(output.contains("memory unset")) + assert(testOutput.stdout.contains("cpus: 2")) + assert(testOutput.stdout.contains("memory unset")) } test("Check set memory in config and CLI") { val newConfigFilePath = configDeriver.derive(""".functionality.requirements := {memory: "3 mb"}""", "memory_set2") - val output = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", "--memory", "2mb", newConfigFilePath ) - assert(output.contains("cpus unset")) - assert(output.contains("memory: 2")) + assert(testOutput.stdout.contains("cpus unset")) + assert(testOutput.stdout.contains("memory: 2")) } test("Check set cpus and memory in config and CLI") { val newConfigFilePath = configDeriver.derive(""".functionality.requirements := {cpus: 3, memory: "3 mb"}""", "cpus_memory_set2") - val output = TestHelper.testMain( + val testOutput = TestHelper.testMain( "test", "--cpus", "2", "--memory", "2mb", newConfigFilePath ) - assert(output.contains("cpus: 2")) - assert(output.contains("memory: 2")) + assert(testOutput.stdout.contains("cpus: 2")) + assert(testOutput.stdout.contains("memory: 2")) } override def afterAll(): Unit = { diff --git a/src/test/scala/io/viash/functionality/dependencies/Dependency.scala b/src/test/scala/io/viash/functionality/dependencies/Dependency.scala index bb1c9fe1c..12199bceb 100644 --- a/src/test/scala/io/viash/functionality/dependencies/Dependency.scala +++ b/src/test/scala/io/viash/functionality/dependencies/Dependency.scala @@ -56,13 +56,123 @@ class DependencyTest extends AnyFunSuite with BeforeAndAfterAll { writeTestConfig(testFolder.resolve("src/dep2/config.vsh.yaml"), fun2) // build - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "build", "-s", testFolder.resolve("src").toString(), "-t", testFolder.resolve("target").toString() ) - assert(stderr.strip == "All 2 configs built successfully", "check build was successful") + assert(testOutput.stderr.strip == "All 2 configs built successfully", "check build was successful") + + // check file & file content + val outputPath = testFolder.resolve("target/executable/dep2/dep2") + val executable = outputPath.toFile + assert(executable.exists) + assert(executable.canExecute) + + val outputText = IO.read(outputPath.toUri()) + assert(outputText.contains("VIASH_DEP_DEP1="), "check the dependency is set in the output script") + + // check output when running + val out = Exec.runCatch( + Seq(executable.toString) + ) + + assert(out.output == "Hello from dep1\nHello from dep2\n") + assert(out.exitValue == 0) + } + + test("Use a local repository with an absolute path") { + val testFolder = createViashSubFolder(temporaryFolder, "local_test_absolute_path") + + // write test files + val fun1 = Functionality( + name = "dep1", + resources = textBashScript("echo Hello from dep1"), + ) + val fun2 = Functionality( + name = "dep2", + resources = textBashScript("$dep_dep1\necho Hello from dep2"), + dependencies = List(Dependency("dep1", repository = Right(LocalRepository(path = Some("/dependencies"))))) + ) + + writeTestConfig(testFolder.resolve("dependencies/src/dep1/config.vsh.yaml"), fun1) + writeTestConfig(testFolder.resolve("src/dep2/config.vsh.yaml"), fun2) + + // build our local repository + val build1 = TestHelper.testMain( + workingDir = Some(testFolder.resolve("dependencies")), + "ns", "build", + "-s", testFolder.resolve("dependencies/src").toString(), + "-t", testFolder.resolve("dependencies/target").toString() + ) + + assert(build1.stderr.strip == "All 1 configs built successfully", "check dependency build was successful") + + // build + val build2 = TestHelper.testMain( + workingDir = Some(testFolder), + "ns", "build", + "-s", testFolder.resolve("src").toString(), + "-t", testFolder.resolve("target").toString() + ) + + assert(build2.stderr.strip == "All 1 configs built successfully", "check build was successful") + + // check file & file content + val outputPath = testFolder.resolve("target/executable/dep2/dep2") + val executable = outputPath.toFile + assert(executable.exists) + assert(executable.canExecute) + + val outputText = IO.read(outputPath.toUri()) + assert(outputText.contains("VIASH_DEP_DEP1="), "check the dependency is set in the output script") + + // check output when running + val out = Exec.runCatch( + Seq(executable.toString) + ) + + assert(out.output == "Hello from dep1\nHello from dep2\n") + assert(out.exitValue == 0) + } + + test("Use a local repository with a relative path") { + val testFolder = createViashSubFolder(temporaryFolder, "local_test_absolute_path") + + // write test files + val fun1 = Functionality( + name = "dep1", + resources = textBashScript("echo Hello from dep1"), + ) + val fun2 = Functionality( + name = "dep2", + resources = textBashScript("$dep_dep1\necho Hello from dep2"), + dependencies = List(Dependency("dep1", repository = Right(LocalRepository(path = Some("../../dependencies"))))) + ) + + writeTestConfig(testFolder.resolve("dependencies/src/dep1/config.vsh.yaml"), fun1) + writeTestConfig(testFolder.resolve("src/dep2/config.vsh.yaml"), fun2) + + // build our local repository + val build1 = TestHelper.testMain( + workingDir = Some(testFolder.resolve("dependencies")), + "ns", "build", + "-s", testFolder.resolve("dependencies/src").toString(), + "-t", testFolder.resolve("dependencies/target").toString() + ) + + assert(build1.stderr.strip == "All 1 configs built successfully", "check dependency build was successful") + + // build + val build2 = TestHelper.testMain( + workingDir = Some(testFolder), + "ns", "build", + "-s", testFolder.resolve("src").toString(), + "-t", testFolder.resolve("target").toString() + ) + + assert(build2.stderr.strip == "All 1 configs built successfully", "check build was successful") // check file & file content val outputPath = testFolder.resolve("target/executable/dep2/dep2") @@ -95,13 +205,13 @@ class DependencyTest extends AnyFunSuite with BeforeAndAfterAll { writeTestConfig(testFolder.resolve("src/dep3/config.vsh.yaml"), fun) // build - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "build", "-s", testFolder.resolve("src").toString(), "-t", testFolder.resolve("target").toString() ) - assert(stderr.strip.contains("All 1 configs built successfully"), "check build was successful") + assert(testOutput.stderr.strip.contains("All 1 configs built successfully"), "check build was successful") // check file & file content val outputPath = testFolder.resolve("target/executable/dep3/dep3") @@ -134,13 +244,13 @@ class DependencyTest extends AnyFunSuite with BeforeAndAfterAll { writeTestConfig(testFolder.resolve("src/dep4/config.vsh.yaml"), fun) // build - val (stdout, stderr, exitCode) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "ns", "build", "-s", testFolder.resolve("src").toString(), "-t", testFolder.resolve("target").toString() ) - assert(stderr.strip.contains("All 1 configs built successfully"), "check build was successful") + assert(testOutput.stderr.strip.contains("All 1 configs built successfully"), "check build was successful") // check file & file content val outputPath = testFolder.resolve("target/executable/dep4/dep4") diff --git a/src/test/scala/io/viash/runners/nextflow/NextflowScriptTest.scala b/src/test/scala/io/viash/runners/nextflow/NextflowScriptTest.scala index 55f213787..8bb48fcbf 100644 --- a/src/test/scala/io/viash/runners/nextflow/NextflowScriptTest.scala +++ b/src/test/scala/io/viash/runners/nextflow/NextflowScriptTest.scala @@ -36,9 +36,7 @@ class NextflowScriptTest extends AnyFunSuite with BeforeAndAfterAll { test("Build pipeline components", DockerTest, NextflowTest) { // build the nextflow containers - // TODO: use the correct CWD to build the pipeline to be ablke - // to detect the correct path to the _viash.yaml file - val (_, _, _) = TestHelper.testMainWithStdErr( + TestHelper.testMain( "ns", "build", "-s", srcPath, "-t", targetPath, diff --git a/src/test/scala/io/viash/runners/nextflow/Vdsl3ModuleTest.scala b/src/test/scala/io/viash/runners/nextflow/Vdsl3ModuleTest.scala index 92144e98d..c048348c7 100644 --- a/src/test/scala/io/viash/runners/nextflow/Vdsl3ModuleTest.scala +++ b/src/test/scala/io/viash/runners/nextflow/Vdsl3ModuleTest.scala @@ -35,7 +35,7 @@ class Vdsl3ModuleTest extends AnyFunSuite with BeforeAndAfterAll { test("Build pipeline components", DockerTest, NextflowTest) { // build the nextflow containers - val (_, _, _) = TestHelper.testMainWithStdErr( + TestHelper.testMain( "ns", "build", "--runner", "nextflow", "-s", srcPath, @@ -101,15 +101,15 @@ class Vdsl3ModuleTest extends AnyFunSuite with BeforeAndAfterAll { val correctedStdOut2 = regex.matcher(correctedStdOut1).replaceAll("") // run Viash's --help - val (stdOut2, stdErr2, exitCode2) = TestHelper.testMainWithStdErr( + val testOutput = TestHelper.testMain( "run", workflowsPath + "/pipeline3/config.vsh.yaml", "--", "--help" ) - assert(exitCode2 == 0) + assert(testOutput.exitCode == Some(0)) // check if they are the same - assert(correctedStdOut2 == stdOut2) + assert(correctedStdOut2 == testOutput.stdout) } override def afterAll(): Unit = { diff --git a/src/test/scala/io/viash/runners/nextflow/Vdsl3StandaloneTest.scala b/src/test/scala/io/viash/runners/nextflow/Vdsl3StandaloneTest.scala index 93447eeb8..31260ac98 100644 --- a/src/test/scala/io/viash/runners/nextflow/Vdsl3StandaloneTest.scala +++ b/src/test/scala/io/viash/runners/nextflow/Vdsl3StandaloneTest.scala @@ -35,7 +35,7 @@ class Vdsl3StandaloneTest extends AnyFunSuite with BeforeAndAfterAll { test("Build pipeline components", DockerTest, NextflowTest) { // build the nextflow containers - val (_, _, _) = TestHelper.testMainWithStdErr( + TestHelper.testMain( "ns", "build", "-s", srcPath, "-t", targetPath, diff --git a/src/test/scala/io/viash/runners/nextflow/WorkflowHelperTest.scala b/src/test/scala/io/viash/runners/nextflow/WorkflowHelperTest.scala index 8a12836b9..ad9a3e9c1 100644 --- a/src/test/scala/io/viash/runners/nextflow/WorkflowHelperTest.scala +++ b/src/test/scala/io/viash/runners/nextflow/WorkflowHelperTest.scala @@ -37,7 +37,7 @@ class WorkflowHelperTest extends AnyFunSuite with BeforeAndAfterAll { test("Build pipeline components", DockerTest, NextflowTest) { // build the nextflow containers - val (_, _, _) = TestHelper.testMainWithStdErr( + TestHelper.testMain( "ns", "build", "-s", srcPath, "-t", targetPath, From 94a8b6b48b4f2de5f4d384940b5102cb647bf36b Mon Sep 17 00:00:00 2001 From: Hendrik Cannoodt Date: Fri, 13 Oct 2023 11:28:03 +0200 Subject: [PATCH 3/4] Change minor references to platforms and one grammatical mistake (#568) --- src/main/scala/io/viash/engines/NativeEngine.scala | 2 +- src/main/scala/io/viash/runners/NextflowRunner.scala | 4 ++-- src/main/scala/io/viash/runners/nextflow/NextflowHelper.scala | 3 ++- 3 files changed, 5 insertions(+), 4 deletions(-) diff --git a/src/main/scala/io/viash/engines/NativeEngine.scala b/src/main/scala/io/viash/engines/NativeEngine.scala index b1f3c484e..0c13ca4af 100644 --- a/src/main/scala/io/viash/engines/NativeEngine.scala +++ b/src/main/scala/io/viash/engines/NativeEngine.scala @@ -30,7 +30,7 @@ import io.viash.schemas._ "yaml") @subclass("native") final case class NativeEngine( - @description("Name of the engine. As with all engines, you can give a engine a different name. By specifying `id: foo`, you can target this engine (only) by specifying `...` in any of the Viash commands.") + @description("Name of the engine. As with all engines, you can give an engine a different name. By specifying `id: foo`, you can target this engine (only) by specifying `...` in any of the Viash commands.") @example("id: foo", "yaml") @default("native") id: String = "native", diff --git a/src/main/scala/io/viash/runners/NextflowRunner.scala b/src/main/scala/io/viash/runners/NextflowRunner.scala index bc1765794..0e345a69c 100644 --- a/src/main/scala/io/viash/runners/NextflowRunner.scala +++ b/src/main/scala/io/viash/runners/NextflowRunner.scala @@ -204,7 +204,7 @@ final case class NextflowRunner( if (mainScript.isInstanceOf[Executable]) { throw new NotImplementedError( - "Running executables through a NextflowPlatform is not (yet) implemented. " + + "Running executables through a NextflowRunner is not (yet) implemented. " + "Create a support ticket to request this functionality if necessary." ) } @@ -212,7 +212,7 @@ final case class NextflowRunner( /************************* MAIN.NF *************************/ val directivesToJson = directives.copy( - // if a docker platform is defined but the directives.container isn't, use the image of the dockerplatform as default + // if a docker engine is defined but the directives.container isn't, use the image of the docker engine as default container = directives.container orElse containerDirective.map(cd => Left(cd.toMap)), // is memory requirements are defined but directives.memory isn't, use that instead memory = directives.memory orElse config.functionality.requirements.memoryAsBytes.map(_.toString + " B"), diff --git a/src/main/scala/io/viash/runners/nextflow/NextflowHelper.scala b/src/main/scala/io/viash/runners/nextflow/NextflowHelper.scala index 7799a0538..e8729adcc 100644 --- a/src/main/scala/io/viash/runners/nextflow/NextflowHelper.scala +++ b/src/main/scala/io/viash/runners/nextflow/NextflowHelper.scala @@ -215,7 +215,8 @@ object NextflowHelper { def renderDependencies(config: Config): String = { // TODO ideally we'd already have 'thisPath' precalculated but until that day, calculate it here - val thisPath = Paths.get(ViashNamespace.targetOutputPath("", "invalid_platform_name", config.functionality.namespace, config.functionality.name)) + // The name of the runner doesn't really matter here as it is just used to generate the relative location, and we don't have access to it anyway. + val thisPath = Paths.get(ViashNamespace.targetOutputPath("", "invalid_runner_name", config.functionality.namespace, config.functionality.name)) val depStrs = config.functionality.dependencies.map{ dep => NextflowHelper.renderInclude(dep, thisPath) From 7f1cf9b321c2ca3b4d61cb5da4c5556c443b59ab Mon Sep 17 00:00:00 2001 From: Hendrik Cannoodt Date: Fri, 13 Oct 2023 12:01:02 +0200 Subject: [PATCH 4/4] Handle exception when __merge__ uses an invalid yaml (#570) * Handle exception when __merge__ uses an invalid yaml Throw a clean exception instead of a stacktrace Handle same issue during dependency resolution for reading config yamls * Add the PR # * Improve exception text when the __merge__ type is invalid Create a new exception type inheriting AbstractConfigException * Add some unit tests --- CHANGELOG.md | 4 ++ .../io/viash/exceptions/ConfigException.scala | 5 ++ .../io/viash/helpers/DependencyResolver.scala | 6 +- .../io/viash/helpers/circe/RichJson.scala | 11 ++-- .../io/viash/helpers/circe/RichJsonTest.scala | 63 +++++++++++++++++++ 5 files changed, 82 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2e89bd247..0edc95b5b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,6 +21,10 @@ TODO add summary * `testbenches`: Add testbenches for local dependencies (PR #565). * `testbenches`: Refactor testbenches helper functions to uniformize them (PR #565). + +## BUG FIXES + +* `__merge__`: Handle invalid yaml during merging (PR #570). There was not enough error handling during this operation. Switched to the more advanced `Convert.textToJson` helper method. # Viash 0.8.0-RC5 (2023-10-11): Fix run workflow diff --git a/src/main/scala/io/viash/exceptions/ConfigException.scala b/src/main/scala/io/viash/exceptions/ConfigException.scala index 98d837979..e053e097e 100644 --- a/src/main/scala/io/viash/exceptions/ConfigException.scala +++ b/src/main/scala/io/viash/exceptions/ConfigException.scala @@ -43,4 +43,9 @@ case class ConfigParserSubTypeException(tpe: String, validTypes: List[String], j case class ConfigParserValidationException(tpe: String, json: String) extends Exception { val shortType = tpe.split("\\.").last override def getMessage(): String = s"Invalid data fields for $shortType.\n$json" +} + +case class ConfigParserMergeException(uri: String, innerMessage: String, json: String) extends AbstractConfigException { + val e: Throwable = null + override def getMessage(): String = json } \ No newline at end of file diff --git a/src/main/scala/io/viash/helpers/DependencyResolver.scala b/src/main/scala/io/viash/helpers/DependencyResolver.scala index b177be112..fe56f439b 100644 --- a/src/main/scala/io/viash/helpers/DependencyResolver.scala +++ b/src/main/scala/io/viash/helpers/DependencyResolver.scala @@ -28,13 +28,13 @@ import java.nio.file.Files import java.io.IOException import java.io.UncheckedIOException import io.viash.helpers.{IO, Logging} -import io.circe.yaml.parser import io.circe.Json import io.viash.config.Config._ import io.viash.ViashNamespace import io.viash.functionality.dependencies.Dependency import io.viash.functionality.resources.NextflowScript import io.viash.exceptions.MissingDependencyException +import io.viash.helpers.circe.Convert object DependencyResolver extends Logging { @@ -227,7 +227,7 @@ object DependencyResolver extends Logging { // The yaml file in the target folder should be final // We're also assuming that the file will be proper yaml and an actual viash config file val yamlText = IO.read(IO.uri(configPath)) - val json = parser.parse(yamlText).toOption.get + val json = Convert.textToJson(yamlText, configPath) def getFunctionalityName(json: Json): Option[String] = { json.hcursor.downField("functionality").downField("name").as[String].toOption @@ -256,7 +256,7 @@ object DependencyResolver extends Logging { def getSparseDependencyInfo(configPath: String): List[String] = { try { val yamlText = IO.read(IO.uri(configPath)) - val json = parser.parse(yamlText).toOption.get + val json = Convert.textToJson(yamlText, configPath) val dependencies = json.hcursor.downField("functionality").downField("dependencies").focus.flatMap(_.asArray).get dependencies.flatMap(_.hcursor.downField("writtenPath").as[String].toOption).toList diff --git a/src/main/scala/io/viash/helpers/circe/RichJson.scala b/src/main/scala/io/viash/helpers/circe/RichJson.scala index 01f31b991..39a0b6fb2 100644 --- a/src/main/scala/io/viash/helpers/circe/RichJson.scala +++ b/src/main/scala/io/viash/helpers/circe/RichJson.scala @@ -26,6 +26,7 @@ import io.circe.{Json, Printer => JsonPrinter} import io.circe.yaml.{Printer => YamlPrinter} import io.viash.helpers.IO +import io.viash.exceptions._ class RichJson(json: Json) { /** @@ -150,6 +151,9 @@ class RichJson(json: Json) { List(y.asString.get) } else { // TODO: add decent error message instead of simply .get + if (y.asArray.get.filter(!_.isString).nonEmpty) { + throw new ConfigParserMergeException(uri.toString, "invalid merge tag type. Must be a String or Array of Strings", y.toString()) + } y.asArray.get.map(_.asString.get).toList } @@ -197,8 +201,7 @@ class RichJson(json: Json) { val str = IO.read(newURI) // parse as yaml - // TODO: add decent error message instead of simply .get - val newJson1 = io.circe.yaml.parser.parse(str).toOption.get + val newJson1 = Convert.textToJson(str, newURI.toString) // recurse through new json as well val newJson2 = newJson1.inherit(newURI, projectDir = projectDir, stripInherits = stripInherits) @@ -212,8 +215,8 @@ class RichJson(json: Json) { // return combined object jsMerged.asObject.get - case Some(_) => - throw new RuntimeException("Invalid merge tag type. Must be a String or Array.") + case Some(j) => + throw new ConfigParserMergeException(uri.toString, "invalid merge tag type. Must be a String or Array of Strings", j.toString()) case None => obj1 } val obj3 = obj2.mapValues(x => x.inherit(uri, projectDir = projectDir, stripInherits = stripInherits)) diff --git a/src/test/scala/io/viash/helpers/circe/RichJsonTest.scala b/src/test/scala/io/viash/helpers/circe/RichJsonTest.scala index 5aefc796d..e3f358265 100644 --- a/src/test/scala/io/viash/helpers/circe/RichJsonTest.scala +++ b/src/test/scala/io/viash/helpers/circe/RichJsonTest.scala @@ -6,6 +6,8 @@ import io.circe._ import io.circe.yaml.parser import io.viash.helpers.{IO, Logger} import java.nio.file.Files +import io.viash.exceptions._ +import java.io.FileNotFoundException class RichJsonTest extends AnyFunSuite with BeforeAndAfterAll { Logger.UseColorOverride.value = Some(false) @@ -265,6 +267,67 @@ class RichJsonTest extends AnyFunSuite with BeforeAndAfterAll { assert(jsonOut == jsonExpected) } + test("check exception when merging invalid yaml") { + val json1 = parser.parse(""" + |__merge__: not_existing.yaml + |a: [1, 2] + |""".stripMargin + ).getOrElse(Json.Null) + + val caught = intercept[FileNotFoundException] { + json1.inherit(temporaryFolder.toUri(), projectDir = None) + } + + assert(caught.getMessage().contains("not_existing.yaml")) + } + + test("check exception when __merge__ isn't a string") { + val json1 = parser.parse(""" + |__merge__: 5 + |a: [1, 2] + |""".stripMargin + ).getOrElse(Json.Null) + + val caught = intercept[ConfigParserMergeException] { + json1.inherit(temporaryFolder.toUri(), projectDir = None) + } + + assert(caught.innerMessage.contains("invalid merge tag type")) + assert(caught.getMessage().contains("5")) + } + + test("check exception when __merge__ is an array but contains something else than a string") { + val json1 = parser.parse(""" + |__merge__: [5, foo.yaml] + |a: [1, 2] + |""".stripMargin + ).getOrElse(Json.Null) + + val caught = intercept[ConfigParserMergeException] { + json1.inherit(temporaryFolder.toUri(), projectDir = None) + } + + assert(caught.innerMessage.contains("invalid merge tag type")) + assert(caught.getMessage().contains("5")) + } + + test("inherit with invalid yaml") { + // write json to file + IO.write("one: foo\n/wqwqws", temporaryFolder.resolve("invalid.yaml")) + + val json1 = parser.parse(""" + |__merge__: invalid.yaml + |a: [1, 2] + |""".stripMargin + ).getOrElse(Json.Null) + + val caught = intercept[ConfigYamlException] { + json1.inherit(temporaryFolder.toUri(), projectDir = None) + } + + assert(caught.innerMessage.contains("invalid Yaml structure")) + } + override def afterAll(): Unit = { IO.deleteRecursively(temporaryFolder) }