feat(baseline): Allow force creating fresh baseline

docs/configuration.md

@@ -452,7 +452,7 @@ Config file: `"since": { }`
 
 Use git information to test only code changes since the given target. Stryker will only report on mutants within the changed code. All other mutants will not have a result.
 
-If you wish to test only changed sources and tests but would like to have a complete mutation report see [with-baseline](#with-baseline-flag-committish).
+If you wish to test only changed sources and tests but would like to have a complete mutation report see [with-baseline](#with-baseline-committish).
 
 Set the diffing target on the command line by passing a committish with the since flag in the format `--since:<committish>`.
 Set the diffing target in the config file by setting the [since target](#sincetarget-committish) option.
@@ -492,36 +492,26 @@ Use [globbing syntax](https://en.wikipedia.org/wiki/Glob_(programming)) for wild
 
 ## Baseline
 
-### `with-baseline` &lt;`flag`&gt; [`:committish`]
+### `with-baseline` &lt;`committish`&gt;
 
 Default: `false`  
-Command line: `--with-baseline:feat-2`  
-Config file: `"baseline": { }`
+Command line: `with-baseline feat-2`  
+Config file: `"baseline": { }`  
 
-Enabling `with-baseline` saves the mutation report to a storage location such as the filesystem. The mutation report is loaded at the start of the next mutation run. Any changed source code or unit test results in a reset of the mutants affected by the change. For unchanged mutants the previous result is reused. This feature expands on the [since](#since-flag-committish) feature by providing you with a full report after a partial mutation testrun.
+Enabling `baseline`, alias `with-baseline`, saves the mutation report to a storage location such as the filesystem. The mutation report is loaded at the start of the next mutation run. Any changed source code or unit test results in a reset of the mutants affected by the change. For unchanged mutants the previous result is reused. This feature expands on the [since](#since-flag-committish) feature by providing you with a full report after a partial mutation testrun. The target of the baseline is a `committish` passed as an extra argument. The target determines which committish is used to establish or locate a baseline.
 
 The report name is based on the current branch name or the [project-info.version](#project-infoversion-committish).
 
-Set the diffing target on the command line by passing a committish with the since flag.
-Set the diffing target in the config file by setting the [since target](#sincetarget-committish) option.
 
 *\* The baseline and since features are mutually exclusive. This feature implicitly enables the [since](#since-flag-committish) feature for now.*
 
-### `baseline.enabled` &lt;`flag`&gt;
-
-Default: `null`  
-Command line: `N/A`  
-Config file: `"baseline": { "enabled": false }`
-
-Enable or disable [with-baseline](#with-baseline-flag-committish). If the enabled property is not set but the `baseline` object exists in the config file it is assumed to be enabled. Use this option to (temporarily) disable `with-baseline` without having to delete the other baseline configuration.
-
 ### `baseline.fallback-version` &lt;`string`&gt;
 
 Default: [since-target](#since-flag-committish)  
 Command line: `N/A`  
-Config file: `"baseline": { "fallback-version": 'develop' }`
+Config file: `"baseline": { "fallback-version": 'develop' }`  
 
-When [with-baseline](#with-baseline-flag-committish) is enabled and Stryker cannot find an existing report for the current branch the fallback version is used. When Stryker is still unable to find a baseline we will do a complete instead of partial testrun. The complete testrun will then be saved as the new baseline for the next mutation testrun.
+When [with-baseline](#with-baseline-committish) is enabled and Stryker cannot find an existing report for the current branch the fallback version is used. When Stryker is still unable to find a baseline we will do a complete instead of partial testrun. The complete testrun will then be saved as the new baseline for the next mutation testrun.
 
 **Example**:
 ```json
@@ -556,9 +546,9 @@ new baseline saved to: feat-2
 
 Default: `Disk`  
 Command line: `N/A`  
-Config file: `"baseline": { "provider": 'AzureFileStorage'}`
+Config file: `"baseline": { "provider": 'AzureFileStorage'}`  
 
-Sets the storage provider for the baseline used by [with-baseline](#with-baseline-flag-committish). By default this is set to disk, when the dashboard [reporter](#reporter-string) is enabled this is automatically set to Dashboard.
+Sets the storage provider for the baseline used by [with-baseline](#with-baseline-committish). By default this is set to disk, when the dashboard [reporter](#reporter-string) is enabled this is automatically set to Dashboard.
 
 Supported storage providers are:
 
@@ -601,6 +591,14 @@ Allowed permissions: `Read`, `Write`, `Create`
 
 For more information on how to configure a SAS check the [Azure documentation](https://docs.microsoft.com/en-us/azure/storage/common/storage-sas-overview).
 
+### `baseline recreate`
+
+Default: `false`
+Command line: `with-baseline main recreate`
+Config file: `N/A`
+
+Sometimes your baseline can get corrupted or out of touch with reality. In that case the baseline can be recreated using this command. This will test all mutations in your project and save the result as the new baseline.
+
 ## Troubleshooting
 
 ### `verbosity` &lt;`log-level`&gt;
@@ -620,7 +618,7 @@ All available loglevels are
 
 ### `log-to-file` &lt;`flag`&gt;
 
-Default: `false`  
+Default: `false`
 Command line: `[-L|--log-to-file]`  
 Config file: `N/A`
 

docs/stryker-in-pipeline.md

@@ -48,22 +48,39 @@ dotnet stryker --break-at 80
 Dashboard compare is very useful when running stryker in pipelines because it cuts down on the total runtime by only testing mutations that have changed compared to for example master
 The following minimal steps are needed to use dashboard compare
 
-1. Enable --with-baseline and choose the comparison target
-1. Choose a storage provider (Dashboard for public projects or Azure File Share for private projects)
+1. Enable [with-baseline](https://stryker-mutator.io/docs/stryker-net/configuration#with-baseline-committish)
+1. Choose a target branch or commit, as the source of the baseline (usually current branch in your pull request)
+1. Choose a [storage provider](https://stryker-mutator.io/docs/stryker-net/configuration#baselineprovider-string) (Dashboard for public projects or Azure File Share for private projects)
 1. Set up authentication for the chosen storage provider 
-1. Set --version to the name of the source branch (usually current branch)
 1. Set any other options needed for your chosen storage provider (see: [storage provider docs](https://stryker-mutator.io/docs/stryker-net/configuration#baselineprovider-string))
 
-Example for azure devops with dashboard storage provider:
+_Example for azure devops with dashboard storage provider:_
 ```
-dotnet stryker --with-baseline:$(System.PullRequest.TargetBranch) --dashboard-api-key $(Stryker.Dashboard.Api.Key) --version $(System.PullRequest.SourceBranch)
+dotnet stryker with-baseline $(System.PullRequest.TargetBranch) --dashboard-api-key $(Stryker.Dashboard.Api.Key) --version $(System.PullRequest.SourceBranch)
 ```
 
 ```json
 {
   "stryker-config": {
     "baseline": {
-      "provider": "Dashboard"
+      "provider": "Dashboard",
+      "fallback-version": "main"
+    }
+  }
+}
+```
+
+_Example for azure devops with azure fileshare storage provider:_
+```
+dotnet stryker with-baseline $(System.PullRequest.TargetBranch) --azure-fileshare-sas $(Stryker.Azure.Fileshare.SAS)
+```
+
+```json
+{
+  "stryker-config": {
+    "baseline": {
+      "provider": "AzureFileStorage",
+      "fallback-version": "main"
     }
   }
 }

src/Stryker.CLI/Stryker.CLI.UnitTest/ConfigBuilderTests.cs

@@ -98,7 +98,7 @@ private static Mock<IStrykerInputs> GetMockInputs()
             inputs.Setup(x => x.MutateInput).Returns(new MutateInput());
             inputs.Setup(x => x.MutationLevelInput).Returns(new MutationLevelInput());
             inputs.Setup(x => x.SinceInput).Returns(new SinceInput());
-            inputs.Setup(x => x.WithBaselineInput).Returns(new WithBaselineInput());
+            inputs.Setup(x => x.BaselineEnabledInput).Returns(new BaselineEnabledInput());
             inputs.Setup(x => x.OpenReportInput).Returns(new OpenReportInput());
             inputs.Setup(x => x.ReportersInput).Returns(new ReportersInput());
             inputs.Setup(x => x.ProjectVersionInput).Returns(new ProjectVersionInput());

src/Stryker.CLI/Stryker.CLI.UnitTest/StrykerCLITests.cs

@@ -59,7 +59,7 @@ public void ShouldDisplayInfoOnHelp()
 
                 var expected = @"Stryker: Stryker mutator for .Net
 
-Stryker mutator for .Net
+The mutation test framework for .Net
 
 Usage: Stryker [command] [options]
 
@@ -413,7 +413,7 @@ public void ShouldSupplyWithBaselineWhenPassed(params string[] argName)
 
             _strykerRunnerMock.VerifyAll();
 
-            _inputs.WithBaselineInput.SuppliedInput.Value.ShouldBeTrue();
+            _inputs.BaselineEnabledInput.SuppliedInput.Value.ShouldBeTrue();
         }
 
         [Theory]

src/Stryker.CLI/Stryker.CLI.UnitTest/filled-stryker-config.json

@@ -49,7 +49,6 @@
       "module": "cli"
     },
     "baseline": {
-      "enabled": false,
       "fallback-version": "master",
       "azure-fileshare-url": "",
       "provider": ""

src/Stryker.CLI/Stryker.CLI.UnitTest/filled-stryker-config.yaml

@@ -39,7 +39,6 @@ stryker-config:
     version: master
     module: cli
   baseline:
-    enabled: false
     fallback-version: master
     azure-fileshare-url: ''
     provider: ''

src/Stryker.CLI/Stryker.CLI.UnitTest/packages.lock.json

@@ -1609,7 +1609,7 @@
           "McMaster.Extensions.CommandLineUtils": "[4.1.1, )",
           "NuGet.Protocol": "[6.9.1, )",
           "YamlDotNet": "[15.1.2, )",
-          "stryker": "[4.0.0, )"
+          "stryker": "[4.0.1, )"
         }
       },
       "stryker": {
@@ -1638,8 +1638,8 @@
           "ShellProgressBar": "[5.2.0, )",
           "Spectre.Console": "[0.48.0, )",
           "Spectre.Console.Analyzer": "[0.48.0, )",
-          "Stryker.DataCollector": "[4.0.0, )",
-          "Stryker.RegexMutators": "[4.0.0, )",
+          "Stryker.DataCollector": "[1.0.0, )",
+          "Stryker.RegexMutators": "[1.0.0, )",
           "TestableIO.System.IO.Abstractions.Wrappers": "[20.0.28, )"
         }
       },

src/Stryker.CLI/Stryker.CLI/CommandLineConfig/CommandLineConfigReader.cs

@@ -34,6 +34,8 @@ public void RegisterInitCommand(CommandLineApplication app, IFileSystem fileSyst
             {
                 RegisterCliInputs(initCommandApp);
 
+                initCommandApp.HelpOption();
+
                 initCommandApp.OnExecute(() =>
                 {
                     _console.WriteLine($"Initializing new config file.");
@@ -68,6 +70,45 @@ public void RegisterInitCommand(CommandLineApplication app, IFileSystem fileSyst
             });
         }
 
+        public void RegisterBaselineCommand(CommandLineApplication app, IStrykerInputs inputs, string[] args, StrykerCli strykerCli)
+        {
+            app.Command("baseline", baselineCmd =>
+            {
+                baselineCmd.AddName("with-baseline");
+                RegisterCliInputs(baselineCmd);
+
+                var committishArg = baselineCmd.Argument("committish", "The committish to compare with the current HEAD. This can be a branch, tag or commit id.");
+                committishArg.IsRequired(true);
+
+                baselineCmd.HelpOption();
+
+                baselineCmd.OnExecute(() =>
+                {
+                    inputs.BaselineTargetInput.SuppliedInput = committishArg.Value;
+                    inputs.BaselineEnabledInput.SuppliedInput = true;
+                    baselineCmd.Description = "Starts a stryker run based on the results of a previous run.";
+                    return strykerCli.StartApp(inputs, args, app, this);
+                });
+
+                baselineCmd.Command("recreate", createCmd =>
+                {
+                    RegisterCliInputs(createCmd);
+
+                    createCmd.HelpOption();
+
+                    createCmd.OnExecute(() =>
+                    {
+                        inputs.BaselineTargetInput.SuppliedInput = committishArg.Value;
+                        inputs.BaselineEnabledInput.SuppliedInput = true;
+                        createCmd.Description = "Creates a new baseline by doing a full stryker run";
+                        // Enable recreate
+                        inputs.BaselineRecreateEnabledInput.SuppliedInput = true;
+                        return strykerCli.StartApp(inputs, args, app, this);
+                    });
+                });
+            });
+        }
+
         public CommandOption GetConfigFileOption(string[] args, CommandLineApplication app)
         {
             var commands = app.Parse(args);
@@ -146,8 +187,8 @@ private static void HandleSingleOrNoValue(IInput strykerInput, CommandOption cli
                     inputs.SinceTargetInput.SuppliedInput = cliInput.Value();
                     break;
 
-                case WithBaselineInput withBaselineInput:
-                    withBaselineInput.SuppliedInput = true;
+                case BaselineEnabledInput baselineEnabledInput:
+                    baselineEnabledInput.SuppliedInput = true;
                     inputs.SinceTargetInput.SuppliedInput = cliInput.Value();
                     break;
 
@@ -182,7 +223,7 @@ private void PrepareCliOptions(IStrykerInputs inputs)
             AddCliInput(inputs.MutateInput, "mutate", "m", optionType: CommandOptionType.MultipleValue, argumentHint: "glob-pattern", category: InputCategory.Mutation);
             AddCliInput(inputs.MutationLevelInput, "mutation-level", "l", category: InputCategory.Mutation);
             AddCliInput(inputs.SinceInput, "since", "", optionType: CommandOptionType.SingleOrNoValue, argumentHint: "committish", category: InputCategory.Mutation);
-            AddCliInput(inputs.WithBaselineInput, "with-baseline", "", optionType: CommandOptionType.SingleOrNoValue, argumentHint: "committish", category: InputCategory.Mutation);
+            AddCliInput(inputs.BaselineEnabledInput, "with-baseline", "", optionType: CommandOptionType.SingleOrNoValue, argumentHint: "committish", category: InputCategory.Mutation);
             // Category: Reporting
             AddCliInput(inputs.OpenReportInput, "open-report", "o", CommandOptionType.SingleOrNoValue, argumentHint: "report-type", category: InputCategory.Reporting);
             AddCliInput(inputs.ReportersInput, "reporter", "r", optionType: CommandOptionType.MultipleValue, category: InputCategory.Reporting);
@@ -195,7 +236,7 @@ private void PrepareCliOptions(IStrykerInputs inputs)
             AddCliInput(inputs.DevModeInput, "dev-mode", null, optionType: CommandOptionType.NoValue, category: InputCategory.Misc);
         }
 
-        private void RegisterCliInput(CommandLineApplication app, CliInput option)
+        public void RegisterCliInput(CommandLineApplication app, CliInput option)
         {
             var argumentHint = option.OptionType switch
             {
@@ -230,7 +271,7 @@ private CliInput AddCliOnlyInput(string argumentName, string argumentShortName,
             return cliOption;
         }
 
-        private void AddCliInput(IInput input, string argumentName, string argumentShortName,
+        public void AddCliInput(IInput input, string argumentName, string argumentShortName,
             CommandOptionType optionType = CommandOptionType.SingleValue, InputCategory category = InputCategory.Generic, string argumentHint = null)
         {
             var cliOption = new CliInput

src/Stryker.CLI/Stryker.CLI/CommandLineConfig/GroupedHelpTextGenerator.cs

@@ -8,7 +8,7 @@
 
 namespace Stryker.CLI
 {
-    [ExcludeFromCodeCoverage] // Not worth the effort to test
+    [ExcludeFromCodeCoverage(Justification = "Not worth the effort to test")]
     internal class GroupedHelpTextGenerator : DefaultHelpTextGenerator
     {
         protected override void GenerateOptions(CommandLineApplication application, TextWriter output, IReadOnlyList<CommandOption> visibleOptions, int firstColumnWidth)

src/Stryker.CLI/Stryker.CLI/FileBasedInput.cs

@@ -112,9 +112,6 @@ public class Since : IExtraData
 
     public class Baseline : IExtraData
     {
-        [JsonPropertyName("enabled")]
-        public bool? Enabled { get; init; }
-
         [JsonPropertyName("provider")]
         public string Provider { get; init; }
 

src/Stryker.CLI/Stryker.CLI/FileConfigReader.cs

@@ -33,15 +33,11 @@ public static void DeserializeConfig(string configFilePath, IStrykerInputs input
 
             if (config.Baseline is not null)
             {
-                // Baseline is implicitly enabled when the object exists in the file config
-                inputs.WithBaselineInput.SuppliedInput = config.Baseline.Enabled ?? true;
-
                 inputs.BaselineProviderInput.SuppliedInput = config.Baseline.Provider;
                 inputs.FallbackVersionInput.SuppliedInput = config.Baseline.FallbackVersion;
                 inputs.AzureFileStorageUrlInput.SuppliedInput = config.Baseline.AzureFileShareUrl;
             }
 
-
             inputs.CoverageAnalysisInput.SuppliedInput = config.CoverageAnalysis;
             inputs.DisableBailInput.SuppliedInput = config.DisableBail;
             inputs.DisableMixMutantsInput.SuppliedInput = config.DisableMixMutants;

src/Stryker.CLI/Stryker.CLI/FileConfigWriter.cs

@@ -20,7 +20,6 @@ private static FileBasedInputOuter CreateConfig(IStrykerInputs inputs)
             {
                 Baseline = new Baseline
                 {
-                    Enabled = inputs.WithBaselineInput.SuppliedInput ?? inputs.WithBaselineInput.Default,
                     Provider = inputs.BaselineProviderInput.SuppliedInput ?? inputs.BaselineProviderInput.Default,
                     FallbackVersion = inputs.FallbackVersionInput.SuppliedInput ?? inputs.FallbackVersionInput.Default,
                     AzureFileShareUrl = inputs.AzureFileStorageUrlInput.SuppliedInput ?? inputs.AzureFileStorageUrlInput.Default,

src/Stryker.CLI/Stryker.CLI/StrykerCLI.cs

@@ -50,7 +50,7 @@ public int Run(string[] args)
             {
                 Name = "Stryker",
                 FullName = "Stryker: Stryker mutator for .Net",
-                Description = "Stryker mutator for .Net",
+                Description = "The mutation test framework for .Net",
                 ExtendedHelpText = "Welcome to Stryker for .Net! Run dotnet stryker to kick off a mutation test run",
                 HelpTextGenerator = new GroupedHelpTextGenerator()
             };
@@ -61,18 +61,11 @@ public int Run(string[] args)
 
             cmdConfigReader.RegisterCommandLineOptions(app, inputs);
             cmdConfigReader.RegisterInitCommand(app, _fileSystem, inputs, args);
+            cmdConfigReader.RegisterBaselineCommand(app, inputs, args, this);
 
             app.OnExecute(() =>
             {
-                // app started
-                PrintStrykerASCIIName();
-
-                _configReader.Build(inputs, args, app, cmdConfigReader);
-                _loggingInitializer.SetupLogOptions(inputs);
-
-                PrintStrykerVersionInformationAsync();
-                RunStryker(inputs);
-                return ExitCode;
+                return StartApp(inputs, args, app, cmdConfigReader);
             });
 
             try
@@ -87,16 +80,29 @@ public int Run(string[] args)
                 {
                     Console.Error.WriteLine();
                     Console.Error.WriteLine("Did you mean this?");
-                    foreach(var match in uex.NearestMatches)
+                    foreach (var nearMatch in uex.NearestMatches)
                     {
-                        Console.Error.WriteLine("    " + match);
+                        Console.Error.WriteLine("    " + nearMatch);
                     }
                 }
 
                 return ExitCodes.OtherError;
             }
         }
 
+        internal int StartApp(IStrykerInputs inputs, string[] args, CommandLineApplication app, CommandLineConfigReader cmdConfigReader)
+        {
+            // app started
+            PrintStrykerASCIIName();
+
+            _configReader.Build(inputs, args, app, cmdConfigReader);
+            _loggingInitializer.SetupLogOptions(inputs);
+
+            PrintStrykerVersionInformationAsync();
+            RunStryker(inputs);
+            return ExitCode;
+        }
+
         private void RunStryker(IStrykerInputs inputs)
         {
             var result = _stryker.RunMutationTest(inputs, ApplicationLogging.LoggerFactory);