Manual Testing ISIS Reflectometry

[warunawickramasingha]

You have been assigned manual testing. The hope is to catch as many problems with the code before release, so it would be great if you can take some time to give a serious test to your assigned area. Thank you!!

The general guide to manual testing:

• The tests must be performed on the installer versions of the final release candidate. Not on local compiled code.
• Serious errors involving loss of functionality, crashes etc. should be raised
  as issues with the current release as a milestone and an email sent to the project manager immediately.
• Minor and cosmetic issues should be raised as issues against the forthcoming
  releases.
• First try things that should work, then try to break Mantid, e.g. entering invalid values, unexpected characters etc.
• Don't spend more than a few hours on the testing as fatigue will kick in.
• If you find errors in the documentation, please correct them.
• Comment against this ticket the OS environment you are testing against.
• Disable Usage reporting before you start testing
• Close the this issue once you are done.
• Time how long this manual test takes for you to do and leave a comment about it in this issue.

Specific Notes:

http://developer.mantidproject.org/Testing/ReflectometryGUI/ReflectometryGUITests.html

[RichardWaiteSTFC]

Testing took ~ 1 hour

Found a bug (not a regression) in preview tab

• Update reflectometry preview tab testing instructions: reduction not re-run when the width of the signal box is changed #38658

I also suspect there are also a couple of inaccuracies in the testing instructions:

• Step 4 in Saving a batch: the instructions state Note that you should not be prompted about discarding unsaved changes. but a warning did appear.

• Step 10 in Save tab: the instructions stat Check that option Additional columns (includes Q resolution) is ticked but disabled, as it is not applicable. - in actuality the box is not ticked (though is disabled) - it doesn't sound like it matter given it's not applicable.

@rbauststfc could you confirm these are not bugs before I raise an issue for updating the test instructions? Thanks!

[rbauststfc]

Thanks @RichardWaiteSTFC, I've added a comment to the issue you opened as that one isn't actually a bug, it just needs some updates to the manual testing instructions.

On the other two, I can't replicate either behaviour I'm afraid, when I test on IDAaaS I get the same behaviour as the testing instructions describe. Have you been able to replicate what you saw during testing?

[RichardWaiteSTFC]

Thanks @RichardWaiteSTFC, I've added a comment to the issue you opened as that one isn't actually a bug, it just needs some updates to the manual testing instructions.

On the other two, I can't replicate either behaviour I'm afraid, when I test on IDAaaS I get the same behaviour as the testing instructions describe. Have you been able to replicate what you saw during testing?

Thanks for checking, I've checked again on v6.11 and nightly and I can't re[produce these on a fresh instance of workbench either...perhaps I hadn't reset some settings, or followed the instructions in a slightly different order? Sorry about that!

[rbauststfc]

No worries, thanks for taking another look!

[RichardWaiteSTFC]

Closing as no issues to address - apart from change to testing docs which I guess can be done under #38658