Marketing Text AI Tests Added #1504
Conversation
…res/521281-entity-text-tests
| "brief": "", | ||
| "description": "Tests for Entity Text", | ||
| "version": "25.0.0.0", | ||
| "privacyStatement": "https://go.microsoft.com/fwlink/?LinkId=724011", |
|
|
||
| codeunit 134640 "Mkt Text Accuracy AIT" | ||
| { | ||
| Subtype = Test; |
There was a problem hiding this comment.
Check if TestPermissions needs to be set to disabled.
|
|
||
| codeunit 134641 "Mkt Text RedTeam AIT" | ||
| { | ||
| Subtype = Test; |
There was a problem hiding this comment.
Check if TestPermissions needs to be set to disabled.
| using System.TestTools.AITestToolkit; | ||
| using System.Text; | ||
|
|
||
| codeunit 134640 "Mkt Text Accuracy AIT" |
There was a problem hiding this comment.
Can we move the tests to a folder for the time being to distinguish between AIT Tests vs regular tests?
In the near future, we maybe introduce a separate codeunit number range.
The test app is also missing the Suite Configuration (.xml) file.
| AITestContext.SetTestOutput(PrepareOutput(BuildFactsText(Facts, Category), Category, Response, Enum::"Entity Text Tone"::Inspiring, Format)); | ||
| end; | ||
|
|
||
| [Test] |
There was a problem hiding this comment.
Tests are missing comments:
[Scenario]
[Given]
[When]
[Then]
Did we stop using these?
| local procedure PrepareOutput(Facts: Text; Response: Text; Tone: Enum "Entity Text Tone"; Format: Enum "Entity Text Format"): Text; | ||
| var | ||
| Context: Text; | ||
| FormatLbl: Label '{"question": "METAPROMPT %1", "answer": "%2", "context": "%3", "ground_truth": "%4", "tone": "%5", "format" : "%6"}', Comment = '%1= Input Prompt, %2= Response Prompt, %3= Context, %4= Ground Truth, %5= Tone, %6= Format'; |
There was a problem hiding this comment.
Consider using 130462 "Test Output Json" instead of creating JSON as a string. Double quotes are not being escaped while setting the json property values.
| begin | ||
| NewLineChar := 10; | ||
| Context := 'Here are some facts about the item:<br /><br />' + Facts.Replace(NewLineChar, EncodedNewlineTok); | ||
| exit(StrSubstNo(FormatLbl, Facts.Replace(NewLineChar, EncodedNewlineTok), Response.Replace(NewLineChar, EncodedNewlineTok), Context, '', Tone.Names.Get(Tone.Ordinals.IndexOf(Tone.AsInteger())), Format.Names.Get(Format.Ordinals.IndexOf(Format.AsInteger())))); |
There was a problem hiding this comment.
Can we directly get the caption from the Tone enum?
| InputPrompt: Text; | ||
| Response: Text; | ||
| begin | ||
| InputJson.ReadFrom(AITestContext.GetInput().ValueAsText()); |
There was a problem hiding this comment.
The properties can be directly read using AITTestContext.GetInput() -> (returns Codeunit "Test Input Json")
AITestContext.GetInput().Element('inputPrompt').ValueAsText()
Can we use 'question' in the dataset instead of 'inputPrompt'? This would standardize the term usage across all the datasets.
And then you can also use AITestContext.GetQuestion().ValueAsText() to read the input.
| NewLineChar: Char; | ||
| begin | ||
| NewLineChar := 10; | ||
| Context := 'Here are some facts about the item:<br /><br />' + Facts.Replace(NewLineChar, EncodedNewlineTok) + '<br /> This is in the category of: ' + Category.Replace(NewLineChar, EncodedNewlineTok); |
There was a problem hiding this comment.
Labels? If this shouldn't be translated, shouldn't than FormatLbl be locked?
| InputPrompt := AITestContext.GetInput().ValueAsText(); | ||
| Facts := LineInputToDictionary(InputPrompt); | ||
| Format := Enum::"Entity Text Format"::Brief; | ||
| Response := EntityTextCod.GenerateText(Facts, Enum::"Entity Text Tone"::Formal, Format, Enum::"Entity Text Emphasis"::None); | ||
| AITestContext.SetTestOutput(PrepareOutput(BuildFactsText(Facts, Category), Category, Response, Enum::"Entity Text Tone"::Formal, Format)); |
There was a problem hiding this comment.
This code is repeated in all procedures, and its difference is minimal:
Can you just make one local procedure with parameter for Enum::"Entity Text Format" and parameter for Enum::"Entity Text Tone"
Than you won't have 5 times declaration of 7 variables and use only one liners
| InputPrompt := InputToken.AsValue().AsText(); | ||
|
|
||
| Format := Enum::"Entity Text Format"::Tagline; | ||
| AddFacts(Facts, InputPrompt, Format); | ||
| Response := EntityTextCod.GenerateText(Facts, Enum::"Entity Text Tone"::Inspiring, Format, Enum::"Entity Text Emphasis"::None); | ||
| AITestContext.SetTestOutput(PrepareOutput(BuildFactsText(Facts), Response, Enum::"Entity Text Tone"::Inspiring, Format)); |
There was a problem hiding this comment.
Here is the same code repeated again, and in the previous codeunit also. I would consider than helper procedure with enum parameters described in previous comment, so you would call it from all test codeunits.
| NewLineChar: Char; | ||
| begin | ||
| NewLineChar := 10; | ||
| Context := 'Here are some facts about the item:<br /><br />' + Facts.Replace(NewLineChar, EncodedNewlineTok); |
Summary
AI Test for Marketing Text added.
Fixes # AB#521281