-
Notifications
You must be signed in to change notification settings - Fork 17
Specification Test Nonconformity & Notes
Here we document any nonconformity in the specification tests.
Expression:
minimum Decimal
Expected result:
-99999999999999999999.99999999
Actual output:
-79228162514264337593543950335
Rationale:
Per the spec:
Note that if implementations support larger and/or more precise values than the minimum required precision and scale for Decimal, DateTime, and Time values, they will return the minimum representable decimal for the implementation.
Since the SDK uses the C# decimal type to represent the ELM System.Decimal type, we can represent smaller decimals than the minimum required by the specification.
Action:
Test skipped
Recommendation:
Change the expression in the test to be minimum Decimal <= -99999999999999999999.99999999 with an expected output of true
Expression:
maximum Decimal
Expected result:
99999999999999999999.99999999
Actual output:
79228162514264337593543950335
Rationale:
Per the spec:
Note that if implementations support larger and/or more precise values than the minimum required precision and scale for Decimal, DateTime, and Time values, they will return the minimum representable decimal for the implementation.
Since the SDK uses the C# decimal type to represent the ELM System.Decimal type, we can represent larger decimals than the maximum required by the specification.
Action:
Test skipped
Recommendation:
Change the expression in the test to be maximum Decimal >= 99999999999999999999.99999999 with an expected output of true
Expression:
1.0 'cm' * 2.0 'cm'
Expected result:
2.0'cm2'
Actual output:
Rationale:
Unit arithmetic is not yet supported. Quantities used in arithmetic expressions must have the default UCUM unit of 1.
Action:
Test skipped
Affected tests:
- Power2ToNeg2
- DecimalOneStep
- DecimalPosOneStep
- DecimalNegOneStep
- DecimalTwoStep
- DecimalPosTwoStep
- DecimalNegTwoStep
- DecimalTenStep
- DecimalPosTenStep
- DecimalNegTenStep
Example expression:
Power(2, -2)
Expected result:
0.25
Actual output:
0
Rationale:
As converted to ELM, the result type of this expression is defined to be System.Integer. Obeying this result type requires that we truncate from 0.25, the mathematically correct result, to 0.
Action:
Test skipped
Recommendation:
The spec discusses behavior for mixed argument types, but does not explicitly state that when Power is used with two Integer arguments that the result type should also be an Integer and decimal portions will be truncated. Given the behavior of the cql-to-elm reference implementation, that behavior is established. This contradicts the expected result of the test. Either the translator has to be updated to match the test, or the test should have an expected outcome of 0. In both cases, the spec should be updated to clarify this.
Expression:
Round(0.5)
Expected result:
1.0
Actual output:
0
Rationale:
The spec uses the language "traditional round" to describe the rounding method. We use the default rounding strategy which is described in the language as "to nearest." For reference, .NET has devoted significant thought to this here.
This test contradicts the others when using the "to nearest" rounding.
Recommendation:
Clarify the rounding strategy, or fix the expected output of this test.
See Round0D5
See Multiply1CMBy2CM
Expression:
1'cm' ~ 0.01'm'
Expected result:
true
Actual output:
false
Rationale:
Quantity comparison of non-same units is not yet supported. Quantities used in comparison expressions must have the same unit.
Action:
Test skipped
See EquivEqCM1M01
Example expression (DateTimeDurationBetweenYear):
Output of test DateTimeDay is expected to be:
@2003-10-29T
Actual result: The reference implementation of cql-to-elm accepts these as valid ISO-8601 dates.
Expected result: An error that this is an invalid literal.
Rationale: The ISO-8601 standard does not allow dates to finish with a trailing T. These strings are invalid and should not be tolerated by any implementation.
Affected tests
- DateTimeDurationBetweenYear
- DateTimeDurationBetweenUncertainInterval
- DateTimeDurationBetweenUncertainInterval2
- DateTimeDurationBetweenUncertainAdd
- DateTimeDurationBetweenUncertainSubtract
- DateTimeDurationBetweenUncertainMultiply
- DateTimeDurationBetweenUncertainDiv
- DateTimeDurationBetweenMonthUncertain2
- DateTimeUncertain
Example expression (DateTimeDurationBetweenYear):
years between DateTime(2005) and DateTime(2010)
Expected result:
Interval[4,5]
Actual output:
5
Rationale:
Per the spec:
The result of this operation is always an integer; any fractional periods are dropped.
Action:
Tests skipped
Recommendation:
Change the expected output of these tests to be a System.Integer, as indicated by the spec; alternatively, update the specification to be clear that the Duration operator returns an interval (to account for uncertainty), where results that can be certainly computed will be expressed as point intervals. The latter would be a breaking change (to existing CQL measures and libraries) and should be done with caution.
Example tests
- DateTimeAdd2YearsByDays
Recommendation:
Per the spec:
For partial date/time values where the time-valued quantity is more precise than the partial date/time, the operation is performed by converting the time-based quantity to the most precise value specified in first argument (truncating any resulting decimal portion) and then adding it to the first argument.
The method for converting 730 days to years would be to divide 730 by 365 days (or to multiply by 1/365). Floating point math computations can produce a value that is not exactly 2.0 but is rather something like 1.99999999986. In this case, if the implementation truncates the decimal portion after the imprecise floating point mathematics, it could result in 1 year instead of 2, as expected by the test (DateTime(2014) + 730 days = @2016).
This SDK rounds all conversions to a precision of 8 decimal places, which will round the imprecise sub-2.0 floating point value to 2.0, thus satisfying this test.
Our recommendation is to update the spec to formalize this rounding process.
Expression:
Interval(null, 5] meets after Interval[11, null)
Expected result:
false
Actual output:
null
Rationale:
This test has the same basic pattern as another test, TestIntersectNull, whose expression is Interval[1, 10] intersect Interval[5, null) and whose expected result is null because having an open null endpoint makes this operation undefined. The meets after operator is specifically concerned with the low value of the first operand and the high value of the second operand, as per the spec:
the meets after operator returns true if the first interval starts immediately after the second interval ends.
Action:
Test skipped
Recommendation:
Interval tests where any of the operands contain a low or high value which is both null and open should behave the same way.
Expression:
Interval[@T12:00:00.000, @T21:59:59.999] properly includes @T12:00:00.000
Expected result:
false
Actual output:
false
Recommendation:
The specification is unclear about this. It says:
For the point overload, this operator returns true if the interval contains (i.e. includes) the point, and the interval is not a unit interval containing only the point.
The language in the spec should be clarified and reworded, e.g.:
For the point overload, this operator returns true if the interval contains (i.e. includes) the point and the point is not exactly equal to either the start or the end of the interval. For point intervals whose value is the point, this operator returns
false.
Expression:
({ DateTime(2012, 10, 5, 10), DateTime(2012, 1, 1), DateTime(2012, 1, 1, 12), DateTime(2012, 10, 5) }) S sort asc
Expected result:
{DateTime(2012, 1, 1), DateTime(2012, 1, 1, 12), DateTime(2012, 10, 5), DateTime(2012, 10, 5, 10)}
Actual output:
{DateTime(2012, 1, 1, 12), DateTime(2012, 1, 1), DateTime(2012, 10, 5), DateTime(2012, 10, 5, 10)}
Rationale:
This test is invalid. Comparing date times with different precision is defined to return null, and that behavior is tested consistently elsewhere in the tests, and is explicitly stated in the greater than operator language:
For Date, DateTime, and Time values, the comparison is performed by considering each precision in order, beginning with years (or hours for time values). If the values are the same, comparison proceeds to the next precision; if the first value is greater than the second, the result is true; if the first value is less than the second, the result is false; if one input has a value for the precision and the other does not, the comparison stops and the result is null; if neither input has a value for the precision or the last precision has been reached, the comparison stops and the result is false. For example:
While sorting, we use the same comparison logic as we do for other logic per the spec, because sorting is implemented automatically by C# using the OrderBy method. During sort, when the comparison is null, we allow the ordering of the resulting list to be arbitrary, which explains the inconsistency in the result where in one case, the value with higher precision is sorted first and later, the value with higher precision is sorted second.
Although it would be possible to implement comparers explicitly used for sorting that unambiguously sort higher precision DateTime values as after lower precision ones when all other values are equal, this contradicts the intent of the spec in other places.
The language in the author's guide does not discuss this situation, nor is there a sort operator in Appendix B.
Action:
Test skipped
Recommendation:
Sort tests should not contain comparisons of DateTime with different precision because the behavior is explicitly undefined by the specification. The specification is clear in multiple places that comparing DateTime with different precision returns null as an analogue of undefined. Thus, the resulting order of this sort does not have only one definitively correct result but several possible ones. The only assertion that is invariant in this test is that in the resulting list, the indices of both DateTime(2012, 1, 1) and DateTime(2012, 1, 1, 12) must be less than the indices of DateTime(2012, 10, 5) and DateTime(2012, 10, 5, 10).
See SortDatesAsc
Affected tests:
- EquivalentABCAnd123
- Equivalent123AndABC
- Equivalent123AndString123
- NotEqualABCAnd123
- NotEqual123AndABC
- NotEqual123AndString123
- ProperIncludes123AndEmpty
- ProperIncludedInEmptyAnd123
- Union123AndEmpty
Example expression:
{ 'a', 'b', 'c' } ~ { 1, 2, 3 }
Expected result:
false
Actual output:
Translation error
Rationale:
The reference CQL-to-ELM implementation produces this error on this expression:
Could not resolve call to operator Equivalent with signature (list<System.String>,list<System.Integer>).
This is the correct error. Comparing two lists of disparate element types is an error.
Action:
Test skipped
Recommendation:
Remove these tests.
Affected tests:
- ProperContainsNullRightFalse
- ProperContainsNullRightTrue
- ProperInNullRightFalse
- ProperInNullRightTrue
- ProperlyIncludesNullLeft
Example expression:
{'s', 'u', 'n'} properly includes null
Expected result:
false
Actual output:
null
Rationale:
Appendix B explicitly states:
If either argument is null, the result is null.
Action:
Test skipped
Recommendation:
Change the expected result of these tests to null.
Affected tests:
- IndexerNullNull
- IndexerANull
- IndexerNull1String
- IndexerAB0
- IndexerAB1
- IndexerAB2
- IndexerABNeg1
Example expression:
Indexer('a', null)
Comment:
The only problem with this test, and several others is that functional usage is not documented in the spec.
Recommendation:
Either add functional usage to Indexer or document it in the specification. Indexer's functional usage is currently supported in the cql-to-elm reference implementation.