Issues addressed in the Fourth Public Working Draft of the Formula Specification - 2008-02-14 ============================================================================================= The following individual issues were raised with the FWG based on the third public working draft and the WG responses are summarised in the list below. Much of this feedback emerged from the process of creating the conformance suite test cases that the FWG is developing for release if/when the specifications reach candidate recommendation status. More detailed information is contained in the revision histories of the specification documents themselves. 1. Herm Fischer and cgh_chen (xbrl-dev mailing list): formula.xsd schema validation error on occDimension element. RESOLVED: This has been fixed by adding a global type declaration, as suggested by Cliff Binstock. 2. Roland Hommes, Herm Fischer and cgh_chen: raised query about absence of bindEmpty and bindAsSequence attributes on general variables. These were dropped as recorded in the revision for 29 November 2007 based on feedback from Victor Morilla who felt that it was peculiar to enable a formula to bind a set of fact variables and then to produce a range of different output facts, depending on which item in a sequence that a general variable's XPath expression has evaluated to, the general variable gets bound to. RESOLVED: The bindAsSequence attribute is reinstated for general variables to support the range of use cases identified by Herm. The bindEmpty attribute will not be reinstated. The kind of perverse formulae that Victor envisaged will be possible but the FWG deemed that this was acceptable given the value of allowing binding to individual items rather than to the entire sequence produced by evaluation of the XPath expression for general variables. Also, see the resolution to issue 63. 3. cgh_chen, Roland Hommes, Pablo Navarro Salvador and Fujitsu all independently noticed that the boolean filter specification refers to a variable:filter arc that is not defined. RESOLVED: This error resulted from not keeping the terminology of the boolean filter specification in line with the variables specification. The up to date arc element name (variable:variableFilter) is now used in the Boolean filter specification. 4. Herm Fischer and Cliff Binstock found unintended consequences of the regex constraint on the variable:expression data type. RESOLVED: The old regular expression has been replaced by one from Cliff that avoids the unintended consequences regarding the formatting of regular expressions. 5. Pablo Navarro Salvador: formula:parameter should be variable parameter in the variables specification. RESOLVED: This has been fixed in the single place where it occured due to a copy-paste error. 6. Pablo Navarro Salvador: Should change "the how the" in the variable specification section on aspect models to "how the". RESOLVED: The correction has been made. 7. Pablo Navarro Salvador: Concept filter specification: There is a missing hyperlink to the normative schema from the section on the concept name filter. This is actually an inconsistency in the hyperlinking of the specification. Some sections have the element name hyperlinked and some hyperlink the text "normative schema". RESOLVED: This inconsistency has been eliminated by hyperlinking the element name only. 8. Pablo Navarro Salvador: Should the concept balance filter take the numeric type of the concept into account? e.g.:xfi:is-numeric(.) and (xfi:balance(xfi:concept(.)) = '#balance') RESOLVED: The change is redundant given the XBRL 2.1 specification and so has not been made. 9. Pablo Navarro Salvador: Value filters: suggestions: 9a) Should we also have a decimal filter? The FWG agreed earlier that one did not make sense from a formula authoring perspective given the variation in meaning of a decimal attribute depending on the value reported for a fact in the instance being processed. No further action required. RESOLVED: FWG agreed to defer this kind of new filter to a future filter specification should it be identified as useful for a particular set of use cases. 9b) The xfi:precision function throws an error if the fact is not numeric. This will cause us to have problems when precision filtering all of the facts in the instance because errors will be thrown up during filtering when they MUST not halt the execution of the formula if the formula is to work properly. The implied XPath expression should test if the fact is numeric before applying the test for precision. RESOLVED: The suggestion has been implemented. 10. Pablo Navarro Salvador: Segment-scenario specification namespace should use the same structure as the namespaces of the other filter specifications. RESOLVED: FWG changed the namespace as Pablo has suggested. 11. Pablo Navarro Salvador: There appears to be a missing implied XPath expression in the definition of the dimension member filter. RESOLVED: There is actually no missing XPath expression. The redundant text has been deleted. 12. Geoff Shuetrim. The explanation of the filter linkrole and its usage is not sufficiently accurate. RESOLVED: See the resolution to issue 16. 13. Herm Fischer: The term "associated with" is too loose when used with dimensions and segments/scenarios. RESOLVED: The offending paragraphs have been deleted. 14. Herm Fischer: SHould we have an error code for when the output concept QName not matching a concept in the DTS that supports the output instance? RESOLVED: This kind of problem, at worst will be detected when doing XBRL validation of the output instance. While that is relatively late, again this falls into the class of situations where early detection of such problems cannot be comprehensive (because output concept QNames are dynamically determined at execution time in some cases). Where this kind of situation exists, the specs are consistently trying to leave problem detection up to the post processing output instance validation step. This reduces the barriers to implementation by leaning heavily on existing XBRL capabilities while not preventing software vendors from doing their own checks for coherence that go beyond those demanded by the specifications. No changes have been made to the specifications as a result. 15. Herm Fischer: Should dimensionMember filters allow more than one member element? In dimensionMember, the member element is maxoccurs defaulted to 1, but that seems inconsistent with explicitDimension allowing any number of filter-member arcs (which maybe should be member constructs instead?), or conceptName allowing any number of names, etc. RESOLVED: Changes have been made in the dimension filter specification to effect this suggestion. See the response to issue 21. 16. Herm Fischer: The filter linkrole is incorrectly associated with linkroles for domain-member relationships. RESOLVED: s that the filter linkrole is identifying meaningful domains of permitted domain members for the filter dimension on explicit We have corrected the erroneous association of the filter linkrole with the linkrole on the extended links contain arcs that express domain-member relationships and instead associated the filter linkrole with the linkrole on the extended links that contain arcs that express has-hypercube relationships between primary items and hypercubes. 17. Herm Fischer: We need a formal error QName when an arc has not meaning as per the formula specification because its source or target are not what would enable it to define a relationship given its arcrole value. RESOLVED: We have taken care not to rule out using the arcrole value with other kinds of sources or targets to define variant relationships. Treating the situations described as errors blocks such an extension path and places a somewhat higher burden on software seeking to implement the formula specifications. That said, software remains free to detect and respond to such situations. No changes have been made to the specifications as a result. 18. Victor Morilla: Default use attribute values on attribute declarations are "optional". The normative schemas are written assuming that the default value is "required". RESOLVED: This has been addressed in all of the core schemas now. This affects: Dimension filters: childrenOnly and inclusive attributes; Concept filters: periodType and balance attributes; Formula: dimension attribute on the occDimension aspect rule; Relative filter: variable attribute; and Tuple filters: variable attribute on sibling and location filters. 19. Victor Morilla and Herm Fischer: Make the dimensionMember filter able to specify multiple members. RESOLVED: See the resolution to issue 21. 20. Herm Fischer: Make the member specification mechanism in the explicitDimension filter able to dynamically determine the member rather than statically express it using XLink relationships. RESOLVED: See the resolution for issue 21. 21: Victor Morilla: The member specification mechanism in dimension-member filters could be simplified by allowing members to be specified by the QName of the fact variable whose dimension value is to be the member. RESOLVED: Issues 19, 20 and 21 are inter-related. We have resolved all of them by: a) merging the dimensionMember and explicitDimension filters into the one filter; b) allowing the member element to occur multiple times in the one filter; c) cutting the XLink model for specifying the member for explicit dimension filters, instead using the member element approach as used in the dimensionMember filter; and d) extending the member element content model to also include a variable QName method for specifying the member. 22. cgh_chen: The variables specification defines input parameter order for custom function declarations in terms of document order but the normative schema also includes an order attribute on the input element for custom function declarations. Which should be used? RESOLVED: The order attribute is obsolete and should have been removed earlier. This has now been corrected. 23. Herm Fischer: The design of the existence and value assertion tests ignores the fact that a variable set with no fact variables still evaluates once. RESOLVED: Victor Morilla removed the constraint on the number of fact variables of existence and value assertions and its corresponding error (emptyVariableSet). This way, existence and value assertions with no fact variables associated are fired once, as defined by the variable specification. This feature can be used, for instance, to express constraints on parameters. 24. Geoff Shuetrim. The explanation of the df:qname element in dimension filters was not aligned to the explanation of the same kind of element in the concept name filter. RESOLVED: The dimension filter specification has been adjusted to achieve this alignment. 25. Geoff Shuetrim. The concept name filter does not state explicitly that the terms making up the implied XPath expression are to be combined using the XPath "or" operator. RESOLVED: This error has been addressed. 26. Geoff Shuetrim. Custom function signatures are required to specify input and output data types as QNames. This is overly restrictive. RESOLVED: We have now relaxed the requirement that data types for function signatures be expressed as QNames given the need to deal in sequences. Instead they can be any string value according to the normative schema. 27. Geoff Shuetrim. The hyperlink to the documentation conventions in the variable specification is not correct. RESOLVED: This has been fixed. 28. Fujitsu. A range of functions are not defined in the XBRL Functions 1.0 specification RESOLVED: These will be added to the function registry that has now been defined. 29. Fujitsu. The formula and validation specifications say nothing about the processing model for formulae and assertions - should the output of processing formulae be considered when testing assertions? RESOLVED: The processing model does not define any such sequencing of evaluations or using one set of formula outputs as input into an assertion testing process. Such processing is possible but, at this stage, the implementation is application/usage pattern dependent. The suggested creation of new resources may be a feature of additional specifications that support specific usage patterns such as the one envisaged in the feedback from Fujitsu but this will not be done for the existing suite of specifications. 30. Fujitsu. The scenario filter resource omits the mixed="true" attributes. RESOLVED: This has been corrected. 31. Fujitsu. The date.time.model needs a mixed content model to be used in period filter resources but it does not have a mixed content model. The same problem applies to the match.model data type in the match-filters normative schema. RESOLVED: This has been corrected. 32. Fujitsu. The use of XLink in explicitDimension filters is excessively indirect. This should be simplified as was done for the concept name filter. RESOLVED: This has been done. See the response to issue 21. 33 Fujitsu. The custom attribute creation rule in formulae was deleted prior to PWD 3 but the formula:attribute element was not deleted from the normative schema. RESOLVED: This has been corrected. 34. Fujitsu. The enumeration for the balance attribute in the concept balance filter is not aligned to the wording of the specification. RESOLVED: This has been corrected. 35. Fujitsu. The variable evaluation order is not defined in the processing model. Fujitsu suggested that it should be based on the order of relationships to variables in a variable set. PROPOSED RESPONSE: The suggestion was a feature of the second PWD and the FWG decided to revert to a model in which the evaluation order was determined by evaluating dependencies among the variables based upon their implied XPath expressions. A sentence has been added to the variable evaluation section of the variable specification to clarify this issue. It states: "Applications are responsible for determining an evaluation order for variables in a variable set that ensures that the variable dependencies for each variable in the variable set are to variables that have already been evaluated." 36. Fujitsu. xs:QName() in the implied XPath expressions should be changed to fn:QName() through the specifications because the xs:QName() function is not defined in the XQuery and XPath functions specification (http://www.w3.org/TR/xpath-functions/). RESOLVED: The suggestion has been adopted although note that there is a constructor function xs:QName() (http://www.w3.org/TR/xpath-functions/#constructor-qname-notation). The limitations on this specific constructor however, indicate that the suggested function choice is more suitable. 37. Fujitsu. If filters that can cover the same aspect are associated with a variable and with a variable set that the variable is a member of, and if both filters do cover the aspect, is it right that filter associated with the variable directly overrides the filter associated with the variable set? RESOLVED: No changes are required to the specifications. First, filters associated directly with a variable set do not cover any aspects. See the section on variable-set-filter relationships in the variables specification. Second, there is no notion of filters over-riding eachother. They all apply. 38. Fujitsu. A number of variable resources can be related to one source with one variable-set relationship arc, if those variable resources have the same xlink:label value. The spec should say that case is an error or show the selection criteria. PROPOSED RESPONSE: This is only a problem if you rely on the order attribute to determine the evaluation order of the variables in the variable set. This is not the case as can be seen in the response to issue 35. This interpretation of the problem has been confirmed with Fujitsu. 39. Fujitsu. Clarification is needed regarding the numeric items with precision=0. The XBRL base spec does not define v-equality where precision is 0. But Example 5 in the consistency assertion specification shows the v-equal result is 'Satisfied'. The reason why the result is 'Satisfied' or the rule that a Formula processor handles it should be explained. RESOLVED: In cases where we are dealing with zero precision facts, consistency cannot be verified, and so the assertion is now not evaluated. 40. Fujitsu. Clarification is needed regarding the XBRL function: xfi:datatype-is-a-restriction-of() because data types can be an extension of a restriction and this creates ambiguity. PROPOSED RESPONSE: The filter has been modified to avoid these kinds of ambiguities by just testing for whether a data type is derived from another datatype. This simplification also eliminates the need for an XBRL function xfi:datatype-is-derived-from() because an XPath 2.0 element test can be used instead as suggested by Mark Goodhand. 41. Fujitsu. The value filter specification indicates that all value filters can cover the entity aspect. Is this appropriate. RESOLVED: No. It is a copy-paste error in the creation of the specification and has been corrected. 42. Fujitsu. The value filter and the general filter imply and identical XPath expression. What is the difference between them? RESOLVED: The FWG agreed to remove the value filter and to shift the value filter examples to the general filter. 43. Fujitsu. The implied XPath expression for the precision filter needs to be modified to handle nil valued facts without producing an error. RESOLVED. The XPath expression has been changed to include a test for facts that are nilled and to return a false response for such facts. 44. Geoff Shuetrim. The implied XPath expression for the precision filter should use a value comparison operator instead of a general comparison operator. RESOLVED: The operator has been changed to ge rather than >= 45. Fujitsu. The implied XPath expression for the precision filter would fail when the inferred precision is infinite. RESOLVED: The function 1.0 specification of the xfi:precision function returns a value of INF if the implied precision is infinite. This is now tested for explicitly in the implied XPath expression for the precision filter. Also the implied XPath expression now tests for infinite minimum precision (rejecting all facts) and it tests for fraction item types explicitly. The implied XPath expression has been adjusted to make sure that an error is always thrown for non- numeric values of the minimum allowed precision other than INF. 46. Fujitsu. How do precision filters work if the XPath expression in the minimum attribute evaluates to 'INF'? RESOLVED: See resolution to 45. 47. Fujitsu. The function signatures for the xs:dateTime() function are not right in the period filter specification. RESOLVED: The functions have been changed to fn:dateTime() calls as suggested by Fujitsu. 48. Fujitsu. Source attributes on formulae must not specify a variable that has evaluated to a tuple because tuples do not have complete aspect models. An error should be required to be thrown in such circumstances. RESOLVED: If a source identifies a fact variable that has evaluated to a tuple then most aspects will not have a SAV defined for them. However, we cannot predict the kind of aspect models that might be used in the future and whether those aspect models might define aspects for tuples. For example, in the existing specifications, two aspects are defined for tuples, the location and the concept aspect. While tuples are not produced by formulae and while formulae do not support aspect rules for the location aspect, we do not want to lock ourselves out of being able to draw on tuple aspects in the future. The error code will not be added. 49. Fujitsu. For OCC dimension rules, an error should be required to be thrown if the specified dimension does not permit the specified member as a dimension value for the fact being output by the formula. RESOLVED: This kind of error will be detected by XBRL dimension validation of the instance that contains the facts produced by the formula. In many places in the existing suite of specifications, we have chosen to leverage such validation rather than impose error checking that partially or fully detects features of formulae that will produce nonsense output. Deviating from this approach in this area would be out of keeping with the approach used throughout the rest of the specifications. On these grounds, no additional error code will be added to the specification. 50. Fujitsu. There is a duplicate row in the value expression example table in the formula specification. RESOLVED: This has been deleted. 51. Fujitsu. The example in the general filter specification is confusing in relation to whether eg:custom is an attribute or an element. RESOLVED: The formatting has been changed to reflect the fact that the eg:custom attribute is an attribute and not an element. 52. Geoff Shuetrim. The example in the general filter specification uses general comparison operators where it should use value comparison operators. RESOLVED: This has been changed. 53. Fujitsu. There is a typo in the implied XPath expression for the match filter for the concept aspect. RESOLVED: The typo has been fixed as suggested. 54. Fujitsu. xs:boolean('true') should be used in the examples instead of true for the scenario and segment filters and for the typed dimension filter. RESOLVED: The examples have been corrected as suggested. 55. Fujitsu. The period-instant filter example is incorrect for the case where the time has been omitted. RESOLVED: The correction has been made in the specification. 56. Fujitsu. In the consistency assertions specification, example 3 should indicate that setting the absolute acceptance radius to 100 results in an acceptance radius of 100 instead of undefined. RESOLVED: The correction has been made in the specification. 57. Fujitsu. The consistency assertion filter example relating to nil value combinations has a typo. RESOLVED: The typo has been corrected. 58. Fujitsu. Change the reference to the eg:pureItemType to the correct xbrli:pureItemType in the concept data-type filter example. RESOLVED: Done as suggested. 59. Fujitsu. Change the reference to the eg:pureItemType to the correct xbrli:pureItemType in the concept data-type filter example. RESOLVED: Done as suggested. 60. Fujitsu. Align the arcrole values in the generic label and reference specifications with the values in the normative schemas. RESOLVED: Done as suggested. 61. Fujitsu. There are errors in the value and precision filter examples relating to gt vs ge and lt vs le interpretation. There is also an XPath node path error in the value filter example. RESOLVED: These have been corrected. 62. Roland Hommes. The omit element in the formula normative schema does not have any content in its complex type declaration. RESOLVED: That is intended. The element is empty. It is not an XML Schema error. 63. Geoff Shuetrim. The sections on general and fact variables are do not give any context about the bindEmpty and bindAsSequence attributes, leaving their purpose unclear in the minds of readers. RESOLVED: Added paragraphs to the sections on general variables and fact variables pointing to the explanations of the attributes impacting on how they bind when they are evaluated depending on the bindAsSequence and bindEmpty attributes. 64. Geoff Shuetrim. Grammatical changes are required to the explanation of consistency assertion formula relationships. RESOLVED: The necessary editing has been completed. 65. Geoff Shuetrim. There are missing RFC2119 tags around a MUST NOT statement. RESOLVED: The tags have been added. 66. Victor Morilla. Axis values in the first example need to be bought into line with the normative schema. RESOLVED: Errors in the example have been corrected. 67. Herm Fischer. Should the specification of the unit aspect rule include an inappropriateAugment error code in the same way as has been done for the occ aspect rules? RESOLVED: The error code is not applicable because there is no notion of a first vs second or third unit aspect rules whereas there is a possibility of there being several OCC dimension aspect rules and the augment attribute only makes sense on the first of them. 68. Geoff Shuetrim. The validation specification splits the discussion of the processing model up in a way that is not helpful to readers. Those parts of the processing model explanation need to be combined. RESOLVED: The rearrangement of the relevant paragraphs has been done. 69. Geoff Shuetrim. Existence assertion reference to the normative schema uses the wrong element name for the existence assertion. RESOLVED: This has been corrected in the text of the specification. 70. Geoff Shuetrim. The caption for the first example in the existence assertions specification refers to dimension filters instead of existence assertion expressions. RESOLVED: An appropriate caption has been provided. 71. Geoff Shuetrim. The new xfi:concept() function in the concept filter specification returns the QName of the concept for the fact provided as an argument. This duplicates the functionality of fn:node-name(.) and so the function should not be defined. RESOLVED: The unnecessary function has been eliminated. 72. Geoff Shuetrim. The period filters do not test that the period types match the period values being retrieved, thus, potentially causing errors to be thrown that would inadvertently halt the filtering process. RESOLVED: This has been corrected by testing for the period type before retrieving a period value. 73. Geoff Shuetrim. The function to access the data type QName for concepts is not conformant to the format for other function names. RESOLVED: The function name has been altered to xfi:data-type. 74. Geoff Shuetrim. The xfi:data-type function should take the concept QName as a parameter rather than an item to make it more generally useful and to conform to the pattern used for other concept property access functions. RESOLVED: The function parameter type has been changed to a QName data type. 75. Geoff Shuetrim. The functions supporting the substitution group filter can be simplified in a way that makes them more generally useful. RESOLVED: The two functions have been simplified to a single function that gets an ordered sequence of substitution group elements for a concept, starting with the element that is named by the substitution group attribute on the concept declaration and working back to the xbrli:item or xbrli:tuple element. 76. Geoff Shuetrim. Concept property accessor functions are not named in a way that makes their association with XBRL concepts readily identifiable. RESOLVED: The names of these functions all now start with "concept-". 77. Geoff Shuetrim. Change the xfi:has-dimension function name and signature to conform to the convention for other concept property accessor functions. RESOLVED: The suggestion has been adopted. 78. Geoff Shuetrim. Eliminate the xfi:nil function because it duplicates a function in the XQUery and XPath 2.0 functions specification. RESOLVED: Done. 79. Geoff Shuetrim. The XBRL function to return all possible member qnames for a given concept and dimension uses an axis that implies that no dimension member needs to be specified. This should be a separate function so that the parameter is not required to be supplied. RESOLVED: Done. 80. Herm Fischer. The concept-custom-attribute function should retrieve any simple data type and not force conversion to the string data type because that would cause problems for custom attributes on concepts when they have data types like QNames. RESOLVED: The suggestion has been adopted as reflected in an update to the function registry. No changes were required to the concept filter specification. 81. Geoff Shuetrim. A consistency assertion should just not be possible to evaluate if it is missing relationshpis to formulae because of the definition of the processing model. This is a good thing because relationship prohibition can then be used to turn off certain checks while not causing the assertion to be invalid. An error like we have in PWD 3 is overkill for such cases. RESOLVED: The error message has been deleted. 82. Geoff Shuetrim In the consistency assertion specification, delete the tables setting out the aspect tests to use for the aspects defined in the two aspect models set out in the variable specification because that is already done in the variable specification. RESOLVED: Done as suggested. 83. Geoff Shuetrim. The consistency assertion specification uses a notion of v-equality for fact consistency testing that requires c-equality preconditions for v-equality to be ignored. This undermines the definitions in the XBRL 2.1 specification and needs to be changed. RESOLVED. The consistency assertion tests for fact consistency are no longer based on the equality predicates of the XBRL 2.1 specification. Instead, consistency tests are specified directly in the consistency assertion specification. 84. Geoff Shuetrim. The consistency assertion specification does not define fact consistency for facts with data types derived from or that are the xbrli:fractionItemType. RESOLVED: Formulae cannot produce fraction item type derived facts yet so this is only a minor issue. The specification just highlights this limitation of formulae and flags that consistency is not defined for in this specification for such facts. 85. Cliff Binstock. DTS accessor function names should contain a consistent indication of whether the functions return nodes in the DTS or values from the DTS that are not nodes. Suggested a "qname" suffix for the names of functions returning QNames and an "element" functions returning elements in the documents representing the DTS. RESPONSE: Instead of using the suggested naming scheme (which does not cover things like attributes, string values etc) , we will use one in which all functions that can return nodes in the documents representing the DTS have a common prefix or suffix. All other functions will not return such nodes. Because we do not need any such functions for formulae, this prefix or suffix does not yet need to be defined. This eliminates the need to change specifications and function registry entries but does still provide scope for reflecting the distinction of interest. When the function definition specification is finalised, it will need to document the naming convention adopted. 86. Fujitsu. The substitution group filter uses the wrong element name for the filter in the first three examples. RESOLVED: This typographic error has been fixed. 87. Fujitsu. The function name for the xfi:period function has been typed as xfi-period. RESOLVED: This typographic error has been fixed. 88. Fujitsu. Some implied XPath expressions in the period filter specification have duplicate then's in their implied XPath expressions. RESOLVED: This typographic error has been fixed. 89. Fujitsu. The xfi:period function calls are expressed as xfi:period[XXX] when they should be xfi:period(.)[XXX]. RESOLVED: This typographic error has been fixed. 90. Fujitsu. The attribute name is wrong in the custom attribute example for general filters. RESOLVED: This typographic error has been fixed. 91. Herm Fischer. Remaining general equality tests in the examples and implied XPath expressions should be changed to value equality tests for consistency. RESOLVED: Done.