Grammar for text-based XBRL Formula (XF) 1.0

Document 10 October 2018

Copyright © 2018 XBRL International Inc., All Rights Reserved.

This version:
<http://www.xbrl.org/WGN/xf-grammar/WGN-2018-10-10/xf-grammar-WGN-2018-10-10.html>
Editors:
Herm Fischer, Mark V Systems <fischer@markv.com>
Paul Warren, XBRL International <pdw@xbrl.org>
Contributor:
Andrew Perkins, Workiva <andrew.perkins@workiva.com>

Status

Circulation of this Document is unrestricted. Other documents may supersede this document. Recipients are invited to submit comments to formula-feedback@xbrl.org, and to submit notification of any relevant patent rights of which they are aware and provide supporting documentation.

Abstract

The XBRL Formula Specifications define an XLink-based syntax for a general purpose rules language for XBRL. The XLink syntax cannot be readily understood by end users and requires specialist tools to view or edit effectively.

The XBRL Formula Working Group is experimenting with the use of a textual representation of the XBRL Formula language (referred to as "XF"), enabling editing and viewing of XBRL Formula rules using a simple text editor. This Working Group Note provides a grammar for XF.

Table of Contents

1 Introduction
1.1 Example XF rule
1.2 Notation
2 XF grammar
3 XPath 2.0 grammar

Appendices

A Using the grammar with TatSu
B References
C Intellectual property status (non-normative)
D Acknowledgements (non-normative)
E Document history (non-normative)


1 Introduction

Text-based Formula, or "XF", provides an alternative to the XLink-syntax for XBRL Formula defined by the XBRL Formula specifications [FORMULA]. XF provides the same functionality as XBRL Formula, and formula rules can be freely converted between the XLink and XF syntaxes.

This Working Group Note provides a formal grammar for the XF syntax, expressed in a variant of EBNF notation.

1.1 Example XF rule

A simple example of a rule conforming to the grammar in this document is shown below:

namespace eg = "http://www.example.com/ns" ; assertion RevenueNonNegative { unsatisfied-message (en) "Revenue must not be negative"; variable $revenue { concept-name eg:Revenue ; }; test { $revenue ge 0 }; };

1.2 Notation

The grammar included in this document uses a variant of EBNF notation suitable for use in the TatSu parser generator. The specification of this syntax can be found in the TatSu documentation.

The grammar can be used to check if XF rules are syntactically correct using the TatSu tool, as described in Appendix A.

2 XF grammar

@@grammar :: XF @@comments :: /(?s)\(:((?!(\(:|:\))).)*:\)/ @@namechars :: "-" module = {namespace_declaration}* {default}* {parameter}* {filter_declaration|fact_variable|general_variable|function_declaration}* {assertion}* $; separator = ";"; namespace_declaration = "namespace" name "=" quoted_url separator; default = default_severity | "default-language" language separator; default_severity = "default-severity" message_severity separator ; severity = "severity" message_severity separator ; message_severity = "ERROR" | "WARNING" | "INFO"; parameter = "parameter" name "{" [ "required" ] ["select" enclosed_expression ] ["as" qname ] "}" separator ; filter_declaration = "filter" name "{" filter {separator filter}* [ separator ] "}" separator ; filter = [ "not" ] ( concept_filter | general_filter | period_filter | dimension_filter | unit_filter | entity_filter | match_filter | relative_filter | tuple_filter | value_filter | boolean_filter | aspect_cover_filter | concept_relation_filter ) | declared_filter_reference ; concept_filter = ( "covering" | `covering` | "non-covering" ) ( "concept-name" ~ {qname | localname | enclosed_expression}+ | "concept-period-type" ~ ("instant" | "duration") | "concept-balance" ~ ("credit" | "debit" | "none") | "concept-data-type" ~ ("strict" | "non-strict") (qname | enclosed_expression) | "concept-substitution-group" ~ ("strict" | "non-strict") (qname | enclosed_expression) ) ; general_filter = "general" ~ enclosed_expression ; period_filter = ( "period-start" ~ (date_constant | date_time_constant | ("date" enclosed_expression [ "time" enclosed_expression ] )) | "period-end" ~ (date_constant | date_time_constant | ("date" enclosed_expression [ "time" enclosed_expression ] )) | "period-instant" ~ (date_constant | date_time_constant | ("date" enclosed_expression [ "time" enclosed_expression ])) | "period" ~ enclosed_expression | "instant-duration" ~ ("start" | "end") variable_ref ) ; dimension_filter = ( ( "explicit-dimension" ~ (qname | localname | enclosed_expression) { "default-member" | "member" ~ (variable_ref | qname | localname | enclosed_expression) ["linkrole" quoted_anyURI [ "arcrole" quoted_anyURI ] "axis" dimension_axis ] }* ) | ( "typed-dimension" ~ (qname | localname | enclosed_expression) ["test" enclosed_expression ] ) ) ; dimension_axis = "child" | "child-or-self" | "descendant" | "descendant-or-self" ; unit_filter = ( "unit-single-measure" ~ (qname | enclosed_expression) | "unit-general-measures" ~ enclosed_expression ) ; entity_filter = ( "entity-scheme-pattern" ~ regexp_pattern | "entity-scheme" ~ enclosed_expression | "entity-identifier-pattern" ~ regexp_pattern | "entity-identifier" ~ enclosed_expression | "entity" ~ "scheme" enclosed_expression "value" enclosed_expression ) ; match_filter = ( "match-concept" ~ variable_ref | "match-location" ~ variable_ref | "match-entity-identifier" ~ variable_ref | "match-period" ~ variable_ref | "match-unit" ~ variable_ref | "match-dimension" ~ variable_ref "dimension" qname ) [ "match-any" ] ; relative_filter = "relative" ~ variable_ref ; tuple_filter = ( "parent" ~ (qname | enclosed_expression) | "ancestor" ~ (qname | enclosed_expression) | "sibling" ~ variable_ref ) ; (* Note: "location" filter is not supported as it involves path expressions *) value_filter = "nilled" ; (* Note: "precision" filter is not supported *) boolean_filter = ("and" | "or") ~ "{" filter {separator filter}* [ separator ] "}" ; aspect_cover_filter = "aspect-cover" ~ { "all" | "concept" | "entity-identifier" | "location" | "period" | "unit" | "dimensions" | "dimension" (qname | localname | enclosed_expression) | "exclude-dimension" (qname | localname | enclosed_expression) }+ ; concept_relation_filter = "concept-relation" ~ (variable_ref | qname | localname | enclosed_expression) [ "linkrole" (quoted_anyURI|enclosed_expression) ] [ "arcrole" (quoted_anyURI|enclosed_expression) ] [ "axis" relation_axis ] [ "generations" non_negative_integer ] [ "test" enclosed_expression ] ; (* Consider keywords as a short hand for standard arcroles, e.g. presentation => arcrole "http://.../parent-child" *) relation_axis = ( "child-or-self" | "child" | "descendant-or-self" | "descendant" | "parent-or-self" | "parent" | "ancestor-or-self" | "ancestor" | "sibling-or-self" | "sibling-or-descendant" | "sibling" ) ; declared_filter_reference = "filter" ~ variable_ref ; fact_variable = "variable" ~ variable_ref "{" [ "bind-as-sequence" ] [ "nils" ] [ "matches" ] [ "fallback" enclosed_expression ] [ filter { separator filter }* [separator] ] "}" separator ; general_variable = "variable" ~ variable_ref "{" [ "bind-as-sequence" ] "select" enclosed_expression "}" separator ; function_declaration = "function" ~ qname "(" { NCNAME_FRAG "as" qname }* ")" "as" qname [ "{" "return" enclosed_expression separator "}" ] separator ; assertion = "assertion" name "{" { label | message | severity | "aspect-model-non-dimensional" separator | "no-implicit-filtering" separator }* {filter separator}* {fact_variable | general_variable | referenced_parameter }* {precondition}* (value_expression | existence_expression) [ separator ] "}" separator ; label = "label" ["(" language ")"] quoted_string separator ; message = ("satisfied-message" | "unsatisfied-message") ~ ["(" ~ language ")"] quoted_string separator ; referenced_parameter = "parameter" ~ variable_ref "references" qname separator ; precondition = "precondition" enclosed_expression separator ; value_expression = "test" enclosed_expression ; existence_expression = "evaluation-count" enclosed_expression ; enclosed_expression = "{" xPath "}"; name = NCNAME_FRAG ; language = /[A-Za-z]{2}(-[A-Za-z]{2})?/; quoted_anyURI = quoted_url; quoted_url = quoted_string; localname = name; variable_ref = "$" variable_name ; variable_name = name ; non_negative_integer = /[0-9]+/ ; date_time_constant = /\b(-?(?:[1-9][0-9]*)?[0-9]{4})-(1[0-2]|0[1-9])-(3[01]|0[1-9]|[12][0-9])T(24:00:00(\.0+)?|(2[0-3]|[01][0-9]):([0-5][0-9]):([0-5][0-9])(\\.[0-9]+))?(Z)?\b/; date_constant = /\b(-?(?:[1-9][0-9]*)?[0-9]{4})-(1[0-2]|0[1-9])-(3[01]|0[1-9]|[12][0-9])\b/; quoted_string = /("([^\\"]|\\.)*"|'([^\\']|\\.)*')/ ; regexp_pattern = /\/([^\\\/]|\\.)*\// ; #include :: "xpath.ebnf"

3 XPath 2.0 grammar

The XF grammar depends on the XPath 2.0 [XPATH 2.0] grammar, a version of which is included below.

(* Based on https://github.com/XPath-Next/XPath-Next/blob/master/grammars/xpath-2.0.ebnf *) xPath = expr ; expr = exprSingle {"," exprSingle}* ; exprSingle = forExpr | quantifiedExpr | ifExpr | orExpr ; forExpr = simpleForClause "return" exprSingle ; simpleForClause = "for" "$" varName "in" exprSingle {"," "$" varName "in" exprSingle}* ; quantifiedExpr = ("some" | "every") "$" varName "in" exprSingle {"," "$" varName "in" exprSingle}* "satisfies" exprSingle ; ifExpr = "if" "(" expr ")" "then" exprSingle "else" exprSingle ; orExpr = andExpr { "or" andExpr }* ; andExpr = comparisonExpr { "and" comparisonExpr }* ; comparisonExpr = rangeExpr [ (valueComp | nodeComp | generalComp ) rangeExpr ] ; rangeExpr = additiveExpr [ "to" additiveExpr ] ; additiveExpr = multiplicativeExpr { ("+" | "-") multiplicativeExpr }* ; multiplicativeExpr = unionExpr { ("*" | "div" | "idiv" | "mod") unionExpr }* ; unionExpr = intersectExceptExpr { ("union" | "|") intersectExceptExpr }* ; intersectExceptExpr = instanceofExpr { ("intersect" | "except") instanceofExpr }* ; instanceofExpr = treatExpr [ "instance" "of" sequenceType ] ; treatExpr = castableExpr [ "treat" "as" sequenceType ] ; castableExpr = castExpr [ "castable" "as" singleType ] ; castExpr = unaryExpr [ "cast" "as" singleType ] ; unaryExpr = {"-" | "+"}* valueExpr ; valueExpr = pathExpr ; generalComp = "=" | "!=" | "<=" | "<" | ">=" | ">" ; valueComp = "eq" | "ne" | "lt" | "le" | "gt" | "ge" ; nodeComp = "is" | "<<" | ">>" ; pathExpr = ("//" relativePathExpr) | ("/" [ relativePathExpr ] ) | relativePathExpr ; relativePathExpr = stepExpr {("/" | "//") stepExpr}* ; stepExpr = filterExpr | axisStep ; axisStep = (reverseStep | forwardStep) predicateList ; forwardStep = (forwardAxis nodeTest) | abbrevForwardStep ; forwardAxis = ("child" "::") | ("descendant" "::") | ("attribute" "::") | ("self" "::") | ("descendant-or-self" "::") | ("following-sibling" "::") | ("following" "::") | ("namespace" "::") ; abbrevForwardStep = [ "@" ] nodeTest ; reverseStep = (reverseAxis nodeTest) | abbrevReverseStep ; reverseAxis = ("parent" "::") | ("ancestor" "::") | ("preceding-sibling" "::") | ("preceding" "::") | ("ancestor-or-self" "::") ; abbrevReverseStep = ".." ; nodeTest = kindTest | nameTest ; nameTest = qname | wildcard ; wildcard = "*" | (ncname ":" "*") | ("*" ":" ncname) ; filterExpr = primaryExpr predicateList ; predicateList = {predicate}* ; predicate = "[" expr "]" ; primaryExpr = literal | varRef | parenthesizedExpr | contextItemExpr | functionCall ; literal = numericLiteral | stringLiteral ; numericLiteral = integerLiteral | decimalLiteral | doubleLiteral ; varRef = "$" varName ; varName = qname ; parenthesizedExpr = "(" [ expr ] ")" ; contextItemExpr = "." ; functionCall = qname "(" [exprSingle {"," exprSingle}*] ")" ; singleType = atomicType [ "?" ] ; sequenceType = ("empty-sequence" "(" ")") | (itemType [ occurrenceIndicator]) ; occurrenceIndicator = "?" | "*" | "+" ; itemType = kindTest | ("item" "(" ")") | atomicType ; atomicType = qname ; kindTest = documentTest | elementTest | attributeTest | schemaElementTest | schemaAttributeTest | pITest | commentTest | textTest | anyKindTest ; anyKindTest = "node" "(" ")" ; documentTest = "document-node" "(" [elementTest | schemaElementTest] ")" ; textTest = "text" "(" ")" ; commentTest = "comment" "(" ")" ; pITest = "processing-instruction" "(" [ncname | stringLiteral] ")" ; attributeTest = "attribute" "(" [ attribNameOrWildcard ["," typeName] ] ")" ; attribNameOrWildcard = attributeName | "*" ; schemaAttributeTest = "schema-attribute" "(" attributeDeclaration ")" ; attributeDeclaration = attributeName ; elementTest = "element" "(" [elementNameOrWildcard ["," typeName ["?"] ] ] ")" ; elementNameOrWildcard = elementName | "*" ; schemaElementTest = "schema-element" "(" elementDeclaration ")" ; elementDeclaration = elementName ; attributeName = qname ; elementName = qname ; typeName = qname ; integerLiteral = DIGITS ; decimalLiteral = ("." DIGITS) | (DIGITS /\.[0-9]*/) ; doubleLiteral = (("." DIGITS) | (DIGITS /(\.[0-9]*)?/ )) /[eE][+-]?/ DIGITS ; stringLiteral = (/"/ {ESCAPEQUOT | /[^"]/}* /"/) | (/'/ {ESCAPEAPOS | /[^']/}* /'/) ; ESCAPEQUOT = '""' ; ESCAPEAPOS = "''" ; qname = ( NCNAME_FRAG /:/ NCNAME_FRAG ) | NCNAME_FRAG ; ncname = NCNAME_FRAG ; (* Multi-line regexs don't currently work in TatSu NCNAME_FRAG = /(?x) ( [A-Z] | _ | [a-z] | [\u00C0-\u00D6] | [\u00D8-\u00F6] | [\u00F8-\u02FF] | [\u0370-\u037D] | [\u037F-\u1FFF] | [\u200C-\u200D] | [\u2070-\u218F] | [\u2C00-\u2FEF] | [\u3001-\uD7FF] | [\uF900-\uFDCF] | [\uFDF0-\uFFFD] | [\U00010000-\U000EFFFF] ) ( [A-Z] | _ | [a-z] | [\u00C0-\u00D6] | [\u00D8-\u00F6] | [\u00F8-\u02FF] | [\u0370-\u037D] | [\u037F-\u1FFF] | [\u200C-\u200D] | [\u2070-\u218F] | [\u2C00-\u2FEF] | [\u3001-\uD7FF] | [\uF900-\uFDCF] | [\uFDF0-\uFFFD] | [\U00010000-\U000EFFFF] | - | \. | [0-9] | \u00B7 | [\u0300-\u036F] | [\u203F-\u2040] )* /; *) NCNAME_FRAG = /([A-Z]|_|[a-z]|[\u00C0-\u00D6]|[\u00D8-\u00F6]|[\u00F8-\u02FF]|[\u0370-\u037D]|[\u037F-\u1FFF]|[\u200C-\u200D]|[\u2070-\u218F]|[\u2C00-\u2FEF]|[\u3001-\uD7FF]|[\uF900-\uFDCF]|[\uFDF0-\uFFFD]|[\U00010000-\U000EFFFF])([A-Z]|_|[a-z]|[\u00C0-\u00D6]|[\u00D8-\u00F6]|[\u00F8-\u02FF]|[\u0370-\u037D]|[\u037F-\u1FFF]|[\u200C-\u200D]|[\u2070-\u218F]|[\u2C00-\u2FEF]|[\u3001-\uD7FF]|[\uF900-\uFDCF]|[\uFDF0-\uFFFD]|[\U00010000-\U000EFFFF]|-|\.|[0-9]|\u00B7|[\u0300-\u036F]|[\u203F-\u2040])*/; DIGITS = /[0-9]+/ ;

Appendix A Using the grammar with TatSu

The grammar in this document can be used to generate a parser using TatSu. This requires Python. TatSu can be installed using pip:

pip3 install tatsu

TatSu provides a command line tool that can be used to generate a parser:

python3 -m tatsu xf.ebnf --outfile xf.py

The resulting xf.py file can be used to test inputs for validity against the grammar:

python3 xf.py test-file.xf

If the input is valid, a parse tree will be output, otherwise an exception will be thrown.

Appendix B References

FORMULA
XBRL International Inc.. "XBRL Formulae 1.0"
Phillip Engel
, Herm Fischer, Victor Morilla, Jim Richards, Geoff Shuetrim, David vun Kannon, and Hugh Wallis.
(See http://www.xbrl.org/Specification/formula/CR-2008-12-31/formula-CR-2008-12-31.html)
XPATH 2.0
W3C (World Wide Web Consortium). "XML Path Language (XPath) 2.0"
Anders Berglund
, Scott Boag, Don Chamberlin, Mary F. Fernández, Michael Kay, Jonathan Robie, and Jérôme Siméon.
(See http://www.w3.org/TR/xpath20/)

Appendix C Intellectual property status (non-normative)

This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. However, this document itself may not be modified in any way, such as by removing the copyright notice or references to XBRL International or XBRL organizations, except as required to translate it into languages other than English. Members of XBRL International agree to grant certain licenses under the XBRL International Intellectual Property Policy (www.xbrl.org/legal).

This document and the information contained herein is provided on an "AS IS" basis and XBRL INTERNATIONAL DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

The attention of users of this document is directed to the possibility that compliance with or adoption of XBRL International specifications may require use of an invention covered by patent rights. XBRL International shall not be responsible for identifying patents for which a license may be required by any XBRL International specification, or for conducting legal inquiries into the legal validity or scope of those patents that are brought to its attention. XBRL International specifications are prospective and advisory only. Prospective users are responsible for protecting themselves against liability for infringement of patents. XBRL International takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights. Members of XBRL International agree to grant certain licenses under the XBRL International Intellectual Property Policy (www.xbrl.org/legal).

Appendix D Acknowledgements (non-normative)

This document could not have been written without the contributions of many people including the participants in the Formula Working Group.

Appendix E Document history (non-normative)

DateAuthorDetails
16 May 2018

Initial draft