Formula Specification Public Working Draft Overview
Public Working Draft dated 2007-01-23
This document:
	XBRL-Formulas-Overview-PWD-2007-01-23.rtf
is non-normative.
Editors
Geoff Shuetrim	geoff@galexy.net	Galexy Pty. Limited	
Abstract
This document explains the various materials issued as a part of the public working draft released by the Formula Working Group (FWG).
Status
Circulation of this Public Working Draft is unrestricted.  Recipients of this draft are invited to submit comments to Geoff Shuetrim, and to submit notification of any relevant patent rights of which they are aware and to provide supporting documentation.
Table of Contents
Formula Specification Public Working Draft Overview
Public Working Draft dated 2007-01-23
1	Introduction
2	Requirements
3	Specifications
4	Schemas
5	Examples
6	Conformance suite tests
7	Focus issues
7.1	Fact production
7.2	Relating formulae to concepts
7.3	Precision
7.4	Fact variables and general variables
7.5	Implicit filters
7.6	Formula resources
7.7	Processing model
7.8	Variable evaluation order
7.9	Verbosity
A.	References (non-normative)
B.	Intellectual Property Status (non-normative)
C.	Document History (non-normative)
D.	Approval process (non‑normative)

1	Introduction
This document provides an overview of the this Public Working Draft (PWD).  It describes the various materials included in the PWD by the FWG.  It also lists a series of issues that the FWG hope to receive feedback on to drive the next steps of the specification development process.
The PWD includes a significant number of documents covering:
•	requirements agreed by the FWG;
•	specifications for various components of the formula specification;
•	schemas that support the specifications; and
•	complete examples showing usage of the specifications.
2	Requirements
The requirements agreed upon by the FWG are included in the PWD.
The requirements have been agreed by the FWG after a systematic review of the previous Candidate Recommendation Formula Requirements [CRFR].
The requirements do not yet include comprehensive use cases.  The FWG expects to future versions of the requirements to include explanations, supporting use cases, and examples based upon the specifications themselves.  Many of these will be drawn from the previous Candidate Recommendation Formula Requirements [CRFR].
3	Specifications
The formula specification has been modularised, resulting in a large number of relatively simple specification documents.
These documents depend on each other.  Indentations in the list of specifications below reflect the dependencies.
The set of specifications include:
The generic link specification (yet to be released as a public working draft)
The formula specification
	The variable specification
		The general filter specification
		The concept filter specification
		The period filter specification
		The entity filter specification
		The unit filter specification
		The matching filter specification
	The generic label specification
	The generic reference specification
Because the generic link specification is yet to be released as a public working draft, this formula PWD only includes a generic link schema.  That schema is non-normative and is only included to enable examples to be complete.  It will be replaced by the normative schema associated with the generic link specification when it is released.
The normative text in most of the specifications is limited to a few pages.
Being a first PWD for the FWG, these specifications are not going to be stable.  In particular, the filter specifications are only included as examples for the kinds of filters that could be defined to support the variable specification.  Some of the XBRL functions referred to by the filter specifications have not yet been defined in a function specification. If filters remain a part of the infrastructure supporting the formula specification, future drafts will include a more complete set covering segments, scenarios and tuples.
Also note that the specifications in this PWD do not cover:
•	a full definition of the result of processing an XBRL instance according to a set of formulae;
•	rules for when an XBRL instance must be deemed valid or invalid according to a set of formulae;
•	how individual business rules should be expressed using formulae;
•	how the results from one formula can be used as inputs for other formula; and
•	specifics features for the XBRL Dimensions Specification 1.0 [DIM].
Such features may be covered, as deemed necessary, in additional specifications or in modifications to these specifications.
4	Schemas
A small number of XML documents are also included in the PWD.  These can be found in the xml directory in the ZIP file containing the PWD.
The xml directory contains a directory called core_schemas.  This directory contains the schemas that support the specifications in the PWD.  Each specification names the schemas that support it.  The names of the files in the core_schemas directory correspond to these schema names.
These schemas can be used to validate the examples provided in the xml/examples directory.
The core schemas for the XBRL 2.1 specification are also provided in the xml/xbrl_schemas directory.  These ensure that the examples can be validated offline.
5	Examples
A very small number of examples are included in the xml/examples directory.  As the specifications stabilise, excerpts from these examples may be included in the specification texts.  Until then, these examples provide the only illustrations of how the specifications work together to support the declaration of XBRL formulae.
The examples and their supporting documentation can be reviewed using a web-browser with XSLT support by opening the index.xml file in the xml directory.
6	Conformance suite tests
As the specifications stabilise, a conformance suite will also be included in future public working drafts.  Each test in the conformance suite will have its own directory under the xml/tests directory.  No conformance suite tests are included in this PWD.
7	Focus issues
This section highlights several features of the suite of specifications released as a Public Working Draft (PWD) by the XBRL Formula Working Group (FWG).  The highlighted features are generally those that are subject to ongoing debate within the FWG about their inclusion and/or their implementation.
While the FWG is interested in any feedback on the PWD, responses to the specific questions raised in this section will be of particular value in enabling progress on the more challenging design decisions.
Responses that are accompanied by explanations and use cases will be of particular value to the FWG when deciding how best to address the limitations of the current PWD.
7.1	Fact production
7.1.1	Limitations on types of facts that can be produced
The requirements currently include: “formulae MUST be able to produce any valid value for any XBRL fact”.  The PWD does not support production of the following kinds of facts:
●	tuples
●	fraction items (e.g.: a tax rate fact specified with a value of exactly 1 / 3)
●	nil-value items (e.g.: an asset fact specified as <eg:asset nill=“true”/>
●	facts with footnotes
Do you anticipate that these limitations will constitute a problem?
7.1.2	Requiring formulae to being rule declarations for XBRL facts
The PWD requires that all formulae fully specify how an XBRL fact is to be produced, including the value, the context, and, for numeric facts, the precision and the unit. Note that a sufficient DTS to support the facts produced by formula evaluation is the DTS of the XBRL instance being processed so the identification of the DTS is not a part of the formula.
Do you perceive the restriction to production of XBRL facts, rather than other kinds of output such as true/false results or string messages, as a significant problem?
Will it be possible to define a rule for context production for every formula?
Do you expect this restriction to significantly increase the burden on formula authors?
7.2	Relating formulae to concepts
In the PWD, formulae define rules for production of XBRL facts.  In general, those rules involve transformations of the information contained in other XBRL facts.  All facts are instances of XBRL concepts.
Some formulae define rules for producing facts that are instances of a single concept.  Such formulae identify the concept using an XLink relationship much like the kind of relationship used to associate labels or references with a concept.  This makes the relationship between a formula and a concept relatively easy to identify.
e.g.: A formula stating that assets can be constructed from: assets = equity + liabilities could use this syntax.
Other formulae do not restrict the facts that can produce to being instances of a single concept.  The syntax for identifying the concepts that such formulae can produce instance facts for is not based on Xlink.  Instead, it relies on an XPath expression that evaluates to the QName of a concept.  This makes the relationship between a formula and the concepts it can be used for considerably less transparent.
e.g.: A formula stating that the total for any concept with a region dimension can be obtained by summing across values for each of the regions could use this syntax.
Is the ability to express formulae that do not always evaluate to facts that are instances of the same concept important to you?
enough to warrant the syntax based on the XPath expression?
Would the use of an XPath expression to identify the concept(s) that a formula can produce facts undermine the value of formula links as a taxonomy documentation resource.
Given that the XPath expression provides much of the functionality of the XLink syntax, should the XLink syntax be eliminated to avoid redundancy or does its concept documentation value warrant its inclusion?
In the event that both the XLink syntax and the XPath syntax is used should the formula be deemed invalid or should a precedence of application be specified?
7.3	Precision
The PWD defaults numeric facts to a precision of zero if no rule for specifying the construction of the result fact precision is provided.  This effectively loses the information on precision that is available in the input XBRL instance.
The optional XPath expression that can be used to specify a rule for determining precision is not forced to be consistent with the rule for construction of the fact value.
Is it important for you to be able to retain precision information from the input instance?
Do you think that the optional XPath expression for specifying the rule for construction of precision is useful, given that different formula processors will have different internal precision for their mathematical operations?
Would you prefer a different default precision to zero precision, for those formulae without a rule for determining precision?
7.4	Precision and decimal attributes
The PWD currently only includes syntax to support the specification of precision for XBRL facts produced by formulae.  If facts with a decimals attribute are required, that representation would have to be achieved by a post-processing operation on the fact produced by the formula.
Should formulae also provide syntax to enable control over the precision vs. decimals representation of the facts produced by formula evaluation?
7.5	Fact variables and general variables
The PWD defines syntax for two types of variables.  Both kinds of variables can be used by formulae.  The first kind of variable is the fact variable.  A fact variable can evaluate only to a single fact or to a sequence of facts from the input instance.  A fact variable is declared as a set of filters on the facts in the input instance.  A fact must pass each of the filters if it is to be matched to the fact variable.
The other kind of variable is the general variable.  It is not restricted in terms of the values that it can evaluate to.  A general variable is specified as an XPath expression in an XLink resource.  General variables are useful for a broad range of purposes, including specification of constant values and providing access to sequences of values to support dimensional analysis.
For each fact variable it would be possible to define an equivalent general variable.  The value of the fact variables lies in the documentation benefits and the simplification benefits provided by the filters that are associated with them.
The PWD includes an extensible mechanism for defining an XML representation of filters.  A filter is an XML syntax that implies an XPath test that can be applied to any fact in the XBRL instance being processed to determine whether it matches the filter criteria.  The PWD includes a range of example filter specifications, most of which have specialised purposes.
For example, the concept filters enable the declaration of filtration criteria based on the element name of the facts in the input instance.  These filters ensure that XLink relationships can be used to identify the concepts for facts that a formula uses as input.
Similarly period filters support declaration of filtration criteria based on the context period.
A number of important filters, including ones for conditioning on segment and scenario content and ones for conditioning on tuple context, have been omitted from the PWD pending a final decision to use filters.
The most general filter is a single attribute that contains the XPath expression implied by the filter.  That filter can express the same filter rules as all of the other more specialised filters.
Do the documentation, extensibility and simplification features of XLink-based filters warrant their inclusion?
Should all filters be replaced by the general filter that requires an explicit statement of the necessary XPath test that it implies?
Should fact variables (and thus, filters) be eliminated entirely in favour of a design that relies only on general variables?
7.6	Implicit filters
In the PWD, all filters must be stated explicitly.  This adds to the verbosity of formulae and potentially increases the maintenance burden associated with formulae.
The matching filter specification has been included in the PWD to provide a simple syntax for expressing the kinds of filtration rules that might be possible to make implicit.
e.g.: a filtration rule that is a candidate for becoming implicit is one implying that its fact variable must only match facts with the same entity identifier as the fact matched another fact variable.
Should implicit filters be included in the specification?
Should implicit filters be used for matching periods?
Should implicit filters be used for matching entity identifiers?
Should implicit filters be used for matching units?
Should implicit filters be used for matching segments and scenarios?  If implicit filters for segments and scenarios are warranted, what filtration rules are appropriate to determine a match between two segments or two scenarios?
Should implicit filters be used for matching user-defined dimensions as specified in the XBRL Dimensions Specification 1.0 [DIM]?
7.7	Formula resources
In the PWD, each formula is associated with an XLink resource.  That resource includes a range of attributes, each of which is contains the rules for construction of a different aspect of a fact.  There are attributes for the value of the fact, the context of the fact, the unit and the precision of numeric facts, and, optionally, the element name of the fact.  Each attribute contains an XPath expression.  Each of the XPath expressions can contain references to variables.  For the formula to evaluate successfully, all variable references have to resolve to either fact or general variables that have been defined outside the formula.
The syntax for formula resources represents a compromise between two competing syntax strategies.
One syntax strategy involves reducing formula resources down to a single XPath expression that evaluates to the root element of a complete XBRL instance (or something else such as a Boolean value or a string message if we relax the restriction on formulae requiring them to always produce XBRL facts).  That expression, and the variables, and the functions that it references, would be responsible for specifying the rules for construction of the fact value, the context, the unit, and everything else.
The competing syntax strategy involves separation of formula construction rules into their various components, one for the fact value, one for the context, one for the unit, etc.  This approach could be taken further, with the single XPath expression for construction of the context being replaced with a set of rules, each of which is responsible for a single aspect of the fact context.  Such rules would be analogous to the filters that are used with fact variables.
Which of the two syntax strategies would best meet your own needs?
7.8	Processing model
The PWD specifies a processing model for a single evaluation of a single formula.  The broader context for how formula evaluations might be co-ordinated is left to other specifications and to implementations of formula processors.
For example, such extensions could document how formulae can be related to each other or to other XLink resources to declare:
●	chains of formulae to be evaluated sequentially
●	groups of formulae that work together to construct a tuple
●	formulae that can be used to test for specific conditions in XBRL instances
●	template XBRL instances, in which formula output is to be inserted.
Is the existing processing model specification sufficient?
Are you comfortable with the strategy of developing extension specifications to standardise different formula usage-patterns?  If so, which usage patterns are most important to you?
7.9	Variable evaluation order
The processing model specification requires variables to be evaluated in the order specified by the order attribute on the XLink arcs associating variables with formulae.  This ordering requirement is imposed to ensure that all variables that are referenced by an XPath expression have already been evaluated prior to evaluation of the XPath expression.
e.g.:
The rule for fact value construction implies that the fact value is given by $a + $b
Fact variable $a involves filters that reference fact variable $b.
Fact variable $b also involves filters that reference another variable, also called $a, where that new variable $a is a general variable that provides a specific entity identifier, as a constant value.
The formula would need to be related to three different variables. The order of these relationships and the name of the variable that they reference would be specified as:
1.	The formula is related to variable $a (the constant) with order 1.
2.	The formula is related to variable $b (the fact variable) with order 2.
3.	The formula is related to variable $a (the fact variable) with order 3.
This example is intended to illustrate that you can avoid specifying an explicit evaluation order if you have unique variable names and if you are prepared to parse XPath expressions.  If you do not want to insist on unique variable names, or you do not want to force parsing of all XPath expressions prior to actual evaluation, then you need an explicit ordering of variables for evaluation purposes.  You cannot have non-unique variable names and no explicit ordering of variables.
Is this usage of the order attribute acceptable or would you prefer an additional custom attribute on the XLink arcs that relate formulae to variables?
Do you think that the processing model can be specified in a manner that makes the order attribute redundant?
If it really is possible to eliminate the need explicit orderings of variables by requiring uniqueness of variables names in a DTS, would such a restriction be preferable to requiring an explicit ordering of variables?
Would it acceptable to you if the processing model required access to XPath's Abstract Syntax Tree or equivalent to identify the variable dependencies in a given formula or variable declaration?
Do you think that we can avoid a trade-off between non-uniqueness of variable names and elimination of explicit variable evaluation ordering?

7.10	Verbosity
The specification leads to verbose formulae declarations because of the extensive use of XLink.  This verbosity is highlighted by the examples that accompany this PWD.  Verbosity could be somewhat reduced by permitting variables declarations to be nested within formula declarations and by permitting filters to be nested within variable declarations.
Does the verbosity implied by the PWD constitute a significant problem for you?
Would the reduction in verbosity resulting from nesting variables and filters be of sufficient value to you to warrant the associated increase in complexity of the formula specification?
Would the use of user-defined functions (functions authored by as part of a formula linkbase) be of value in reducing verbosity for you?
A.	References (non-normative)
[CRFR]	Walter Hamscher, Geoff Shuetrim, David vun Kannon
XBRL Formula Requirements, Candidate Recommendation, 2005-05-21
http://www.xbrl.org/SpecRequirements/ 
	
[DIM]	Ignacio Hernández-Ros, Hugh Wallis.
XBRL Dimensions 1.0 Recommendation, 2006-09-18.
http://www.xbrl.org/SpecRecommendations/	
B.	Intellectual Property Status (non-normative)
This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works.  However, this document itself may not be modified in any way, such as by removing the copyright notice or references to XBRL International or XBRL organizations, except as required to translate it into languages other than English.  Members of XBRL International agree to grant certain licenses under the XBRL International Intellectual Property Policy (www.xbrl.org/legal).
This document and the information contained herein is provided on an "AS IS" basis and XBRL INTERNATIONAL DISCLAIMS ALL WARRANTIES, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.
The attention of users of this document is directed to the possibility that compliance with or adoption of XBRL International specifications may require use of an invention covered by patent rights. XBRL International shall not be responsible for identifying patents for which a license may be required by any XBRL International specification, or for conducting legal inquiries into the legal validity or scope of those patents that are brought to its attention. XBRL International specifications are prospective and advisory only. Prospective users are responsible for protecting themselves against liability for infringement of patents.  XBRL International takes no position regarding the validity or scope of any intellectual property or other rights that might be claimed to pertain to the implementation or use of the technology described in this document or the extent to which any license under such rights might or might not be available; neither does it represent that it has made any effort to identify any such rights.  Members of XBRL International agree to grant certain licenses under the XBRL International Intellectual Property Policy (www.xbrl.org/legal).
C.	Document History (non-normative)
Date	Editor	Summary	
2007-01-23	Geoff Shuetrim	First draft.
Incorporated feedback from Victor Morilla on the focal issues.	
D.	Approval process (non‑normative)
This document will not be released as a part of any final recommendation.