xBRL-CSV: mapping from Open Information Model 1.0

Candidate Recommendation 9 October 2019

This version
https://www.xbrl.org/Specification/xbrl-csv/CR-2019-10-09/xbrl-csv-CR-2019-10-09.html
Editors
Herm Fischer, Mark V Systems Limited <fischer@markv.com>
Paul Warren, XBRL International Inc. <pdw@xbrl.org>
Mark Goodhand, CoreFiling <mrg@corefiling.com>
Contributors
David Bell, Donnelley Financial Solutions <david.x.bell@dfsco.com>
Daniel Dracott, CoreFiling <djd@corefiling.com>
Paul Hulst, De Nederlandsche Bank N.V. <P.J.Hulst@dnb.nl>
Eleanor Joslin, CoreFiling <ejj@corefiling.com>

Table of Contents

1 Introduction

This document defines a CSV-based representation of the information in an XBRL report, as defined in the XBRL Open Information Model OIM. It provides significant flexibility in the layout of the CSV tables, in order to enable tables that are efficient and intuitive, allowing related facts to be grouped into rows, and share common aspects. The structure of the tables is controlled by a JSON metadata file.

The mapping of an xBRL-CSV report into the OIM report model is described in this specification, enabling lossless transformation of xBRL-CSV reports into other formats, including the XML-based xBRL-XML representation.

It should be noted that the flexibility in table layout means that there are many different ways of representing the same OIM report using xBRL-CSV. It is anticipated that processors performing a transformation into the xBRL-CSV format will require some level of user input into the specification of the metadata file in order to achieve intuitive and efficient tables.

2 Identifiers in xBRL-CSV

xBRL-CSV makes use of QNames to represent expanded names and SQNames to represent some other values. The prefix map for resolving QNames and SQNames is provided by the namespaces object.

3 xBRL-CSV report structure

An xBRL-CSV report is a CSV-based representation of a XBRL report, consisting of a JSON metadata file and zero or more CSV data tables. The metadata file provides links to the data tables that contain the report data, and defines the meaning of the columns in each data table.

An xBRL-CSV report is identified by a path or URL to a metadata file, referred to as the primary metadata file.

The metadata file may optionally extend other metadata files in order to re-use definitions that are common to multiple xBRL-CSV reports.

The metadata files contain references to the associated CSV files.

3.1 xBRL-CSV metadata file

An xBRL-CSV metadata file is a JSON file that conforms to the format described in this section. A metadata file name SHOULD have an extension of .json.

A metadata file consists of a JSON object with the following properties:

documentInfo
(required) A documentInfo object.
tableTemplates
(optional) A tableTemplates object.
tables
(optional) A tables object.
reportParameters
(optional) An object or string that provides report parameters. See Section 3.1.9.
links
(optional) A links object.

3.1.1 Metadata: documentInfo

The documentInfo object provides document-level information about the xBRL-CSV report. The object has the following properties:

documentType
(required) The fixed value http://xbrl.org/CR/2019-10-09/xbrl-csv
reportDimensions
(optional) A reportDimensions object defining dimension values that are common to all facts in the report.
namespaces
(optional) A namespaces object defining the prefix map for QName and SQName values in this report.
taxonomy
(optional) A list of URLs defining the taxonomy for the report. Relative URLs are resolved relative to the primary metadata file.
decimals
(optional) An integer providing a value for the {decimals} property of numeric facts in the report.
extends
(optional) A URL to another xBRL-CSV metadata file. The current metadata file is combined with the referenced file, as described in Section 3.1.12. Relative URLs are resolved relative to the file in which the property appears.
final
(optional) A final object, specifying objects that cannot be extended by an importing metadata file (see Section 3.1.12).

3.1.1.1 Metadata: documentInfo/namespaces

The namespaces object provides the prefix map for QName and SQName values in this report.

Each key provides a prefix, which is mapped to the URI provided by the value.

3.1.1.2 Metadata: documentInfo/linkTypes

The linkTypes object is a URI map defining URI aliases used to identify link types in this report.

3.1.1.3 Metadata: documentInfo/linkGroups

The linkGroups object is a URI map defining URI aliases used to identify link groups in this report.

3.1.1.4 Metadata: documentInfo/reportDimensions

The reportDimensions object provides default dimension values for all facts in a report. The reportDimensions object is a dimensions object.

3.1.1.5 Metadata: documentInfo/final

The final object defines objects which cannot be extended in importing files. The object has the following properties:

All properties are optional, and if present take the fixed value of true, indicating that the corresponding object cannot be extended by importing metadata files (see Section 3.1.12).

3.1.2 Metadata: tableTemplates

The tableTemplates object provides table template definitions.

A table template defines the structure of a table that may be included in an xBRL-CSV report. Each CSV data table is the instantiation of a table template in a CSV file. Each table template may be used by zero or more tables with the report.

The keys of the tableTemplates object are identifiers, providing a unique table template identifier for each table template.

Each value is a table template object with the following properties:

rowIdColumn
(optional) An identifier specifying the name of the column containing a unique identifier for each row in the table.
columns
(required) A columns object.
decimals
(optional) An integer providing a value for the {decimals} property of numeric facts in the report.
tableDimensions
(required) A tableDimensions object.
transposed
(optional) A boolean value indicating whether the table is transposed. Defaults to false, meaning that lines in the CSV file correspond to rows in the table.

3.1.3 Metadata: tableTemplates/tableDimensions

The tableDimensions object provides default dimension values for all facts in a table. The tableDimensions object is a dimensions object.

3.1.4 Metadata: tableTemplates/columns

The columns object specifies the columns that may appear in a table.

Keys of the columns object MUST be identifiers.

Each value is a column object with the following properties:

decimals
(optional) An integer providing a value for the {decimals} property of any numeric facts produced by the column.
columnDimensions
(optional) A columnDimensions object.

If the columnDimensions property is present then the column is a fact column. Note that the columnDimensions object MAY be empty.

The decimals property MUST NOT be present unless the columnDimensions property is also present.

3.1.5 Metadata: tableTemplates/columnDimensions

The columnDimensions object provides dimension values for all facts in a fact column. The columnDimensions object is a dimensions object.

3.1.6 Metadata: dimensions object

A dimensions object is a JSON object that defines a set of dimension values. It defines a common format for reportDimensions, tableDimensions and columnDimensions objects.

A dimensions object has the same format and constraints as an xBRL-JSON dimensions object, except as noted below.

A dimensions object defines a set of dimension values which map to the {dimensions} property of facts as described in Section 4.2.5.

If the value of a dimension starts with $ (but not $$) then the value MUST take one of the following forms:

Where parameter-name and period-specifier are identifiers.

parameter-name MUST be at least one of the following:

If present, period-specifier MUST be either start or end.

The value for the dimension is determined by the cell, C, which is in the column specified by column and the current row, as follows:

If the value of a dimension starts with $$ then it is treated as a literal $.

If the value of a dimension is #none, the dimension has an explicit no-value.

An explicit no-value for a dimension is equivalent to not specifying the dimension at all in the dimensions object, except that it also prevents inheritance of a value for that dimension from another location (see Section 4.2.5).

If the value of a dimension starts with ## then it is treated as a literal #.

See Appendix A for a summary of the special values available in different contexts.

In addition to the syntax supported by xBRL-JSON, the period dimension supports a number of additional formats, described in Section 3.1.6.1.

3.1.6.1 Period formats

The period dimension may be specified in a number of different formats. Either of the date time-based formats permitted by xBRL-JSON may be used:

In addition, a number of abbreviated forms may be used to specify a duration:

Time zone specifiers MUST NOT be included in any of the abbreviated forms.

Examples of these abbreviated forms, and their equivalent ISO8601 date time forms, are shown below:

xBRL-CSV format Period type Equivalent date time-based representation Notes
2019-01-01..2019-12-31 duration 2019-01-01T00:00:00/2020-01-01T00:00:00 inclusive date range
2019-06-01 duration 2019-06-01T00:00:00/2019-06-02T00:00:00 duration of a single day
2019-06 duration 2019-06-01T00:00:00/2019-07-01T00:00:00 calendar month
2019 duration 2019-01-01T00:00:00/2020-12-31T00:00:00 calendar year
2019Q2 duration 2019-04-01T00:00:00/2019-07-01T00:00:00 calendar quarter
2019H1 duration 2019-01-01T00:00:00/2019-07-01T00:00:00 calendar half
2019W29 duration 2019-07-15T00:00:00/2019-07-22T00:00:00 ISO week

3.1.7 Metadata: tables object

The tables object associates table template definitions with CSV data tables .

The keys of the tables object are identifiers, providing a unique table identifier for each data table.

Each value is a table object with the following properties:

url
(required) A URL to the CSV file for this table. Relative URLs are resolved relative to the primary metadata file.
template
(optional) A table template identifier. If this property is omitted, then it defaults to the table identifier. The value MUST be the identifier of a table template.
optional
(optional) A boolean value indicating that the CSV file specified by the url property is not required to exist. Defaults to false. If false, the the file specified MUST exist.
tableParameters
(optional) A tableParameters object.

3.1.8 Metadata: tables/tableParameters

A table parameter is a table-wide value that can be referenced in the table template definition for the table.

Table parameters are defined by a tableParameters object.

The keys and values of the tableParameters object provide the names and values for the table parameters, respectively.

Names for table parameters MUST be identifiers.

3.1.9 Metadata: reportParameters

A report parameter is a report-wide value that can be referenced in any table template definition.

Report parameters are defined by the reportParameters property (see Section 3.1), which can be either an object or a string.

If the value is an object, the keys and values of the object provide the names and values for the report parameters, respectively.

If the value is a string, it is treated as a URL to a CSV file, as defined in Section 3.2.1. Relative URLs are resolved relative to the primary metadata file. The first and second fields of each row provide the name and value for the report parameter, respectively. Rows in which the first field is the empty string are ignored. There is no header row; if the first field of the first row is non-empty, it is treated as the name of a report parameter.

Names for report parameters MUST be identifiers.

The links object defines relationships between facts in the report.

The object has the following form:

The links object defines a list of relationships between S and each fact in TF, with link type G and link type T.

3.1.11 Identifiers

An identifier is a value used to identify a table template, table, row or column. Identifiers MUST be a valid NCName and MUST NOT contain the . character.

3.1.12 Extending metadata

In order to allow re-use of common definitions, xBRL-CSV metadata may be split across multiple JSON metadata files. The metadata file for a report may reference another metadata file via the extends property on the documentInfo object. Referenced metadata files may in turn extend other metadata files, forming a chain.

The metadata files in an extension chain combine to form the effective metadata for the report. The process for combining metadata files is described in this section.

When a metadata file, X, extends another metadata file, Y, the properties of the following objects in X are combined with the corresponding object in the effective metadata of Y:

X MUST NOT define a key that exists in the corresponding object in the effective metadata of Y.

The base file, Y, MAY specify that an object is final (see the final object). In this case, the specified object MUST NOT appear in the importing file, X.

Though the final object itself cannot be marked final, it still respects the constraint that keys must not be redefined during extension. This means that X inherits, and can only add to, final definitions from Y. If another metadata file, Z, extends X, Z cannot extend any objects that are specified as being final in either X or Y.

All objects not listed above are not extensible, and, with the exception of documentInfo, MUST NOT appear in more than one metadata file.

The documentInfo object MUST appear in all metadata files, and MUST have at least the documentType property. All metadata files within a chain of referenced metadata files MUST have the same value for the documentType property.

3.1.12.1 Table template extensibility

It should be noted that whilst an extending metadata file cannot modify the definition of a table template from another file, it can add additional dimensions to an imported table template by specifying them as reportDimensions. If this is undesirable, the base file should include reportDimensions in final.

3.2 CSV data tables

A data table is a CSV file that is referenced by table object in a metadata file. Data tables are interpreted as described in this section.

3.2.1 File format

CSV files MUST follow the format defined in RFC 4180, with the following modifications:

This gives the following modified version of the RFC 4180 ABNF grammar:

file = [header line-end] record *(line-end record) [line-end]

header = name *(COMMA name)

record = field *(COMMA field)

name = field

field = (escaped / non-escaped)

escaped = DQUOTE *(TEXTDATA / COMMA / CR / LF / 2DQUOTE) DQUOTE

non-escaped = *TEXTDATA

COMMA = %x2C

CR = %x0D

DQUOTE = %x22

LF = %x0A

line-end = (CR LF) / CR / LF 

TEXTDATA = %x09 / %x20-21 / %x23-2B / %x2D-D7FF / %xE000-FFFD / %x10000-10FFFF

3.2.2 Transposed tables

A table would normally be mapped to a CSV file such that each row in the table maps to a line in the CSV file, and each column maps to fields at the same position within each row.

A transposed table is mapped to a CSV file with the axes of the table swapped. In this case, each line in the CSV file represents a table column, and fields with corresponding positions within a line respresent the table rows.

This arrangement can be convenient for tables have a large number of columns but few rows. In particular, it can be used with a single row to provide a natural representation for key/value pairs, by defining the "keys" as columns, and providing a single value for each column. Using a transposed table means that the values are more obviously associated with the keys:

column1,value1
column2,value2
column3,value3
column4,value4
...

The terms "row" and "column" in this specification refer to logical rows and columns within the table, and the mapping to lines and fields within the CSV file depends on whether the table is transposed.

3.2.3 Structure

The first row in a table is the header row. Each cell in the header row MUST either be empty, or contain a valid identifier.

Each value in the header row provides a column identifier for the column.

Column identifiers MUST be defined in the table template definition for the table.

3.2.4 Cell values

A cell has a value if it falls within the range of data provided in the CSV file and has a literal value of anything other than the empty string or the string #none.

The following special treatment is applied to cell values:

See Appendix A for a summary of the special values available in different contexts.

3.2.5 Unmapped cell values

A column that is referenced by a parameter reference in the same table template or the reportDimensions object is a parameter column. Note that a column MAY be both a fact column and a parameter column.

If a cell in a parameter column has a value and the column is not also a fact column, then the column MUST be the target of a parameter reference in the definition of at least one fact column that has a value in the current row.

Values MAY appear anywhere in a column that is not a parameter column. If such a column is not a fact column then the values will not be mapped into the OIM Report Model.

3.2.6 Unmapped parameters

All table parameters MUST be referenced by at least one fact column in the table template definition for the table which has a value in that column in at least one row.

All report parameters MUST be referenced by at least one fact column in any table template definition which is used by at least one table which has a value in that column in at least one row.

3.2.7 Column ordering

Parameter columns MUST appear before any fact columns that reference them.

4 Mapping from xBRL-CSV to OIM

This section describes how an xBRL-CSV document is mapped to an OIM Report Model. The starting point of the model is the report component, which corresponds to an xBRL-CSV report.

4.1 Report component

The report component has the following properties:

{taxonomy}
The list of URLs that is the value of the taxonomy member of the documentInfo object.
{facts}
The union of the sets of facts defined by each data table in the xBRL-CSV report (see Section 4.2).

4.2 Facts

Each cell in a fact column of a data table that has a value is mapped to an fact component, with the properties described in Section 4.2.2.

4.2.1 Definitions

4.2.2 Fact properties

The fact has the following properties:

{id}
See Section 4.2.3.
{dimensions}
See Section 4.2.5.
{decimals}
See Section 4.2.6.
{value}
The value of the cell, after applying the treatment described in Section 3.2.4
{links}
A set of link groups containing all relationships defined in the links object for which the current fact is the source.

4.2.3 Fact identifiers

The {id} property for a fact is in an xBRL-CSV report is the concatentation of the table identifier for the current table, the row identifier for the current row and the column identifier for the current column, separated by ., i.e.:

{table-identifier}.{row-identifier}.{column-identifier}

4.2.4 Row identifiers

A row identifier provides a unqiue identifier for a row that is used, in combination with the table identifier and column identifier to produce fact identifiers. The row identifier for the current row is:

The row number for a row is an integer corresponding to the position of the row within the table, with the value 1 corresponding to the row immediately after the header row (i.e. the second row of the table), and other rows numbered sequentially.

If a rowIdColumn is provided, the values in the column have the following constraints:

4.2.5 Fact dimensions

The {dimensions} property of a fact consists of the union of dimension values defined in the following locations:

  1. The columnDimensions object for the current column.
  2. The tableDimensions object for the current table.
  3. The reportDimensions object.

If multiple locations include the same dimension for a single fact, then the first location (considered in the order listed above) that specifies either a value or an explicit no-value is used.

Note that it is an error to provide a duration value for an instant fact. Where the period is obtained from a duration value via a parameter reference, a period-specifier can be used to obtain an instant value that is either the either the start or end of the duration.

4.2.6 Decimals

The {decimals} property of a numeric fact is determined by the first of the following properties that is present:

  1. The decimals property of the column object for the current table.
  2. The decimals property of the table object for the current table.
  3. The decimals property of the documentInfo object.

If no such property exists, the value of the {decimals} property is "infinite".

Otherwise, the value of the decimals property is interpretted as follows:

Appendix A Syntax for special values (non-normative)

The table below shows a summary of the special values that are available for use as cell values in CSV files and xBRL-CSV metadata files. For reference, the equivalent representation in xBRL-JSON is also included.

Unles otherwise stated, all values in monospaced font are string literals. This means that in JSON files they should be enclosed between quotation marks.

Meaning In CSV files In xBRL-CSV metadata (JSON) In xBRL-JSON
Value from CSV column col n/a $col n/a
Period start from CSV column col n/a $col@start n/a
Period end from CSV column col n/a $col@end n/a
Nil value of fact/dimension #nil null (JSON literal) or #nil null (JSON literal)
Empty string #empty Empty string or #empty Empty string
The string literal #val ##val ##val #val
The string literal val# val# val# val#
The string literal $val $val $$val $val
The string literal val$ val$ val$ val$
Nonexistent fact Empty cell n/a No fact object
Decimals value of 3 3 3 (JSON number) 3 (JSON number)
Infinite decimals Empty cell or #inf #inf No decimals property
Inherit decimals (defaults to infinite) n/a4 No decimals property1 n/a
Absent dimension Empty cell or #none #none2 No property for dimension
Inherit dimension n/a4 No property for dimension1 n/a
No period3 Empty cell or #none #none2 No period property
Inherit period n/a4 No period property1 n/a
No entity3 Empty cell or #none #none2 No entity property
Inherit entity n/a4 No entity property1 n/a
No unit3 Empty cell or #none #none2 No unit property
Inherit unit n/a4 No unit property1 n/a

Notes:

  1. When specifying xBRL-CSV metadata, omitting a property for a dimension or decimals will result in the value being inherited from higher-level metadata, if any. If no such value exists, the fact will not have that dimension (or in the case of decimals will have the value "infinite").
  2. Specifying a property in xBRL-CSV metadata with a value of #none (an explicit no-value) overrides any inherited value. Facts will not have the corresponding dimension (or will have infinite decimals), even if a value is specified in higher-level metadata.
  3. Period, entity and unit are optional in the Open Information Model, but are required for some or all facts in xBRL-XML. xBRL-XML defines specific values for the XML syntax that map to the absence of these dimensions in the model.
  4. Where xBRL-CSV metadata specifies that the value for a dimension (or decimals) is to be obtained from a column in the CSV file (using the "$col" notation), it is not then possible to inherit a value for that dimension. An empty cell in the relevant column will result in the absence of that dimension (or infinite decimals).

Appendix B Example xBRL-CSV report (non-normative)

The table below shows an example data set:

Loan ID Firm ID Country of incorporation Local currency Fixed Rate period Deposit amount (home currency) Deposit amount (local currency) Loan to value Rate
L001 09209384832 GB GBP 2017-01-01 to 2019-12-31 €1,000 2,100 45.6% 5.1%
L002 12312312123 FR EUR 2017-06-21 to 2019-03-20 €1,100 1,374 38.8% 4.6%
L003 02393823489 DE EUR 2017-07-01 to 2017-12-31 €800 871 34.8% 4.3%

Appendix B.1 Metadata example

A sample metadata file for the above table is shown below.

{
    "documentInfo": {
        "documentType": "http://xbrl.org/YYYY/xbrl-csv",
        "namespaces": {
            "ld": "http://xbrl.org/oim/conformance/firm-loans",
            "lei": "http://standards.iso.org/iso/17442",
            "iso4217": "http://www.xbrl.org/2003/iso4217"
        },
        "reportDimensions": {
            "entity": "$entityLEI"
        },
        "taxonomy": [
            "https://xbrl.org/oim/conformance/firm-loans.xsd"
        ]
    },
    "tableTemplates": {
        "loan_data_templates": {
            "columns": {
                "country_inc": {
                    "columnDimensions": {
                        "concept": "ld:CountryOfIncorporation"
                    }
                },
                "deposit_amount_hc": {
                    "columnDimensions": {
                        "concept": "ld:DepositAmount",
                        "period": "$fixed_rate_period@start",
                        "unit": "iso4217:EUR"
                    }
                },
                "deposit_amount_lc": {
                    "columnDimensions": {
                        "concept": "ld:DepositAmount",
                        "period": "$fixed_rate_period@start",
                        "unit": "$local_currency"
                    }
                },
                "ltv_end_fr": {
                    "decimals": 3,
                    "columnDimensions": {
                        "concept": "ld:ExpectedLoanToValueRatio",
                        "period": "$fixed_rate_period@end"
                    }
                },
                "rate": {
                    "decimals": 4,
                    "columnDimensions": {
                        "concept": "ld:InterestRate",
                        "period": "$fixed_rate_period"
                    }
                }
            },
            "rowIdColumn": "loan_id",
            "tableDimensions": {
                "ld:Firm": "$firm"
            }
        }
    },
    "tables": {
        "loan_data": {
            "template": "loan_data_template",
            "url": "loan-data-facts.csv"
        }
    },
    "reportParameters": {
        "entityLEI": "lei:00EHHQ2ZHDCFXJCPCL46"
    }
}

Appendix B.2 CSV Example

Corresponding CSV data (load-data-facts.csv) is shown below:

loan_id,firm,country_inc,local_currency,fixed_rate_period,deposit_amount_hc,deposit_amount_lc,ltv_end_fr,rate
L001,09209384832,ld:GB,iso4217:GBP,2017-01-01..2018-12-31,1000,2100,0.456,0.051
L002,12312312123,ld:FR,iso4217:EUR,2017-06-21..2019-03-20,1100,1374,0.388,0.046
L003,02393823489,ld:DE,iso4217:EUR,2017-07-01..2017-12-31,800,871,0.348,0.043

Note: