The Known Unknown: A Regulator’s Perspective
In the latest instalment of his guest series for XBRL International, Björn Fastabend, head of the XBRL collection and processing unit at BaFin, Germany’s Federal Financial Supervisory Authority, shares a fascinating hands-on experiment in using AI to analyse XBRL data from the SEC’s EDGAR database.
Björn built a proof-of-concept tool to query financial data using a large language model — and quickly ran into a surprising problem. Asking the same question five times produced five different answers. The culprit? Not hallucination, but semantic ambiguity: with 80 different revenue-related concepts in the SEC taxonomy, the LLM simply didn’t know which one to use, and guessed differently each time.
His solution? A ‘Financial Domain Intelligence Layer’, designed to inject specialist domain knowledge into the querying process. This offers a compelling illustration of a broader truth: structured XBRL data and taxonomies are necessary but not sufficient for high-quality AI-driven analysis. Domain expertise matters too.
The piece is a thought-provoking read for anyone interested in the intersection of AI and regulatory data, and a timely reminder of the extraordinary promise and some of the current limits of AI in this space.
Read the post here.
