Skip to content

Dataset

FilingFacts: an open dataset of SEC filing questions for AI

2,243 questions on 402 companies with answers computed from SEC XBRL filings, for testing whether AI models get finance numbers right. Free under CC BY 4.0.

Download

Each release is one JSON Lines file, with one question on each line
ReleaseQuestionsCompaniesWhat it isFile and SHA256
v1, all2,243402v0's question types plus restatements: what a company first reported for a period, against what it reports now.filing-facts-v1-all.jsonl (2.1 MB)889b70205de40640b52c7f343d92bf5c638edc0ec61cbee08ec6cc7884b3a8a0
v1, fresh24155Only facts filed after 2026-04-01, for testing models on filings newer than their training data.filing-facts-v1-fresh.jsonl (0.2 MB)4587aa097b2182e6c901d02d439ba3b63eae6eee7a1608731ee454f196fe65de
v01,882402The first release. The published benchmark runs use this set.filing-facts-v0.jsonl (1.7 MB)c01b9e3e660ff341017532718e6bc8f7fe96d1ffa68d86236c53e73ae3149d7d

Question types

Number of questions of each type in each release
TypeWhat it asksv1, allv1, freshv0
lookupOne value as reported in a named filing.40255402
changeThe percent change of one figure between two years or balance dates.40255402
ratioA ratio of two figures for the same period, such as a margin.40254402
net assetsTotal assets minus total liabilities at one date.33847338
restatementThe value first reported for a year, which the company may have changed since.36115None
unanswerableA period the XBRL filings do not cover. The right answer is to say so.33815338

One question from the fresh set

Using AAR CORP.'s annual XBRL data, by what percentage did net deferred tax assets change from its balance as of 2025-05-31 to its balance as of 2026-05-31? Give the change relative to the earlier value's magnitude, as a percent to two decimals.

Answer: 14.72 percent.

The filing facts behind this answer
FigurePeriod endValueFiledAccession number
DeferredTaxAssetsNet2025-05-31489000002026-07-220001104659-26-085459
DeferredTaxAssetsNet2026-05-31561000002026-07-220001104659-26-085459

Every answer names the filing facts it comes from, down to the accession number, so you can check any item at the SEC.

Load it

curl -O https://canlicapital.com/datasets/filing-facts/v1/filing-facts-v1-all.jsonl
shasum -a 256 filing-facts-v1-all.jsonl

python3 -c "
import json
items = [json.loads(line) for line in open('filing-facts-v1-all.jsonl')]
print(len(items), items[0]['question'])"

Results and tools

Limits

  • Answers are computed from the companies' XBRL data and rechecked by a separate program. No expert has reviewed them yet; you can help check them.
  • XBRL can differ from the filing's text, and a company can tag a figure wrongly. A computed answer inherits that.
  • 15 restatement items in v1 ask for a year's figure but cite a shorter period. The errata lists them; score them out.