Data lake scale
Data lake scale is one of 31 measurements this engine publishes in full. It is regenerated from a real run rather than transcribed, and it is shown here with its own claim boundary so the number and its limits arrive together.
The measurement
- Schema
- canli.alphac-data-lake-scale.v1
Corrections this measurement forced
- '392K+ point-in-time fundamentals' was OVERSTATED — the true figure is 380,878. An overstated count is the only kind of staleness that matters.
- '8,436 US stocks, survivorship-free' matched neither store and has been replaced by the point-in-time membership count, which is what the phrase means.
- '3.5M+ hourly bars' and '2,820+ automated tests' were understated by enough to mislead in the other direction; the '+' made them true and uninformative.
Facts
Automated tests
- Value
- 3961
Definition. tests pytest collects under tests/, summed from its own per-file collection report
Crypto hourly bars
- Definition
- rows across every partition of data/lake/ohlcv, counted from parquet footers
- Partitions
- 2250
- Value
- 13662316
Crypto instruments with bars
- Value
- 775
Definition. distinct instrument ids with at least one hourly bar, live and delisted. NOT the traded universe, which is a much smaller subset chosen by the sleeve's own filters
Equity daily bars
- Definition
- rows across every partition of data/lake/ohlcv_1d, counted from parquet footers
- Partitions
- 116037
- Value
- 24750503
Equity instruments in pit membership
- Value
- 6835
Definition. distinct US-listed instrument ids the point-in-time universe store has ever admitted — the narrower answer, and the one that means 'survivorship-free research universe'
Equity instruments with bars
- Value
- 18015
Definition. distinct US-listed instrument ids that have at least one daily bar, including delisted ones — the broadest defensible answer to 'how many stocks'
Equity year span
- Definition
- first and last year partition present in the equity daily lake
- Value
- 1997-2026
- Years
- 30
Point in time fundamentals
- Partitions
- 105144
- Value
- 380878
Definition. rows in data/lake/fundamentals — one (instrument, fiscal period) record carrying an available_at stamp, which is what makes it point-in-time
Why each figure carries a definition. 'How many US stocks' has at least two defensible answers here and they differ by 2.6x: 18,015 instruments have bars, and 6,835 have ever been in the point-in-time universe. Publishing one without saying which it is would replace a stale number with an ambiguous one.
Check it yourself
Every figure above is read from /glassbox/data_lake_scale.json, the
artifact the engine wrote. Nothing on this page is typed by hand: the page is generated from that
file, so a figure that moves in the artifact moves here and a figure that is not in the artifact
cannot appear here at all.