Data

Every corpus, and how far it has come.

Every corpus moves along the same pipeline. This page says where each one stands.

How a corpus gets here
01

Choose the field

Only fields where the answer has to come from the original document, and where the licence lets us hold it.

02

Take the originals

Fetched at the publisher’s own cadence, at the crawl delay it asks for, and followed from then on.

03

Govern, then store

De-duplicated, parsed into retrievable units — sections, articles, fields — the original kept beside them.

04

Read it into structure

A model turns each document into fields, summaries and key points — the structure filters work on, and the reason a reader can take one layer at a time. How hard that reading is decides which model gets it.

05

Index, then serve

search returns cards, read goes brief → detail → full, ask answers with citations. Nothing comes back whole unless it was asked for.

04 · The extraction is split across three tiers
Flagship700B+

The judgements a later step will trust: disambiguation, holdings, anything a mistake would propagate.

Standard~100B

Structured extraction at volume — fields, summaries, translations, the long middle of the work.

Swift~30B

What is cheap to check and expensive to skip: keywords, classification, normalising names and units.

Dataset Scale Updates Stage Domain Open the row
Research6

Whole corpus refined by LLM: a TLDR and keywords per paper, a summary per section, section map with token counts.

Source arXiv · open access

One LLM-researched profile per person (education, career, representative work, open source); Google Scholar metrics rendered live, staleness flagged.

Source Public homepages, Scholar, OpenAlex

Every citation sentence in the fellows’ own papers is extracted ahead of time, so any paper can be matched by title, arXiv id or DOI with no API call. 5.26M references parsed, 5M sentences classified by type and stance — this is what the appraise domain answers from.

Source arXiv full text + 15 public fellow and academician rosters

Every ExPORTER category plus patents, in hand; the current fiscal year arrives through the API. In the grants domain with NSF and CORDIS — about 1.3M projects.

Source NIH RePORTER · public domain

Raw exports in hand. Indexed with NIH as one grants domain; institutions and investigators still to be normalised against the researcher corpus.

Source NSF Award Search · public domain

In the grants domain beside NIH and NSF (106,859 projects, ids prefixed cordis:); amounts are in euros. Institutions still to be normalised against the entity backbone.

Source CORDIS · EU open data

Law8

Article-level LLM refinement: heading, norm type, subjects, conditions, consequences, key figures, cross-references resolved to article ids; zh/en translation.

Source Official gazettes and legislative portals

“Court documents” is a wide bag — judgments, rulings, mediations, decisions, notices — and the published ones run into the hundreds of millions. We take the judgments out of it: 2.65M criminal, complete, and 14.8M civil, administrative and enforcement, backfilled to the end. LLM brief per judgment: cause, disputed issues, facts, circumstances, reasoning, outcome, cited articles, amounts. Behind the briefs sit 34.55M original documents — more than the index holds, the surplus still without a brief — split into five sections: parties, claims, facts, the court’s reasoning, the decision. The verbatim text is not served by default: it carries party names, dates of birth and addresses, so the API gives the parsed layer and the text itself needs the cases:fulltext permission. Every response carries coverage.

Source Published judgment documents

3,438,487 opinions answer through the Cases domain as country=us; extraction has run over 10.07M and the rest follows as the store behind it lands.

Source CourtListener, incl. the Caselaw Access Project · public domain

The National Archives have confirmed our use counts as computational analysis, which needs a transactional licence and carries no fee. The application is written and waiting to be sent.

Source Find Case Law, The National Archives

4.98M provisions answer through the US law domain, each extracted article by article. The live eCFR replaces the quarterly CFR snapshot; 928k provisions are marked superseded. Compilation licensed CC BY, attributed.

Source Open statute compilation (CC BY) + eCFR · official

CELEX notices in hand, text only rather than full metadata. Parsing into articles comes next.

Source EUR-Lex / Cellar · EU open data

CLML XML keeps structure and amendment notes. Fetched at the crawl delay the site asks for. Open Government Licence v3: attribution required.

Source legislation.gov.uk · OGL v3

1,008,931 documents answer through the Federal Register domain — action kind, issuing agencies, affected parties, effective and comment-close dates.

Source Federal Register / govinfo · public domain

Life sciences3

In the trials index, all 603,730. 586k refined: conditions, interventions, endpoints, eligibility in structured form.

Source ClinicalTrials.gov · public domain

Drugs index: 256,282 label versions keyed by set_id, each readable by section — indications, warnings, adverse reactions.

Source DailyMed, NLM · public domain

Cross-checked against DailyMed and kept as the machine-readable view. Adverse-event files (FAERS, MAUDE) need an explicit pull; label endpoints refresh daily.

Source openFDA · public domain

Markets7

392,258 8-K and 46.5k 10-K/20-F extracted item by item. The filings domain answers over two layers — 1.21M with vectors, and only the 105k 10-Q quarterlies by keyword alone. Financial figures come from XBRL, not from a model.

Source SEC EDGAR · public domain

Backfill finished after a paging bug was found: 16,133 reports recovered. 63k refined into the same item structure as SEC filings.

Source CNINFO, the designated disclosure venue

507k of them sit in the full store behind the filings domain: searchable by words, marked layer=pg, with coverage saying how much of the corpus carries vectors.

Source CNINFO, the designated disclosure venue

The authoritative source for US financial numbers — no model is asked to read a financial table.

Source SEC · public domain

Full decisions extracted — parties, conduct, legal basis, sanction — and in the filings index.

Source CSRC · official

10,434 letters extracted with their replies; in the filings index. Shenzhen inquiry and concern letters, Shanghai inquiries, plus disciplinary decisions.

Source SZSE and SSE · official

Feeds the entity backbone (organisation → federal money); it complements NIH and NSF rather than replacing their abstracts.

Source USAspending.gov · public domain

Security2

384k records answer through the vulnerabilities domain, with CISA exploited-in-the-wild flags and EPSS probabilities to rank by. Showing NVD data carries a not-endorsed notice.

Source NVD · CVE Program · CISA KEV · FIRST EPSS

Raw advisories in hand; they join the vulnerability records by CVE id.

Source OSV.dev · open

Standards1

Catalogue metadata only — number, title, status (72,083 recommended, 6,201 mandatory, 1,027 guiding), adoption, where the text lives. All 79,311 are in the standards index; the standards themselves are not ours to republish.

Source SAMR national standards platform

Reference1

4.58M organisations and 5.52M aliases answer through /v1/entities, with 5.30M links over 2.66M documents back to the organisation that filed, ran, received or labelled — 1.09M on an identifier, 1.13M on an unambiguous name, 3.08M adjudicated by a model and marked as such.

Source ROR CC0 · GLEIF CC0 · MeSH public domain

Yours1

The same pipeline behind every dataset here — parse, section, refine, cross-link — run on your documents, under your keys.

Source Public contribution or private engagement

Planned Chosen, licensing checked, not fetched yet.Raw Originals downloaded and kept current; parsing not started.Parsed Split into retrievable units — sections, articles, fields.Refined A model has extracted, translated and summarised each unit.Indexed In the search index; the API endpoint is being wired.Live Answering through the API.Awaiting licence The data exists; we will not touch it until the licence is in hand.

Inventory updated 2026-10-10