Domain List: every registered domain in the generic TLDs

Domain List is out, under Domain, a new kind beside the IP databases: every domain registered under the generic top-level domains, one name per row. As of October 6 it holds 279,011,357 names under 1,044 TLDs.

One name per row

A row is the name as registered: one label under its TLD, or two under .name, which registers first.last.name. Names are lowercase, and the 1,168,248 internationalized ones are in their ASCII form, xn-- and the encoding. There are no subdomains, no nameserver hosts such as ns1.example.com, no bare TLDs, and no country-code domains such as .uk or .de.

A week apart

Each build is the whole list, so two of them diff to the names that came and went. Between the lists of September 29 and October 6, 2,755,971 names appeared and 1,982,974 left, 772,997 more in all:

.games went from 85,018 names to 161,523 that week.

A build is sorted, so with last week's file kept beside this week's, the diff is two commands:

LC_ALL=C comm -13 <(zcat last_week.csv.gz | tail -n +2) <(zcat domain_list_v1.csv.gz | tail -n +2) > appeared.txt
LC_ALL=C comm -23 <(zcat last_week.csv.gz | tail -n +2) <(zcat domain_list_v1.csv.gz | tail -n +2) > left.txt

appeared.txt is where a lookalike of your brand shows up the week it's registered, and the full list seeds a crawler or sizes a market from every name rather than the ones a search engine happens to know.

Getting access

Domain List ships as CSV, rebuilt weekly, and its docs page has the schema. Sign up to evaluate it from its sample. A license is per database and never metered: see licensing, or talk to us about what you need.