Nominet Zone File Released in May

[armistice]

Any one know what software should be used to open these files???

They are text files inside the zip, you can view them with any text viewer (although since the files are big notepad won't work well ).
You could import them into sql server, or even excel. Depends what you want to do with the data.

 

[ian]

They are text files inside the zip, you can view them with any text viewer (although since the files are big notepad won't work well ).
You could import them into sql server, or even excel. Depends what you want to do with the data.

Tried all that, too big to open in any mainstream software and excel struggles to import more than just the letter A's!

 

[redbird]

According to Nominet stats http://www.nominet.uk/news/reports-statistics/uk-register-statistics-2016/ there should be around 10.5m domains, I am only seeing less than 9.8m. Anyone else see more?

 

[GUI]

According to Nominet stats http://www.nominet.uk/news/reports-statistics/uk-register-statistics-2016/ there should be around 10.5m domains, I am only seeing less than 9.8m. Anyone else see more?

I'm seeing 10,700,061 domains.

 

[redbird]

9,783,544 in the csv dump file. I thought that was all of them?
Maybe my import into msql failed somewhere. I used LOAD DATA LOCAL INFILE

 

[AssetDomains]

Tried all that, too big to open in any mainstream software and excel struggles to import more than just the letter A's!

Haven't tried it on the Zone but bigdump was handy when I was looking to import the uk postcode data that runs into millions of records
http://www.ozerov.de/bigdump/

 

[redbird]

Haven't tried it on the Zone but bigdump was handy when I was looking to import the uk postcode data that runs into millions of records
http://www.ozerov.de/bigdump/

Is that level of postcode data publicly available, appreciate a link if free?

 

[AssetDomains]

Is that level of postcode data publicly available, appreciate a link if free?

You cant get the official royalmail one free but there are open source alternatives if you google

https://www.freemaptools.com/download-uk-postcode-lat-lng.htm I think I used

 

[ian]

Even the provided CSV file complains about size and only shows around 1mil results.

 

[ian]

Yes. Sorry. Too advanced for you.

Thanks David, great help as usual; you're a credit to Acorn

For anyone else looking for a simple solution, this text editor can cope with the file size:
https://www.emeditor.com/download/

 

[GUI]

Even the provided CSV file complains about size and only shows around 1mil results.

I guess you're attempting to open in Excel? If you're on Windows, grab something like Notepad++ to view the whole file.

 

[ian]

I guess you're attempting to open in Excel? If you're on Windows, grab something like Notepad++ to view the whole file.

Notepad++ suffers the same issue it seems. Will look at the config for it when I have more time. Thanks.

 

[GUI]

Notepad++ works fine for me.

 

[ian]

Notepad++ works fine for me.

.uk file size is much smaller than the .co.uk

 

[jmcc]

You might do better using Linux with vi. TextWrangler for Mac imports them. Clearly some regular expressions will need to be performed on the data to remove the unnecessary parts before processing.

If it follows the usual zonefile format, then it should be possible to use Awk or Perl to pull out the domain names if that's all that is required. However it will be necessary to filter out any non-domain name data.

Regards...jmcc

 

[GUI]

.uk file size is much smaller than the .co.uk

This is the full domain file, db-dump-20160516.csv, with the line number on the left, showing 10,700,061 domains.

Notepad++ works fine for me. Use whatever works for you.

 

[jmcc]

So is the zone file the full zone file complete with nameservers and glue records?

Regards...jmcc

 

[Edwin]

Could somebody kindly post an example line or two from the files, obfuscating the actual domains? Say one for a .uk domain and one for a .co.uk. That way, people monitoring this thread have some idea of the level of detail available without having to download the whole thing...

 

[jmcc]

Could somebody kindly post an example line or two from the files, obfuscating the actual domains? Say one for a .uk domain and one for a .co.uk. That way, people monitoring this thread have some idea of the level of detail available without having to download the whole thing...

Would be very helpful, Edwin,
The full .uk zonefile (including subs) should be around 27 million or more lines based on a quick line count of the .org zone.

Regards...jmcc

 

[armistice]

OK Edwin,

Todays zip file is ukdata-20160516.zip its 232mb compressed.
When unzipped it contains:
1.2G May 16 08:32 co.uk-20160516
211M May 16 09:07 db-dump-20160516.csv
983K May 16 08:32 ltd.uk-20160516
11M May 16 08:32 me.uk-20160516
80K May 16 08:32 net.uk-20160516
64M May 16 08:32 org.uk-20160516
51K May 16 08:32 plc.uk-20160516
4.4M May 16 08:32 sch.uk-20160516
68M May 16 08:30 uk-20160516

The first few lines of each file are: the NS records for the namespace.
Following that one line per domain nameserver record. This means that a domain with 2 nameservers has two lines. heres the first few lines from me.uk
0.me.uk. 172800 IN NS art.ns.cloudflare.com.
0.me.uk. 172800 IN NS beth.ns.cloudflare.com.
00.me.uk. 172800 IN NS ns1.chesternet.com.
00.me.uk. 172800 IN NS ns2.chesternet.com.
000.me.uk. 172800 IN NS ns33.domaincontrol.com.
000.me.uk. 172800 IN NS ns34.domaincontrol.com.
0003.me.uk. 172800 IN NS ns-uk.1and1-dns.co.uk.
0003.me.uk. 172800 IN NS ns-uk.1and1-dns.biz.
0003.me.uk. 172800 IN NS ns-uk.1and1-dns.com.
0003.me.uk. 172800 IN NS ns-uk.1and1-dns.org.