bulk whois ? 100 million plus domains

[domainmanuk]

Does anyone know the very fastest way to check 100 million .co.uk domains with whois ?

If the domains are registered the details are taken from whois, if they are not registered they are discarded.

There seems to be restrictions of 1000 queries per ip per day, but on our figures we would be needing results for 500k to 900k per day. We could use proxies and change etc etc, but then the sites we are using will use their 'quota' for the day. For instance, what is the daily quote for who.is or domaintools.com ?

Any suggestions / help etc ?

Many Thanks

 

[Brassneck]

I think there are less than 10 million .uk domains (rather than 100 million) - however most ways of harvesting Nominet's whois records would be against their terms of service, hence the limits they set.

 

[grantw]

As per your other post, you need to do use the dac. If you have enough money you could register 20,000 domain names to your TAG which would then give you an additional 4,100,000 lookups per day on the time delay dac.

Failing that you'll need to do what everyone else had to do and put the time in!

Grant

 

[domainmanuk]

Hi,

Thanks for your replies -

Ive is use the real time DAC -can i query myself 432,000 per day ?

Or just 1000 ?

Is there anyway round me using the DAC for around the 400k a day ?

Thanks

 

[grantw]

The dac can be queried 432000 times a day, there is no limit on it apart from a speed limit.

Grant

 

[monaghan]

Everything you need to know is at http://www.nominet.org.uk/other/dac/

If there are other ways, then it is very unlikely that anyone will share their secrets on a public forum full of competitors

 

[domainmanuk]

Many Thanks Again.

Ok, here we go - ive managed a way to get a list from the 100 million of the registered .co.uk domains - the full 100 million will be scanned, and a Y or N will be returned.

This can be done in a week.

So i will then have approx 10 million of registered domains that i need to query against whois.

So - my question is if i use DAC. can i Query the list of 10 million domains with DAC real time, 432,000 per day with no issues - as i have been drawn to this part of the T&Cs -

5.3.6. knowingly provide more than an insubstantial part of the register or a sub-database of it (e.g. a database containing information released under the whois or the DAC) to any one third party or end user in any 24 hour period. For the avoidance of doubt and as an example, more than 1000 registered domain names taken from the DAC would be more than an insubstantial part;


What does this actually translate to for someone in my position ?

Thanks

 

[newly listed!]

Many Thanks Again.

Ok, here we go - ive managed a way to get a list from the 100 million of the registered .co.uk domains

There isnt 100million registered lol!