Scraper wanted
Search
Software Development -Scripts & Utilities
Description
I require a well known UK directory site scraping.
I am only interested in two categories from this site.
I understand that the combined total of results to be in the region of 55k. I require these results to be in a format that can easily be imported into a database, eg CSV would be a good format.
I would like to keep and reuse the scripts afterwards.
Known problems are that the site (apparently) does not allow pagination past ten pages. To circumvent this, I have a list of about 3k of Postcodes to reduce pagination down to less than 10 pages, per search.
Also the site is aware of potential scrapers and does not allow sequential pagination viewing, so it has to be random.
The use of Proxy servers may have to be incorporated as IP banning is also known.
This project is a re-listing due to the problems mentioned above.
Thank you! Additional Info (Added 10/9/2009 at 4:15 EST)... It's been pointed out that I omitted what info I need scraping!
From each result, I need the...
1) Company Name
2) Tel Number
3) Full Address including Postcode
Project Bids
| Expert | Location | Message | last login | ||
|---|---|---|---|---|---|
|
|
3
|
Infodigita |
Pakistan |
|
|
|
|
3
|
Acmesoft_cn |
Pakistan |
|
|
|
3
|
Codewalla |
Michigan USA |
|
||
|
|
2
|
Codeshastra |
India |
|
|
|
|
2
|
Endeavoursoftware |
India |
|
|
|
|
2
|
Anyware |
Nebraska USA |
|
|
|
|
2
|
Keith_duncan |
Pakistan |
|
|
|
|
2
|
Fusioncses |
Georgia USA |
|
|
|
2
|
Cryo |
Connecticut USA |
|
||
|
|
2
|
Dougallbright |
Utah USA |
|


