BookScouter.com
Sell your books for the best price
| Home :: About our price scraper |
|
About our price scraper
In order to display prices to our visitors, we have written a number of custom scripts that "scrape" prices from other sites. The bot runs each time that a user requests a particular ISBN number only if it has not retrieved the same ISBN from a site within the past hour.
For site ownersOur site identifies itself by sending an HTTP User-Agent header of "BookScouter.com Price Scraper". I've tried to make the scraper as simple and efficient as possible.
- The script is doing nearly the exact same thing that a regular visitor would do.
- It is causing less load on your server and less bandwidth than a direct user because it requests only the required pages, and none of the associated images, stylesheets, or supporting files
- Pages are requeseted only when a visitor requests them. We don't proactively index your site