Live TPC: The bargain hunter’s radar

Live TPC is an auto-updating listing of items posted on a very popular Philippine buy and sell website tipidpc.com for computer related stuff. It basically screenscrapes TipidPC’s front page every 5 seconds and then determines which among the items are new. It maintains a 5-second old cache of the front page and “pauses” the 5-second polling when the focus is off the page. I made use of Troy Wolf’s PHP Screenscraping class (with some modifications) and The Fade Anything Technique script by Adam Michela. Currently, its been tested to work with Firefox 2.+ and Safari. It was reported to be working on IE7 too but I have yet to confirm myself.

The idea is, to keep from unintentionally DDoS‘ing tipidpc.com, I implemented a 5 second old cache so that at most, Live TPC will only screenscrape from their servers every 5 seconds, no matter how many users are using Live TPC. This routes the bulk of the traffic on my server. To work around this, I actually have to implement some pseudo load balancing from different “scraping” servers which I’ll be implementing only if the need arises.