I need someone who knows php and has experience in screen scraping with writing results to a MySql database. I have found a decent PHP scraper which submits a search term to multiple search sites, visits sites and scrapes some data according to a regular expression. Specifically what I need is:
1. At the moment the scraper takes a single search term string. I want my script to look up search terms that I have provided to it (into a db ?)
2. I need a way of getting the search terms from my home PC to the server (into the db) for the use of the script.
3. The scraper currently just sends the data it scrapes to a DB but I want the table to have a little more detail: URL Visited (domain), Scraped Data, Search Term.
4. A simple user interface to control the script through which contains Start, Stop buttons plus a few stats (eg # sites visited, # scraped elements, % through search terms etc)
5. An email sent to a nominated email address identifying the bot to state that it has finished.
6. Installation of the above (not sure if shared hosting is possible for this or would need to be a VPS - freelancer to advise thoughts / opinions on this) and demonstration of the installation process (I may need to replicate myself so need to be able to do it).
So, should be bread and butter for a seasoned PHP developer with some scraping and server experience I think.
The successful freelancer should:
a. Be VERY communicative and not make any assumptions without clarifying with me.
b. Be willing to have communication (typed) over Skype. I am in Australia but can stay up till whatever time suits.
c. Be proactive - I am open to suggestions on better ways of doing things / considering other things if there is merit.
Think that's it. Any questions let me know. Steve.
This includes (basic) server setup on VPS, you can use managed web hosts, but they can shut you down more easily because this will require a lot of resources.. (not guaranteeing they won't shutdown vps, but its a bit better when you want to push resources, you're allowed to do it on VPS) PM me when you are ready!