This is primarily a web scraping application running on Linux/Apache/MySQL/PHP (LAMP) framework. Must use a batch framework to allow parallel processing of module execution. Must implement or extend a scraping framework which will allow the information to be scraped and stored in the database. Must allow modules to be easily added to the application and to the scraping tests.
The URLs which are scanned may return a 200 (OK) or one of several error responses. We'll only want to scrape data from the successful requests.
Must be able to create the Schema for the database.
Must be able to work well with me (good communication and willing to ask questions rather than make assumptions.) Must be able to complete the project by mid-June. Must be able to make this production-ready for use by non-technical users. Cannot cut corners or take short-cuts.
Must be willing to sign an NDA upon accepting the project.
Hello, just completed a scraping project and I believe that I could do this project with a fast turnaround and low cost. Please see my private message for more details.