*Advanced* Web Scraper

Advanced Web Scraper of eCommerce content: Data and Pictures MUST work FAST, FAST, FAST, EFFICIENT and MASTERLY *** Do not bid if you cannot deliver what we’re asking for! 25,000 Products: - Products and details are the same between both sites and the domain address is only slightly different - Properties and specs are NOT the same for every product, nor are they all formatted the same Site 1a: Retail Site 1b: Wholesale - Private Account 50,000 Products - Products and details are the same between both sites, but domain addresses are fundamentally different - Properties and specs are NOT the same for every product, nor are they all formatted the same, some categories may also be different Site 2a: Retail - Turnkey site Site 2b: Wholesale - Private 2,500 Products - Products and details are often the same between both sites, but there are differences and the domain addresses are fundamentally different - Properties and specs are NOT the same for every product, nor are they all formatted the same, some categories may also be different Site 3a: Retail Site 3b: Wholesale 2,500 Products Site 4: Retail 1,500 Products Site 5: Retail Output file will need to be integrated… - as a database to populate a website - to update a website site with product and detail changes as well as stock - for financial analysis - Price comparison, wholesale vs retail; profit and loss, break even MUST HAVE FEATURES: - Windows based GUI - Crash Proof - Pause, stop, continue options - Visual and audible alerts when scrape is complete - "Save to..." option to save scrape to a specific folder and defaulting future scrapes to that folder - Date and Time stamp for each scrape, so previous scrapes are NOT overwritten - Picture scrape (1000x1000 or bigger) - Check box to scrape product data and/or picture - Check box to scrape only certain categories (include main and sub-categories) - Have "Check all"/"Uncheck all" feature - Scrape History: Continue from last successful scrape point should scraper fail, but I would expect this to rarely or never happen - Scrape History: Continue from last successful scrape point should our computer crash - Scrape delay feature (in seconds) - Scrape # Products between delays - Auto update database website price comparison, archiving previous data - Output Report in Excel: outlines the scrape process and highlighting "Errors" - Output options: Excel; csv; XML, etc (options for use with MySQL; Drupal; Zend/Symfony) - Option to merge all scrapes into one file to compare product details and pricing ... Including a filter to associate comparative details, but the product name is different, despite being the same product - Password and Password change option for private sites - Simple options for user to review and update scraper due to website changes (including, but not limited to, product, data, http fields, links, address, etc.) - If there are other useful features I have not noted, please specify what they are and how they are of use - Must provide support and updates for scraper functionality and accuracy *** A high attention to detail is a MUST because product properties/specifications and their respective labels vary. If you have read and understood the details outlined, at the beginning of your message, tell us: What software are you an expert in to create our scraper (Python, PHP, C, other) * All our projects are tracked using Freelancer Time tracker, but our "billing and progress" is tracked via Snagit Desktop Video Recorder. We have ongoing projects with website developers and graphic designers/illustrators. This process works for us and our contractors. It's simple to use and as long as you have a strong work ethic and integrity, it is a simple process to record the work you do for us and upload the videos on a mutually shared Dropbox or Google Drive for us to receive. Bidding on this project means you are accepting the terms outlined in this post and the attached employment agreement.