We're a firearms accessories online retailer and we sell a huge range (10,000+) of products. To get our quantities and prices up to date we have to scrape data from our suppliers online portals as APIs aren't available.
To date we've been using executable files based on java to scrape the data we need, but going forward, we want this to happen automatically from our AWS instance.
This project is to transition one of our scrapers to AWS. If successful, we will continue the work for our other suppliers (4 in total).
- I will provide the executable we currently use as well as the output that it produces.
- The scraper should automatically run at a pre-determined frequency (daily) and append time stamp data to a table which can be queried in SQL
- If the scraper fails there needs to be an alert
- The scraper needs to be well documented so that it can be troubleshooted
- An example of the type of fields that we required scraped is attached. This would include images.
23 freelancers are bidding on average $435 for this job
I am an expert in developing web crawling and data scraping solutions. I can provide you such a scrpit to transform your price portals to API . Looking forward to be working with you.
Hello. I can help you with this scraper. I have experience with scrapers and I can show you some examples of my work. Hope to discuss more details about the project. Regards, Dan.