Scrapy script development

I need an easy to use program designed for the purpose of locating where on the internet videos, hosted on a variety of platforms, have been embedded. This will be done by scraping websites iframe source code to find instances of embedded url code. EX. "http://www.youtube.com/embed/sV4_wHvP7b8" or "[url removed, login to view]; and other key players. It is important to note that this method is needed because access to the specific channels insight statistics and site player servers are not available.

Over one thousand websites could be provided to format a seed a spider's search, with a strong focus on some 400-500 webpages. Additionally, it would be great if it could be utilized to do larger internet wide searches scrapes for the same content. Located webpages do not need to be analyzed or registered in any way, simply have the page's url exported to a text document, excell spreadsheet or readable list.

It is suggested that the script available at [url removed, login to view] be tailored to this project, but any working program will do. Leads to free working software that could accomplish this task will be rewarded as well.

Big Data Sales 数据挖掘 网页搜罗 网页搜索

项目ID： #4049427

关于项目