0. Use python 3.7/3.8, and please leave some comments and notes 1. A small crawler program on Windows; I can download the file and click an .exe for running it. This program has an interface for some simple operations: there is a search box and search button in the page, a status bar, a conversion button, an import button, and a visualization button. 2. First, I can type a word in the search box of the program’s interface. 3. According to the word I typed in step2, the program searches for the corresponding words on 4. Create a list1 to document the explanations on of the word searched in step2. Please notice that only level-1 explanations are needed (as shown in the jpg "无标题", the level-1 explanations of “this” are the sentences rounded by
1. 需求: 根据用户输入的目标关键词，通过多维度搜索，获取尽可能全面的结果，基于这些结果进行联想，从而获取跟目标关键词相关的尽可能多的结果。提供用户数据抽取关键词输入界面，在结果中根据用户提供的关键词进行结果抽取，并进行自动合并以及去重。 2. 输入 手动输入关键词 导入Excel 从数据库获取关键词 3. 输出 结果另存为Excel(2010以上版本) 直接导入DataBase 可根据用户需求选择导出保存为Office文件(Word,Excel) 4. 开发工具 Eclipe J2ee 5. 数据库 MSSQL2008 6. 是否需要界面 需要 7. 开发文档 程序流程图 程序开发文档 代码中必要位置添加注释 如有用到开源项目，请说明 8. 其他说明 该项目需要程序能够进行二次开发，支持分布式部署，并发多线程进行搜索，自动平衡各节点资源消耗 完成后的项目要作为一个独立的项目存在，将整个项目连带所用到的JAR包一起打包作为最后的交付内容。
SQL注入/代码审计/数据库权限/渗透测试技术 接渗透单子高手，联系电子邮箱：aqcp#（#改成@）语言要求：中文 熟练掌握各种内网渗透测试工具与相关知识，精通常见安全攻防技术：sql注入、xss、文件上传、文件包含、命令执行等漏洞 熟悉linux、unix、Windows、oracle、J2EE等各种环境下WEB的安全配置与安全检查及WEB漏洞防范； 熟悉各种脚本语言(asp,php,jsp,java,net、perl、python等)具备独立挖掘web安全漏洞与入侵手段 具备一定的网络安全知识，对网络安全结构、系统漏洞、入侵检测、病毒防护等有深入理论基础和实践经验； 熟悉渗透测试黑客攻防的步骤、方法、流程、熟练掌握各种渗透测试攻防工具有实际渗透经验者优先； 求大虾接单，脱裤，要求，电话，手机，姓名，邮箱，其他的一概不要。每笔单1000美金起步，根据站的难度而定价格,具体可以详谈，寻找长期合作. 态度认真、中介勿扰,请技术实力说话。
整个产品拍卖系统采用B/S结构、J2EE技术框架，采用基于J2EE技术是为了充分利用众所周知的Java所具有独特的跨平台的优势。 技术框架流程 技术标准 1．基于J2EE的技术标准 2．基于SOA的标准 3．基于XML的技术标准 4．缓存技术 5．Comet技术 逻辑层采用Spring 3.0技术，主要实现对数据层的操作、事务管理及系统调度等。 持久层采用OR-MAPPING工具Ibatis 来完成与数据库、缓存池的交互。 控制层采用Struts2.3.4 来完成与数据库、页面的交互。 系统安全控制采用目前业界优秀的安全组件Acegi Security来保护网络资源及领域对象。 数据访问接口采用符合SOA（SCA ）标准的JAVA +Webservices（Xfire） + XML技术 。 客户端与服务器的数据交换采用缓存技术（Ehcache）减少数据库服务器的压力，并提高客户端和服务器端交互效率。
...the integration of a web crawler that will search and gather coupon codes from various online sources. Key Requirements: - Development of a coupon code aggregator website - Integration of a web crawler that will search for coupon codes on a daily basis - A search bar where users can enter websites like they are browsing normally and coupon codes will be applied similar to how microsoft edge or Honey does it - The website should have the functionality to automatically apply coupon codes to a shopping cart - Ability for users to sign up and receive daily coupon codes and deals - Ability for the website admin to be able to add deals or coupon codes sourced from advertisers/partners - Payment functionality Ideal Skills and Experience: - Proficiency in web...
i need a backend preferably in nodejs. one endpoint. that submit some birh date data. then the backend should request the html from anotherwebsite, fill its form click on submit get the data and send them
I'm looking for an experienced crawler specialist to help me crawl databases, and specifically collect certain datasets. Key Project Requirements: - Crawler Type: The focus for this project will be on databases, not websites or APIs. - Objective: The primary goal is to extract specific datasets from these databases. - Data Format: The extracted data should be delivered in CSV format. Ideal Skills and Experience: - Proficiency in database crawling and data extraction is a must. - Previous experience in collecting specific datasets from databases is highly preferred. - Familiarity with working with CSV data format. Please share relevant experience when bidding.
I'm seeking a meticulous and experienced professional in Octoparse to construct a responsive web crawling template. This crawler will be solely focused on an eCommerce platform, aiming to capture extensive product information. Key Responsibilities: - Capture specifics like the URL, category, and SKU for every product. Ideal Candidate: - Knowledgeable in Octoparse. - Proven experience in data mining and web crawling. - Detailed understanding of eCommerce platforms and product specifications. - Ability to maintain precision across multiple product categories and large datasets. I look forward to seeing your proposals and discussing this project further with you.
I'm looking for an experienced web crawler developer to build a tool for me. This tool needs to be able to extract text content and links from various websites. I need data from the whale watch section of The full crypto address within the links should be associated with their name counter part in a spreadsheet or organized file. I have included a highlted picture demostrating the fields I need. They should be done for the full 6 pages. Requirements: - Experience: Please detail your experience in web crawling and provide examples of previous work. - Delivery: The extracted data should be delivered in a spreadsheet file. It's crucial for the freelancer to have in-depth knowledge of web scraping and data extraction. The tool should be eff...
I am seeking an expert to build a web crawler specifically suited for extracting a broad range of data from social media platforms. The unique objective of this project is to efficiently and accurately amass textual content, images, links and more from social media sources. Specifically, the project will revolve around: - Gathering user posts and related content - Documenting user interactions, encompassing likes, comments, and shares - Extracting comprehensive details from user profile Ideal candidates for this project will have: - Proven experience in Data Extraction and Web Crawling - Knowhow of Social Media platforms and their structure - Proficiency in programming languages used for web crawling - Excellent attention to detail and a com...
I am seeking an expert capable of creating a highly detailed and intricate graphic line art rendition of a lattice boom crawler crane with a luffing jib (two independently controlled booms). This rendition must strictly capture the exterior features of the equipment ( there's a lot). Your focus should also be on all the components that make up the crane. I am open to starting with a more basic line art (outline) and seeing what we need to add. This would be the first design for a construction apparel line You need to have a meticulous eye for detail and at least have a portfolio that showcases any automotive art.
...of creating a customized web crawler for eBay and AliExpress. The primary function for this crawler will be to accept a list of keywords of any length and return specific data about those products. Here's what I need in detail: • Find top hot selling products at any given time • Crawl eBay and AliExpress for specific product data based on entered keywords. • The data retrieved should include product names, quantities sold and locations of sellers, images, • The crawler should be capable of providing this information for a variable timescale, ranging from 1 to 30 days in the past. • The results should be sortable. Php or python preferably The task would best suit a freelancer with previous experience designing similar cr...
I'm looking for a skilled professional who can crop the complete content of a website and save it in a zip file to be deployed on a new server. The website is a small Wordpress with about 41 pages (as per sitemap). Content must not be saved as Wordpress, but static html to be deployed on another server. Key Requirements: - Crop all website content including subpages. - Save the cropped content in a compressed zip file. - The purpose of this cropping is to save the content for future use. - all files must be included, esprecially but not limited to images, js, css, ... - inner paths must be relative paths. If you're confident in your ability to deliver this and can ensure all the content is correctly saved, please bid accordingly. IMPORTANT READ THIS: Auto-generated bids wil...
I'm looking for an expert to design a web crawler to extract email addresses from specific company websites. The ideal candidate should be experienced both in web crawling and email data extraction. Key Tasks: - Crawl targeted company websites for email data extraction - Ensure the accuracy and integrity of all extracted emails Skills and Experience: - Experience with web crawling and data extraction technologies - Proven track record in email data extraction - Excellent analytical and problem-solving abilities - Familiarity with common web interrogatories for more efficient data capture is an asset - Strong attention to detail in order to capture complete and valid email addresses The successful candidate will deliver a list of valid emails ex...
As the project lead, I'm seeking a highly skilled developer to construct a Scrapy web crawler for multiple sites. Although I omitted website specifics for now, rest assured we will provide the necessary details post-engagement. The ideal candidate is capable of: - Building an optimized web crawler with Scrapy - Extracting text data proficiently from various web pages, accurately and efficiently For the application, please provide: - Detailed project proposals outlining your approach Experience in web crawling design and Scrapy framework are a must for this project. Your technical capacity to generate the explained particulars is critical for success here. This is a golden opportunity to exhibit your proficiencies and contribute to an ex...
...://www.facebook.com/oefabc/&docid=bW04nRkl5sarEM&w=1631&h=1631&hl=en-US&source=sh/x/im/m1/3&kgs=944815503abde356&shem=abme,trie Key Design Elements: - Main Colors: Blue, Yellow, and White. Certainly, the combination should reflect intensity, energy and teamwork. Ideal Candidate: The perfect freelancer will have a significant experience in logo designing, specifically for sports brands. They should also have a deep understanding of color theory and be able to deliver high-quality designs in a tight timeframe. Your Task: - Design high-quality logo, using the specified colors - Delivery required ASAP - Final product should be optimized for different platforms-print, web, within-stadium display
I require an IAM role with specific permissions to interact with s3, glue, and Athena on AWS. This role will allow for efficient management of the AWS stack, specifically enabling the creation of a crawler to populate a glue catalog database and tables, and also allowing interaction with an Athena workgroup. What I need: - Read-only access: This permission should allow the IAM role to view S3 bucket contents, view Glue catalog information, and view Athena workgroup details. - Write access: This level of permission should enable the IAM role to create or update tables in the Glue catalog and modify Athena workgroup settings. Ideal Experience and Skills: - Solid understanding of IAM roles and permissions on AWS - Experience with AWS s3, Glue, and Athena - Capability to work on AWS I...
I need an experienced and proficient website crawler who can help me extract Text content from Database Tables on my website. Key requirements: • Acing in finding, fetching, and structuring data. • Proficient in delivering the retrieved data in Plain text format. • Emphasis on fetching Text content from the Database Table, thus understanding of database structures is a must. Ideal skills include web scraping, data mining, and database understanding. It would be best if you had an eye for detail as accuracy is key in this project. With your expertise, you should be able to navigate through complex website layers to get to the desired content.
We are in need of a knowledgeable and skillful programmer who can edit on GoLang code. The main task involves Ethereum network crawling. We already have a running code to crawling the Ethereum network. However, we still need to do some changes on GoLang code (designing crawler), such as partition the ID_space into sections, and crawl using multiple crawlers in Ethereum not only single crawler as what we have. In addition, there might be more few changes on the code. The ideal candidate will: - Understand DHT-based network and Kademlia - Knows a little about Ethereum (blockchain) - Have experience in GoLang - Possess a deep understanding of DHT network crawling (p2p) - Have previous experience with crawling networks Your responsibility will be building on top of running code...
...and collaborate closely with our team to deliver exceptional solutions. Qualifications: Strong proficiency in Java programming Prior experience developing server-based document management systems Proficiency in the struts, hibernate Framework Demonstrated experience in software development Bachelor's or Master's degree in Computer Science or a related field Hands-on expertise in Java/J2EE technologies Strong working knowledge of Core Java, including OOPS principles, Multithreading, Classes, Collections, and Memory Management Exceptional problem-solving and analytical skills If you're ready to take your Java development skills to the next level and contribute to innovative software solutions, we want to hear from you! Join us at MedXview and embar...
Google Jobs Scraper project requiring skills in .NET, Python, Selenium and SQL. The task is to create a crawler to extract job listings based on predefined queries, manage the data in a structured manner, and provide an interface for controlling and monitoring the scraping process. The solution requires adept handling of rate limits and the use of rotating proxies for efficient data scraping. Data To Be Scraped: - Scrape all specified job listing data points. - Store and manage the scraped data in a SQL database for effective access and organization.
I'm currently facing issues with my web crawler code. Specifically, it's not managing to scrape all the required data and it's throwing errors inconsistently. This code has been built to scrape information from certain e-commerce platforms. The ideal freelancer for this task should show: - Proficiency in Node.js - Extensive experience with Puppeteer and web crawling, especially focusing on e-commerce platforms - Exceptional debugging skills Your challenge will be to inspect the code, identify the underlying issues and then fix them to ensure a comprehensive and seamless data scraping process. I look forward to collaborating with someone who can effectively resolve the issues and optimize the performance of this web crawler.
your website is causing the load spikes in question. You can consider reviewing this on your side additionally. We have also noticed that there is a lot of bot/crawler access to the website. Access requests originating from bots are registered every 2 to 5 seconds.
Hi, Today i have created a Scrapy crawler to crawl the website , but i'm not able to crawl the website because when i run the crawler, any request i do to the website, i got the 403 error to me. I just neeed you fix my code to do a request at someway the website return the code 200 instead of 403. I'm sending my code attached. Send me a print screen in the chat showing you received code 200 using the same url, then i will hire you to send me your code.
Looking for a seasoned AWS expert to assist in optimizing EC2 instances, S3 storage, and Lambda functions for improved performance. The ideal candidate will have: Mainly working on Aws glue jobs , S3,lambda, Crawler, step functions, dynamo DB ,Clouldwatchs, Appflow, parameter store , API Gateways, ETL Jobs,splunk logs CloudFormation - important big bucket,Spunk, Jenkins pipelines* CI/CD * , YAML for creating templates and manifest files * Sql mandatory for date base * using Postman for API calls * Python and spark Musk know
Remote Opportunity Job Description: Good knowledge of the product including BPM Fundamentals, Archi...knowledge of the product including BPM Fundamentals, Architecture Components, Technical Features such as Designer Features, Configurations in Appian etc. Should have hands-on experience in design and development in Appian BPM including components such as Tempo, Mobility Features, Forums, Discussions, Pages, Smart Services, Reports, Deployment etc. Should have good knowledge of Java, Angular JS, J2EE, Ajax, JavaScript, JS, XML, XSLT etc. Appian Certification L2. Should have handled teams of size 5-7 and should be able to mentor the team. Excellent analytical and communication skills. Required Skills: Java, Angular JS, React JS, Appian Apply only if you have good experience in abo...
Wir haben ein Projekt bei einem Großkunden. Dabei brauchen wir ein Architekten, der jetzt vornehmlich in eine Task Force mitarbeitet, um mit uns gemeinsam folgende Fragestellung zu bearbeiten - Governance - DEV-OPS - Sizing Fragestellungen - Operational Manual überarbeiten bzw. neu aufsetzen Projektsprache: Deutsch/ Englisch 50% remote und 50% Stuttgart onsite Wichtige Componenten sind Java / Wildfly / PostgresSQL/Jenkins. Wir stellen das Entwicklungsteam. Der Kunde sollte die genannten Aufgaben bearbeiten. Braucht dort aber Unterstützung. Rückmeldungen bitte unter E-Mail MFG Frank Mercier ewocon Advisory GmbH
...developer to create a specialized web crawler focused on extracting vital data from e-commerce websites, primarily targeting ''. Key Requirements: The web crawler must adeptly parse text-based content and efficiently navigate the site's structure to extract pertinent information and links. Extensive expertise in Web crawling, Web scraping, and Data mining is essential for this project's success. Prior successful engagements in web scraping projects centered on e-commerce platforms would be greatly advantageous. Proficiency in programming Python is required. Familiarity with web scraping frameworks and libraries like BeautifulSoup, Scrapy, or Selenium is highly desirable. While the initial focus is on ...
...registration and login: A system for users to create new accounts and log in as desired. 2. Chat functionality powered by GPT: A well-equipped Chat-bot that's capable of handling inquiries based on proprietary documents. 3. Comprehensive integration with LLM (GPT): A system that is able to process different types of documents (text, web-pages, PDFs etc.), inferring from both internal and external sources. 4. Web-link Web-Crawler: An engine that can gather data from the web and cross-link it with Azure. 5. Advanced SEO: Optimization techniques to help improve website ranking and visibility. 6. Google Maps integration: An option for website users to connect with Google Maps. 7. Multi-platform accessibility: The site will need to function seamlessly ...
I'm looking for a Google Domains admin to help me fix an issue with page indexing and device permissions for my in-domain email addresses. Tasks: - Analyzing and resolving an issue with page indexing (the Google crawler is complaining about something but I can't tell what) - Setting device permissions for in-domain email addresses (I'm having trouble because users seem to be currently prohibited from logging in on their own devices and I can't find a setting to allow better functionality) Experience and Skills: - Expert knowledge of Google Domains - Proven track record of resolving similar issues Applications: - Only applications showcasing past work in Google domain administration will be considered. Please include the details of previous projects you have ...
This project involves conducting a dark web websites crawl with the primary objective of gathering dynamic intelligence and monitoring online activity. Key Details: - ACHE Crawler needs to be used to crawler dark web and collect available websites urls Required Skills and Experience: - You need to be proficient in Python programming and have previous experience in web crawling, especially on the dark web. - Familiarity with network security and understanding of dark web dynamics is highly preferred. - Familiarity with ACHE Crawler, Docker and Scrapy is needed. On completion, a detailed report of the findings, as well as the script code used, will be expected.
I am seeking an experienced Intermediate level Fullstack JAVA trainer who can structure a comprehensive learning experience. While I am unaware about the specific areas in JAVA we'd like to cover, your job would be to plan, organize and deliver a well-rounded training schedule. Ideally, you should have solid experience covering Core Java, J2EE (JavaEE), and the Spring Framework. For your application, instead of focusing on experience or past work, I am interested in receiving detailed project proposals. In your proposal, please explain: - How you will structure the course - Teaching methodologies you'll use - Timeline for the curriculum Your training should cultivate confidence and competence in JAVA, cater to a variety of different learning styles, and foster a robu...
...to create a data crawler that can efficiently harvest text data. The main area of focus for this project is to extract data from the MCA (Ministry of Corporate Affairs) of newly registered companies. Key Functionalities: • The crawler should be designed to extract text data. • It should specifically retrieve MCA data concerning newly registered companies. Expected Results: The solution should result in an efficient, accurate extraction of data from the designated sources, presented in a convenient, readily usable format. Timeline: The project needs to be completed within a month. Ideal Candidate: The ideal candidate for the project will have: • Extensive experience in data crawling and programming. • Demonstrated ability handling similar projects....
As an indie game developer, I am in need of a Unity Game Designer to create multiple engaging, cave-like dungeons for a fantasy-themed dungeon crawler. The ideal candidate should possess: • Expertise in Unity3D and procedural level generation mechanisms • Creativity to design dungeons encapsulating a dark and eerie vibe cursed with an infused ancient mystery. Key Project Components: • Procedural Dungeon Creation: Dungeons must be designed using procedural generation techniques to bring forth the essence of unpredictable gameplay and replayability. Upon refreshing, a new layout should be generated. • Triple-trap system: The incorporation of multi-tier traps such as spikes and pitfalls, moving platforms, and hidden passages, is a must. The designer must come u...
I have a requirement to develop a Software as a Service (SaaS) solution which includes a crawler and a data presentation dashboard. Crawler: - The crawler needs to be designed to extract both text and images. - This data extraction has to happen on a daily basis. Dashboard: - The dashboard should present the data collected by the crawler in an understandable manner, ideally through visual graphs or charts. - It must also include a search function so specific data can be easily located. Ideal skills for this project include: - Proficiency in SaaS development. - Experience in buildling crawlers. - Strong knowledge in data visualization techniques. Please note, any additional recommendations for enhancements to this project are welcomed and appreciated....
...recommendations for the management of the focal firm. You are expected to use at least 250 reviews of each selected firm (not more than 1 year old review data). You may choose 3 different firms or 3 different units of the same firm. e.g., Taj hotel reviews across 3 different cities or reviews of 3 different hotels. For collecting recent review data, you are allowed to take external help (publicly available crawler scripts, scripts from your group members or batchmates). The actual data analysis scripts and report must be prepared based on your own original work. Prepare a report – in not more than 12 pages, excluding title, references, and appendix – outlining the steps you have followed while performing customer preference analysis. You can explore different text ana...
Check out my website for model download: What I will do with the wagon: My business goal is to create a software that can take any track of any roller coaster from the noLimit software, open it in my software, configure it, and click build. Then, I'll have a 3D printed track with all supports. I'll sell the software with the STL of the wagon. Futuristic Roller Coaster Wagon Design I'm looking for an experienced 3D designer specialized in Solidworks to create a body for a roller coaster wagon. This project requires a keen eye for aesthetics, as well as a solid understanding of mechanical engineering principles to incorporate a working seat restraint mechanism with a lock system. The design will be built upon an existing base that includes a differential-like
I'd like a web crawler that can go out to a list of US university websites and find documents (or pages) that describe certain aspects of university policy. I'd like the documents labeled and organized in a folder. I will provide a set of likely keywords and the domains to be searched. The domains will be provided in a spreadsheet containing university names and homepages. There are approximately 3000 domains to be searched. The keywords will be samples and may not be the exact names of the policy. A flexible search is required. A successful project will return the academic integrity or misconduct policy for each university as a PDF along with a link to the original document online. Note: this is part of a research project for a university professor.
I'm seeking a detailed-oriented freelancer to develop a data crawler capable of collecting specific information from various e-commerce websites and social media platforms. The objective is to harvest both text and numerical data efficiently. Key Requirements: - Target Data: Retrieve product prices as efficiently as possible. - Sources: Focus on well-known e-commerce platforms and social media sites where product listings are prevalent. - Essential Skills: Proficiency in web scraping tools, data extraction techniques, and handling bot detection mechanisms. - Experience: Past projects involving e-commerce or social media data scraping are highly beneficial. - Output Expectation: The data should be organized and provided in a structured format such as CSV, JSON, or Excel ...
...related to entertainment content. This program should be able to automatically retrieve, validate, and ensure the functionality of the live stream links across mobile platforms, specifically iOS and Android. **Key Requirements:** - Develop a crawler that targets live stream links dedicated to entertainment content. - Ensure the retrieved live stream links are live and functional 24/7. - The solution must be operational and fully compatible with iOS and Android platforms. **Ideal Skills and Experience:** - Proficiency in programming languages suitable for web crawling and automation (e.g., Python, JavaScript). - Experience in working with APIs and understanding of how to manipulate them for live stream data retrieval. - Knowledge in mobile platform compatibility, specific...
I need a skilled web crawler expert to extract specific information daily from a designated grocery website and deliver the collected data in a spreadsheet format. My primary focus is on the following details: - Product names - Prices - Images - Descriptions (including size and other relevant information) Ideal Skills and Experience: - Proficiency in web crawling tools and technologies - Experience in extracting and parsing large datasets - Ability to schedule and automate crawling tasks on a daily basis - Expertise in data formatting and spreadsheet management The extracted data should be neatly organized and presented in a spreadsheet, with clear and distinct columns for product names, prices, images, and descriptions. This job requires a high level of attention t...
I'm seeking an experienced developer to create a web crawler specifically for the bookmaker site. The primary task for the crawler is to efficiently and accurately extract odds information focused on football events. Required Skills & Experience: - Proficiency in Python or another scripting language suitable for web crawling. - Previous experience working with or creating web crawlers. - In-depth understanding of the bookmaker betting industry, specifically for football. - Proven ability to extract data from complex websites effectively. Project Specifics: - The main data point to extract will be the betting odds. - The crawler should primarily focus on football events. - It is key that the crawler is designed to handle changes ...
Role Overview: Develop and maintain high-quality web and mobile applications using Java-based technologies like J2EE, JSP, Struts, Spring, Hibernate, Angular JS and more. Requirements: 1. Bachelor's or Master's degree in Computer Science or related field (B.E./B.Tech./MCA) 2. 5+ years of experience in web/mobile app development 3. Proficiency in Java, HTML5, CSS3, and JavaScript 4. Strong understanding of design patterns and configuration management tools 5. Experience with Android programming and MySQL is a plus.
Creating a fiverr crawler that scraps gigs, uses api to get gig details and using proxy exports data to the csv file using a desktop application..
Poszukujemy osoby zdolnej do zaprogramowania narzędzia, które umożliwi nam analizę i selekcję informacji na stronach www. Celem jest stworzenie zaawansowanej bazy danych, która pozwoli na efektywne sortowanie i filtrację stron według specyficznych parametrów, takich jak poziom aktywności użytkowników. Naszym priorytetem jest zgromadzenie i organizacja informacji w sposób umożliwiający łatwe zarządzanie i dostęp do danych.
We've a opening for a Scrapy Developer. Description: - Designing crawler for extracting data for various market sources. - Utilising country switcher whenever possible for multi version of sites. I,e en-us, zh-hk version of site. Desired skills: - Scrapy - Splash (nice to have) - Ajax/XHR API simulation for JS rendered site - Git/Bitbucket - Phyton - Jira (nice to have) First you make one crawler for a fixed price. You will receive access to a sample Bitbucket project with an example crawler. When the first crawler you've build is good we can continue on an ongoing basis (9 EUR per hour).
I am looking for an experienced developer who can design and implement an application that will break image-based captcha and in return issue a certificate. I need full professionalism on this since the system will be used to bring efficiency in our processes. The scope of work includes: - Designing a crawler that bypasses the image-based captcha - Test the crawler on different browsers to ensure efficiency - Documenting the testing processes and results - Identifying any potential risks and ways to mitigate them The ideal professional must: - Have demonstrable experience in designing and implementing crawlers - Have exceptional knowledge of image-based captcha - Show integrity, professionalism, and confidentiality in all the tasks - Be able to start work immediately - S...
We are looking for a crawler/scrapper for a specific android app. e.g. This app has a map which is overlaid by a marker on specific locations, once you click on it it shows further details like address, availability, amenities available etc. We would like a tool which on a given frequency (calibratable) pulls the data from the app and update in a excel file ( or similar database). The tool can be developed in any environment, but should be executable on Windows machine. The deliverable is the tool and its source code.
Generate a spreadsheet with ingredients, ingredient URLs, and total cost for 2,500 recipes as an initial POC to demonstrate that recipe data can be accessed from API provided by the client and then the crawler can retrieve and save the ingredient information from Tesco's website.
I'm looking for a Python deve...listings data Preferably, this information should be acquired from Google and any other online resources. The successful bidder should have experience in Python, web scraping, data extraction, and working with APIs. Knowledge in text mining and natural language processing will be an added advantage. Your scraper must be capable of handling hundreds of webpages effectively and efficiently. We have already completed 50% of the project as we have developed the crawler in-house using Python , we are now doing the AI part. We want to handle the project to a developer to do the remaining website functionalities and to enhance the crawler. Figma and other files will be shared along with the source code. $300 is my budget. I look forw...
