Scrapy Jobs

我有若干已确定的 APP 及其对应的网页链接，需要从中批量抓取用户个人信息，范围仅限“姓名和联系方式”。 目标结果： • 一段可复用的爬虫代码（Python + requests/Scrapy/Selenium 均可，按实际需求选择），能够按我提供的链接或包名自动登录/访问页面并抓取目标字段； • 一个示例数据文件，展示成功提取的字段结构； • 简明的运行说明文档，包含环境配置、依赖库与常见故障排查。 关键要求： 1. 只采集我指定的 APP 或网页上的信息，支持后期轻松更换目标链接； 2. 抓取字段仅限姓名、联系方式，确保输出格式一致； 3. 代码应处理简单反爬措施（如 User-Agent、Cookie、基本验证码识别或人工插入）； 4. 结果可直接写入 CSV 或存入 MySQL/PostgreSQL，二选一即可； 5. 交付前请先在少量数据上自测，确认字段完整无误。 如果你熟悉 Python 数据抓取并可在约定时间内交付，请在提案中简述你以前处理登录、分页或验证码的经验，并说明计划使用的主要库或框架。

I am building a real estate “market radar” for Uruguay that identifies undervalued properties and potentially motivated sellers. The goal of this project is to create a data workflow that automatically collects property listings from real estate portals and organizes the data so it can be analyzed in Excel. Primary data sources: • MercadoLibre Inmuebles (primary source) • InfoCasas Uruguay (secondary source, optional) No public records or auction data are required at this stage. PROJECT SCOPE The system should automatically collect property listings and store the data in a structured dataset. The scraper should run once per day and collect the following fields: • listing ID or unique identifier • property price • property location or neighborhoo...

No worries, that's way more workable. Here's the full post trimmed to fit under 10,000 characters: Nationwide Property Auction Web Scraping & Intelligent Alert System (Ongoing) About Us We're a commercial real estate investment firm that acquires distressed properties nationwide. We have the capital to close on any deal in the U.S. — our bottleneck is finding opportunities before competitors. We're building an automated system that monitors every property auction source in the country, filters against our criteria, and alerts us only on qualified deals. This is not a data dump project. We don't want spreadsheets with thousands of rows. We want a smart radar system that scans everything, filters ruthlessly, and only pings us when something matches. Long-t...

Florida Judiciary Web Scraper — Config-Driven, Resilient Architecture I need a Python-based web scraping application to collect judge data from all 20 Florida judicial circuits and output it to a standardized CSV. The tool must be built for long-term maintainability — when a circuit website changes layout, only minimal configuration updates should be needed, not code rewrites. Background: Florida has 20 circuits covering 67 counties. Each circuit publishes judge data differently: some offer Excel/CSV downloads, others publish HTML pages and subpages with varying structures. The master data source is: Required Output Fields: (CSV)ID, Type, Name, Lastname, Assistant, Phone, Location, Street, City, State, Zip, County, Circuit, District, Courtroom, Hearingroom, Subdivision(S...

I need a lightweight, repeatable scraper that gathers every publicly visible customer review talking about Bayer from social-media sources—right now the focus is on Goole. The crawler should pull the full review text, star rating (or reaction score, if available), reviewer name or handle, date, and the direct URL to each post. Please build it so I can run it on demand, ideally from a simple command line or Jupyter notebook. Python with requests / BeautifulSoup, Selenium, or Scrapy is fine; if you prefer another stack, let me know why it would be a better fit. Deliverables • Clean, well-commented source code • One sample export in CSV or JSON showing at least 100 live reviews • A short README explaining environment setup, run instructions, and how to alter s...

I need a Python-based scraper that pulls complete car-listing information from every day. At a minimum the script has to capture make, model, price, and mileage but, in practice, I want every publicly visible field on each listing so that nothing useful is missed. Here’s what matters to me: • Reliability – the code must navigate pagination, work around basic anti-bot measures (rotating user-agents / respectful delays), and throw clear errors if the site layout changes. • Clean output – save to CSV or an SQLite database with consistent column names, ready for later analysis. You’re free to choose libraries you trust (requests, BeautifulSoup, Selenium, Scrapy, Playwright, etc.); just document any setup steps and keep third-party dependencies to a mi...

No worries, that's way more workable. Here's the full post trimmed to fit under 10,000 characters: Nationwide Property Auction Web Scraping & Intelligent Alert System (Ongoing) About Us We're a commercial real estate investment firm that acquires distressed properties nationwide. We have the capital to close on any deal in the U.S. — our bottleneck is finding opportunities before competitors. We're building an automated system that monitors every property auction source in the country, filters against our criteria, and alerts us only on qualified deals. This is not a data dump project. We don't want spreadsheets with thousands of rows. We want a smart radar system that scans everything, filters ruthlessly, and only pings us when something matches. Long-t...

I need a Python-based solution that automatically gathers companies and shareholders data, pulls supplementary details via external APIs, and outputs a clean, unified dataset I can query at any time. Scope of the scrape • Sources: company websites, financial databases and relevant public records. • Website focus: company profiles, turnover figures and any available Demat / share-holding particulars. What the tool should do 1. Crawl or call the above sources, respecting and rate limits. 2. Parse the required fields, normalise names and IDs, then enrich each record through one or more APIs (for example OpenCorporates, Clearbit or any better suggestion you have). 3. Store results in a structured format (CSV plus an SQLite or Postgres option). 4. Offer a simple comma...

I need a reliable script or windows-application that automatically gathers text content from specified websites and online databases, then saves everything into a clean, well-structured CSV file. A Windows-software would be preferred. The crawler should be able to crawl the website and spider a list of urls for approval or automatically go through the website Or just scrape a given list of urls (from a txt-file) Key details • Sources: public-facing websites and shops (also with login using username:password) • Data type: text only—no images or binary files. • Output: one CSV per run, UTF-8 encoded, with a header row • should be able to read/exrtract data from !! various shops & websites !! -> generally i need a basic software + "plugins" fo...

We are looking for an experienced developer who can build an automated system to extract daily newly incorporated company data from the MCA (Ministry of Corporate Affairs) website – https://www.mca.gov.in. The system should automatically collect and deliver the list of companies incorporated each day in structured format (Excel / CSV / API / Database). Scope of Work: Develop a web scraping or API-based solution to extract daily incorporated company data from the MCA portal. The tool should automatically fetch newly incorporated companies every day. Data should include the following fields (minimum): CIN Company Name Date of Incorporation ROC (Registrar of Companies) State Company Type (Private Limited / LLP / OPC / Public Limited) Authorized Capital (if available) Regist...