Find Jobs
Hire Freelancers

Check for Valid URLs Rapidly - Using Rotating IP and Proxies - Mac OS

$30-250 USD

Cerrado
Publicado hace más de 7 años

$30-250 USD

Pagado a la entrega
I need a piece of software, preferably desktop based for Mac OS, that cycles through an increasing integer and enters it into a URL. The software should check if the URL leads to a valid product page, and if so, saving the integer in a list within the software's GUI. The general template would require the base site address, and the starting integer. Both numbers defined by the user. For instance: http://(sitenamehere)/cart/(integer here):1 So (sitenamehere) would be the site we're scraping info on, and (integer here) would be the number we are starting at. The software should increase by one integer for every check, and should use Rotating IPs and proxies to avoid bans. I would like to software to perform these checks as quickly as possible, looking to check about 800k-1 million product pages per day.
ID del proyecto: 11128270

Información sobre el proyecto

7 propuestas
Proyecto remoto
Activo hace 8 años

¿Buscas ganar dinero?

Beneficios de presentar ofertas en Freelancer

Fija tu plazo y presupuesto
Cobra por tu trabajo
Describe tu propuesta
Es gratis registrarse y presentar ofertas en los trabajos
7 freelancers están ofertando un promedio de $152 USD por este trabajo
Avatar del usuario
Hi, it's not a problem to make such software, however sites usually ban your ip after few hundred-thousand attempts. this can be avoided by using proxies, but it's not cheap to hold your own proxies, so I can quote you per-million pricing if you'll tell me which website you're scraping
$147 USD en 3 días
5,0 (344 comentarios)
9,0
9,0
Avatar del usuario
Dear Sir, I'm very much delighted to let you know that i did data scraping with PHP-cURL, PhantomJS, Node.js, Selenium from many sites. I just scraped the data from web site and then wrote the data in mysql database or excel or csv or xml file. I worked on many similar projects, I have big experience in data mining projects. I have written hundreds of web scrapers which scrape millions of pages each day. I'm ready to fulfill your requirement. I can finish this task in short time, with the best quality. I can assure 100% accuracy. Please give me the opportunity to do the work. With Kind Regards, Debdulal Roy Proshanta
$250 USD en 5 días
4,9 (101 comentarios)
7,5
7,5
Avatar del usuario
Hello, You project were very clear, but I have some questions: 1) "The general template would require the base site address, and the starting integer. Both numbers defined by the user. For instance: http://(sitenamehere)/cart/(integer here):1", is this going to be just one site to test? 2) "I would like to software to perform these checks as quickly as possible, looking to check about 800k-1 million product pages per day.", this will generally need a very high speed connection, like the VPS or a dedicated server has, so are you open for that?
$190 USD en 10 días
5,0 (7 comentarios)
4,5
4,5
Avatar del usuario
because I have developed the scraper before. for yellowpages, justdial and dx.com. auto increment in pages to scrape next page data and proxy script is implemented. I can provide you the software after minor changes. because your said requirement is already implemented.
$133 USD en 10 días
5,0 (6 comentarios)
3,1
3,1
Avatar del usuario
A proposal has not yet been provided
$155 USD en 1 día
0,0 (0 comentarios)
0,0
0,0

Sobre este cliente

Bandera de UNITED STATES
Seattle, United States
5,0
1
Miembro desde jul 21, 2016

Verificación del cliente

¡Gracias! Te hemos enviado un enlace para reclamar tu crédito gratuito.
Algo salió mal al enviar tu correo electrónico. Por favor, intenta de nuevo.
Usuarios registrados Total de empleos publicados
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Cargando visualización previa
Permiso concedido para Geolocalización.
Tu sesión de acceso ha expirado y has sido desconectado. Por favor, inica sesión nuevamente.