Find Jobs
Hire Freelancers

Data collection and connection

€18-36 EUR / hour

En curso
Publicado hace casi 6 años

€18-36 EUR / hour

We are looking for a freelancer that can support us in requesting huge datasets (up to 500 Mio lines) from APIs, enrich them with more data from other APIs and store it in a database or file. Therefore you should know how to do such things very effective and fast, since we cant wait weeks to get the things performed. We are flexible in which programming language you work, but it should run on a linux server at the end. The project will be extended and can take a few weeks with up to 200 hours in total. First task is to get all domain lists from [login to view URL] and save them in the format domain | tld in a text file. Then we want all of these domains resolved with their ip adress. Please let me know how you would technically solve this and how many hours it would take you.
ID del proyecto: 17494981

Información sobre el proyecto

16 propuestas
Proyecto remoto
Activo hace 6 años

¿Buscas ganar dinero?

Beneficios de presentar ofertas en Freelancer

Fija tu plazo y presupuesto
Cobra por tu trabajo
Describe tu propuesta
Es gratis registrarse y presentar ofertas en los trabajos
Adjudicado a:
Avatar del usuario
Hi, I'm very interested in working on your project. I have experience working with data sets of this size from previous projects, and I am able to deliver quickly. The first task I would solve by fetching the list from [login to view URL]<USERNAME>/<PASSWORD>/ and then parsing that using standard Linux CLI tools to generate the text file. I estimate around 4 hours for this task. In general I would be using a mix of Python and Linux CLI tools, and if needed, PostgreSQL for storage. But I am also flexible regarding specific tools and languages, these are just my personal preferences. Hope to hear from you, Mathias M. Hansen
€25 EUR en 40 días
5,0 (1 comentario)
2,1
2,1
16 freelancers están ofertando un promedio de €26 EUR /hora por este trabajo
Avatar del usuario
hi there, I just checked your proposal. I am Senior PHP and Laravel Developer with 9+ years strong practical experience in building various kinds of web projects. Especially I have good experience with PHP, Laravel, CI, Cakephp, Mysql, Linux and familiar with several JS frameworks such as Angular.js, React.js and Vue.js. I developed number of projects so far and I can provide perfect results to you. How about work with me ? Please note that I am top 6th developer at freelancer.com with around 600 reviews. My ultimate focus is customer and team satisfaction by "shipping" the best work possible in as timely a manner as possible. I can guarantee high quality of code and high availability as I worked at home. I want to build trustworthy, reasonable contract with you. Best Regards, Heng
€30 EUR en 40 días
4,9 (443 comentarios)
9,8
9,8
Avatar del usuario
Hi plusserver, I have gone through your project "Data collection and connection" and could assure you the best of the solution. I am taking it as an opportunity and ready to deliver my best. please check our freelancer portfolio Also do have similar experience doing something similar to yours therefore I am looking forward to discuss and move ahead. Relevant Skills and Experience I have 10+ years of experience in Linux, MySQL, PHP, Python, Software Architecture I can do this within your budget too €18 - €36 EUR / hour Look forward to your response Thanks Mmadi We are looking for a freelancer that can support us in requesting huge datasets (up to 500 Mio lines) from APIs, enrich
€21 EUR en 30 días
5,0 (50 comentarios)
7,1
7,1
Avatar del usuario
Hello There, Greeting of the day! As per your project requirement, would be able to scrap data from a database. I can use PHP that would be suitable for Linux. Technically I would be able to resolve the issue and extarct all data in the format domain | tld in a text file. I need to run a custom script for it. I have been working as developer last 9 yrs. In My past, I have worked in India based MNC, Techmahindra Ltd. and also got an opportunity to work in el Segundo California and San Jose,USA onsite client location for development support. I was working with British Telecom on client support. I have extensive experience as the Full-stack developer. This is Ambikesh having experience of 8+ years in Software and web development. I have extensive experience in , PHP/WordPress,Joomla,Magento,Cake php, codeigniter,Laravel,Yii2.0,,HTML/HTML5,Bootstrap3.0,CSS3,Angular.JS, JQuery, SQL, My SQL, REST API's and SOAP API's I value consistent and continuous communication with you to understand your project concept and our experts do EXACTLY what you have envisioned. So it’s time to turn your ideas into reality with the help of experienced developer. So let’s get started on your dream project. We look forward to hearing from you. Thanks, Best Regards! Ambikesh
€18 EUR en 40 días
4,9 (55 comentarios)
6,9
6,9
Avatar del usuario
Hello, After reading your project details I believe I'm suitable for this project. As I'm expert on this kind of project with more than 5 years experience. Please feel free to contact me. I am looking forward to hear from you. Best regards
€18 EUR en 1 día
4,9 (139 comentarios)
6,8
6,8
Avatar del usuario
Hi I have have done it before , Kindly connect with your requirement. I have years of experience and expertise in a gamut of technologies that include: • Php, Magento, Laravel, Wordpress, Android & ios etc • Web (Html, Laravel & Core Php, Wodpress, Joomla) • Linux, MySQL, PHP, Python, Software Architecture Let’s connect and kick start the project !!! For any query, feel free to connect me.
€30 EUR en 40 días
4,8 (70 comentarios)
6,0
6,0
Avatar del usuario
Hi, We can use Python for grabing data from API, and use mongo DB as destination, the combination give high performance in large data management tasks, it also run in linux server we are an IT company from India, more than 10 years experience in ndutry, come in chat we can discuss more
€30 EUR en 40 días
5,0 (21 comentarios)
5,3
5,3
Avatar del usuario
I have good team for website design and development, I am terribly excited for the chance to figure with you in accomplishing your goals. We have developed thousands of websites in many programming languages for clients all over the world. plusserver might you be able to please inbox me more insight about this venture
€30 EUR en 40 días
4,9 (53 comentarios)
5,1
5,1
Avatar del usuario
Hi. Pro account is needed for api access (50/month). I do this kind of stuff on aws so it depends on you how much are you willing to pay for bandwidth, storage, etc.
€36 EUR en 40 días
5,0 (9 comentarios)
4,7
4,7
Avatar del usuario
Hi, as an expert in software development, I am very interested in your project since I have rich experiences in database management. Almost all of apps use its own database to store data which is needed in the program. Through my programming history I found that Oracle database is very effective in huge database management system. Since the database contains huge number of lines, selection of database engine is not a perfect solution. The DB design is also very important to reduce the duplication. If you hire me, I will do my best to design the database using with 3NF (this is the recommended database normal form). Also, in my opinion, both Oracle and MySQL can be used. Let's talk about the project in more details. I am looking forward your response. See you later. Thanks.
€30 EUR en 40 días
5,0 (3 comentarios)
1,6
1,6
Avatar del usuario
I am an Web Scrapper doing for many websites as freelancer. delivering into excel database through spider or phyton i can do your project.
€20 EUR en 40 días
0,0 (0 comentarios)
1,7
1,7

Sobre este cliente

Bandera de GERMANY
Cologne, Germany
4,8
13
Forma de pago verificada
Miembro desde feb 26, 2010

Verificación del cliente

¡Gracias! Te hemos enviado un enlace para reclamar tu crédito gratuito.
Algo salió mal al enviar tu correo electrónico. Por favor, intenta de nuevo.
Usuarios registrados Total de empleos publicados
Freelancer ® is a registered Trademark of Freelancer Technology Pty Limited (ACN 142 189 759)
Copyright © 2024 Freelancer Technology Pty Limited (ACN 142 189 759)
Cargando visualización previa
Permiso concedido para Geolocalización.
Tu sesión de acceso ha expirado y has sido desconectado. Por favor, inica sesión nuevamente.