Convert a Template to a Website

General information for the business: We provide automotive trade sales Kind of development: Customization of existing program + training and adhoc support Description of requirements/functionality: We require a Web scraping specialist to assist us with setting up software that will scrape car dealers websites for automotive adverts and generate a database of vehicles in a MongoDB database. Scraper must be able to handle a range of websites some which have a straight forward cms, some that will be a little more complex. We would prefer to have a visual setup tool per website and manually set each websites scraper from our backoffice. This must be quick and efficient to setup, and adhoc cupport requests must be responded to quickly. More details available on request. Specific technologies required: Web scraping but open to options OS requirements: Windows, Linux Extra notes: We will want to setup 700 websites once the tool is in place and will need general training and adhoc support for more complex websites. The tool will need to scale to 5000 websites over a 12 month period, and will need to be hosted on Amazon. The tool will be licensed to Motortrademe Ltd once the project is completed, or we will handle licensing fees with any third parties and hosting partnerships directly. We will need to extract the following fields (as separate database fields): ** Please note: At times all fields will not be available, or they will be hidden in an advert details sub page for each advert, we will need to either leave fields out where not available, or enter sub pages to collect it all. Also, these fields may be in the same element and we will require some intelligence to break down a header column that is one line of text into our field format. Make Model Reg Year Fuel Type Doors Body Derivative Tranmission Colour Mileage (Number of) Owners Price Images (All available) Description From sites such as (Sample websites): [login to view URL] [login to view URL] [login to view URL] [login to view URL] [login to view URL] [login to view URL] [login to view URL] [login to view URL] [login to view URL] [login to view URL] [login to view URL] [login to view URL] [login to view URL] [login to view URL] [login to view URL] [login to view URL] [login to view URL] [login to view URL] [login to view URL] [login to view URL]