Big Data Engineer

What do we do at BluePi: BluePi is a born in cloud organization with one of the main focus area being Big Data and Analytics. At BluePi we solve different business problems for clients from different industrial domains like Media, Retail, Logistics, Clothing, etc. At Bluepi we build, develop and deliver big data solutions including ETL, Data warehousing, DataLake, BI and Analytics using various technologies of hadoop ecosystem and other distributed computing frameworks like Storm, Flink, etc. Roles and Responsibilities: As a Big Data Engineer at BluePi, you will : 1. Integrate with different data sources to collect and process and transfer data in realtime and batch using different Big Data tools and frameworks. 2. Setup Pipeline for ingesting data in Realtime and Batch using tools/frameworks like kafka, Kinesis, DMS, Data Pipeline etc. 3. Write ETL processes using various tools/ frameworks like Spark, Storm, Talend, Glue, etc in Java/Python/Scala 4. Integrate with different databases like Hadoop ecosystem (like Hive, Impala, HBase, etc.), Redshift etc. 5. Setup data lake on S3 or similar storage services. Skills Required: 1. 3+ years of Hands-on experience on Big data tools and frameworks. Experience on Hadoop ecosystem, integrating and implementing solutions using technologies like - Hive, Pig, Mapreduce, HDFS etc. 2. Should have a proficient understanding of distributed computing paradigm and realtime processing vs batch processing paradigm. 3. Should be proficient in designing and implementing Dimensional models and Data Marts. 4. Should have an experience in writing ETL processes and connect to different data sources using different mediums/technologies. 5. Proficient in at least one of the following programming language - Java, Python, Scala. 6. Experience in AWS EMR/Redshift/Glue would be an added advantage.