Job Location: Gurgaon/Gurugram
We are scouting for Data Engineer Lead Data Engineer – Big Data for one of our clients (a Fin-Tech organisation revolutionizing Financial Products through technology.)
Responsibilities :
– Gather and process raw data at scale.
– Design and develop data applications using selected tools and frameworks.
– Read, extract, transform, stage, and load data to selected tools and frameworks.
– Build and execute data warehousing, mining, and modelling activities using agile development techniques.
– Perform tasks such as writing scripts, calling APIs, write SQL queries, etc.
– Deploy and monitor products on AWS Cloud platform
– Monitoring data performance and modifying infrastructure as needed.
– Developing ETL flows based on HadoopBig Data stack technology: Apache Nifi, Hive, Spark, Kafka
– Troubleshooting and performance optimization for data processing flows, data models.
Sounds Like You?
– 2-7 years of Big data development experience with working proficiency in Java, Scala, Python to write data pipelines and data processing layers.
– Strong in Data Structures & Algorithms, Analytical, and problem-solving skills.
– Management of the Hadoop cluster, with all included services.
– Proficiency with Spark, Hadoop v2, MapReduce, HDFS.
– Experience with building stream-processing systems, using solutions such as Storm, or Spark-Streaming.
– Experience with the integration of data from multiple data sources.
– Experience with SQL, NoSQL databases.
– Experience with various messaging systems, such as Kafka, or RabbitMQ.
– Experience with confluent packages like Kafka connect, schema registry.
– Experience with ClouderaMapRHortonworks.
– Experience with AWS big data solutions.
– Good knowledge of Big Data querying tools, such as Pig, Hive or Impala
– Knowledge of various ETL techniques and frameworks, such as Flume.
Submit CV To All Data Science Job Consultants Across India For Free
