Job Location: Bangalore/Bengaluru
Data Engineers Data Pipeline Implementation for MNC
Job Overview:
To work on development, monitoring and maintenance of Data pipelines across AE clusters.
Primary responsibilities:
Develop, Monitor and Maintain data pipeline for various AE plants.
Create and maintain optimal data pipeline architecture.
Assemble large, complex data sets that meet functional / non-functional business requirements.
Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability.
Work with stakeholders including the Data officers and stewards to assist with data-related technical issues and support their data infrastructure needs.
Work on incidents highlighted by the data officers.
Incident diagnosis, routing, evaluation & resolution.
Analyze the root cause of incidents.
Create incident closure report.
Qualifications
BE Degree in computer science or equivalent
English on business level
Min 3 years of Experience in Data management within the area of software development rollout and maintenance
Min 2 Years of specific experience with data pipeline development, monitoring and maintenance.
Work Experience Required: 3-5 years
Additional information
Key Competencies-
Experience in developing data pipelines using spark & scala.
Experience in debugging pipeline issues.
Experience in writing python and shell scripts.
In-Depth Knowledge of SQL and Other Database Solutions
Having a strong understanding of Apache Hadoop-based analytics
Technical skills:
Hands on experience on Spark sql and Spark streaming.
Hands on experience on Scala language.
In depth knowledge of SQL and NoSQL (HBase).
Good in writing Shell and python script.
Good understanding of Hadoop ecosystem.
Hands on experience on InteliJ, Github /Bitbucket, HUE.
Understanding of ETL process.
Submit CV To All Data Science Job Consultants Across India For Free
