Job Location: Bangalore/Bengaluru
Data Engineer ( Associate / Sr. / Lead )
Exp : 2 – 7 yrs
Work Location : Bangalore
Job Description :
Activities :
– Develop Ingestion framework using Sqoop, Nifi, Kafka, Spark Streaming, WebHDFS, Python to enable seamless data ingestion process on to the Hadoop platform
– Exposure of handling structured, Un Structured and Streaming data
– Building data processing framework using Talend, Spark, HQL
– Enabling Data Governance and Data Discovery on Hadoop Platform
– Enabling Security Framework with Kerberos, Ranger, Atlas
– Enabling Data Pipeline Automation using DevOps tools
– Enable Job Monitoring framework along validations automation
– Create Automated testing framework.
Roles & Responsibilities :
– Understanding the business requirements
– Preparing the Design and work on Data Ingestion, Preparation and Transformation.
– Develop data streaming applications.
– Develop ELT code to move the data to curated zone.
– Developing end to end Pipeline automation
– Develop the scripts for data sourcing and Parsing scripts using Unix, Python.
– Debugging the production failures and identifying the solution.
– Monitor Performance of the Jobs and tuning them as required.
– Write extensive Unit and Regression test cases
– Create Project Technical Documentation
Skills :
Data Engineering: Hadoop, Hive, Sqoop, Hbase, Spark, Nifi, Presto, Kafka, Python, Sql, Unix Scripting, Talend, Atlas
Who can join Immediate – 20 days is only preferred.
Submit CV To All Data Science Job Consultants Across Bharat For Free
