Job Location: Bangalore/Bengaluru
Roles and Responsibilities
- Design and implement data engineering projects. • Integrate multiple data sources to create data lake/data mart Perform data ingestion and ETL processes using SQL, Scoop, Spark or Hive • Knowledge of new components and various emerging technologies in on-premises and Cloud (AWS/Azure/Google)
- Collaborate with various cross-functional teams: infrastructure, network and database
- Work with various teams to setup and manage users, secure and govern platforms and data and maintain business continuity through contingency plans (data archiving etc.)
- Monitor job performances, manage file system/disk-space, cluster & database connectivity, log files, manage backup/security and troubleshoot various user issues
- Design, implement, test and document performance benchmarking strategy for platforms as well as for different use cases
- Setup, administer, monitor, tune, optimize and govern large scale implementations
- Implement machine learning models on real time input data stream
- Drive customer communication during critical events and participate/lead various operational improvement initiatives
Desired Candidate Profile
- 3 – 5 years relevant experience in data engineering
- Exposure to any or all latest data engineering ecosystem platforms such as AWS, Azure, GCP, Cloudera and Data bricks
- Sound knowledge of Python/Scala/Java
- Good knowledge of SQL / NoSQL databases and data warehouse concepts
- Hands on experience of working on databases such as Sql Servers, PostgreSql, Cloud infrastructure, etc.
- Excellent knowledge of data backup, recovery, security and integrity
- Sound knowledge on Spark, HDFS/HIVE/HBASE, Shell Scripting, and Spark Streaming
- Excellent communication skills
- Must be proficient with data ingestion tools like Sqoop, flume, talend, and Kafka
Submit CV To All Data Science Job Consultants Across India For Free
