Job Location: Bangalore/Bengaluru
We are looking for an experienced data engineer to join our team.
- Develops, maintains and documents data models, data dictionaries and data flow diagrams
- Develop and automate large scale, high-performance data solutions (batch and/or streaming) according to defined data strategies
- Build scalable data pipelines leveraging best-in-class technologies (e.g., StreamSets, Databricks, Confluent Kafka) that integrate with high-level orchestration frameworks such as Apache Airflow
- Design and implement multi-tier distributed data platforms in cloud environments (AWS, Azure) using infrastructure as code and automation tools (e.g., Terraform) as well as container technologies (Docker, Kubernetes)
- Develop complex processing logic, custom services and reusable libraries using Python, or C#
- Incorporate best practices from DataOps, DevOps and Agile software development, including build/deployment automation (CI/CD), testing automation, and operational monitoring/alerting
- Contribute to shared Data Engineering standards and tooling to establish best practices, to improve productivity, and to increase the quality and repeatability of solutions
- Support a wide range of data/analytics use cases across oil and gas, including cloud data warehousing and event streaming
- Assists data scientists in integrating diverse datasets for machine learning, and deep learning models
Desired Candidate Profile
- 6+ years of experience as a Data Engineer or Software Engineer
- Bachelor’s degree in Computer Science or a related field, or equivalent experience
- Programming expertise in at least one of the following languages: Python, C#, C++, F#, Java
- Familiar with the full software development lifecycle including unit testing, integration testing, etc.
- Production deployment experience to at least one of the following platforms: Snowflake, Databricks, Hadoop, AWS, Azure
- Experience with at least one mainstream data ingestion tool (e.g., StreamSets, Apache NiFi, Azure Data Factory) and distributed storage (e.g., Kafka, HDFS, S3/ADLS, Elastic) strongly preferred
- Strong working knowledge of SQL and the ability to write, debug and optimize queries
Nice to Have skill sets :
- General knowledge of database programming including relational (SQL, Maria) and non-relational (NoSQL, MongoDB, Cassandra) databases is a plus
- Experience in DevOps in Azure cloud environments is a plus
- General knowledge of scripting Languages such as Python, Scala, or PowerShell (at least one of them) is a plus.
Submit CV To All Data Science Job Consultants Across India For Free
