Job Location: Noida
Roles and Responsibilities
About the Team
Data Team plays a key role in driving the business of Client. Gathering data from each part of the company,
integrating different kinds of information into a well-designed data lake, and also analysing data and providing
management dashboards and KPI metrics are among the daily tasks of the team. To achieve this mission, not
only cooperation to other team members including data scientists, DevOps engineers, Infrastructure experts is
a must, but also collaboration to other teams in is inevitable and it’s the exact point that made the team as
dynamic and active as possible.
About the Role
As a Data Engineer, you will develop and maintain our Data platform and improve existing architecture for
incoming projects. Your passion is to work with the latest and greatest technologies that make working with
large amounts of data easy. You’re pro-active in keeping yourself up to date and are always searching for new
ways to discover new technologies. You also enjoy laying the architectural foundations for the things you’re
working on. You combine both thinking of the future and a hands-on, right now attitude. You will work in a team
with highly skilled people and enjoy a creative atmosphere where trying things out is encouraged.
Responsibilities
- Design and deliver scalable big data systems supporting both streaming and batch modes
- Implement and maintain the data-lake ecosystem to unify all diverse data sources.
- Build reliable data pipelines and ETLs to deliver data from diverse data sources.
- Work closely with stakeholders, including engineering, product, and analytics teams, to fulfill their requirements.
- Explore and assess novel technologies to address issues in the big data stack.
Qualifications
- BS/MS or more in computer engineering/science or related experience
- 2+ years of industry experience in software development using Python, Java/Scala and SQL
- Understanding of distributed systems related to data processing and storage
- Knowledge of Cloud Technologies like GCP/S3, Glue, BigQuery, Athena
- Experience using data ETLs including Airflow or Nifi
- Familiar to Docker deployment, Kubernetes and CI/CD automation based on Gitflow
Preferred Qualifications
- Experience in Programming Data Pipelines via Python/Java and work with open source data pipelines
- tools like Airflow/Nifi/Spark/Glue
- Experience in working either AWS/Google Cloud Ecosystems
- Experience in data pipelines including Logstash or Filebeat or FluentD.
- Experience with stream processing such as Flink or Spark.
- Experience in NoSQL databases like Cassandra/Hbase/MongoDB and Columnstore DB and Row
- Store Databases like Sql Server etc is plus.
Submit CV To All Data Science Job Consultants Across India For Free
