Job Location: Hyderabad/Secunderabad, Bangalore/Bengaluru
Roles and Responsibilities
What you will be doing…
- Designing and implementing data processing pipelines, as part of diverse, high
energy teams
Working with our data scientists to take our MLAI models to production
Hands-on programming in Python, Java, Scala
Deploying data pipelines in production based on Continuous Delivery practices Recommending the right distributed storage and computing technologies to our clients from a large number of options available in the ecosystem
Ensuring that the data is available to the consumers in a reliable, trustworthy and
predictable manner
Evolving the data platform to make it more robust, scalable
Desired Candidate Profile
What we are looking for….
- 6-8 years of experience working as a data engineer
- Proficient understanding of distributed computing principles
- Proficiency HDFS, Spark, Kafka
- Experience with integration of data from multiple data sources
- Experience with NoSQL databases Redis, MongoDB, ElasticSearch,
- Knowledge of various ETL techniques and frameworks
- Knowledge of how to create and maintain optimal data pipeline architecture,
- Good understanding of Lambda Architecture, along with its advantages and drawbacks
Great to have:
Ability to solve any ongoing issues with operating the cluster
- Experience with building stream-processing systems, using solutions such as Flink or Spark-Streaming
- Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
- Experience with Cloudera/MapR/Hortonworks
Perks and Benefits
Submit CV To All Data Science Job Consultants Across Bharat For Free
