• Design, develop and deploy modern data warehouse solutions in AWS/Azure cloud platform.
• Gather and document business requirements and translate it into technical design documents.
• Create technical documentation, e.g. data flow diagrams, end user guide and technical operations guides.
• Create technical documentation, e.g. data flow diagrams, end user guide and technical operations guides.
• Monitor data load jobs and perform root cause analysis of production issues.
• Lead the efforts in building end to end streaming and batch data analytics pipelines. From data ingestion, processing, storage, analysis, machine-learning to visualization. Understand big-data principles and best practices. Deliver projects in data analytics, machine learning and AI.
• Design architectures publish reference code and establish data structure design based on business requirements. Should be pretty hands on.
• Perform code reviews, ensure code quality and encourage a culture of excellence.
• Communicate/work effectively in a team environment.

What You Know

• 5+ years of experience designing and building solutions on Big Data and Data Analytics.
• Azure / AWS / GCP cloud services experience is mandatory.
• Must have hands-on experience with Databricks.
• At least two full implementation project experience with heterogenous source system.
• Experience transforming data in various formats, including JSON, XML, CSV, AVRO, Parquet , ORC and zipped files is required.
• Experience in building Data Lake on Cloud.
• Experience in some of the following: Python, Scala, HDFS/MapReduce, Hive, Sqoop, Flume, Spark, Kafka, Apache Airflow Advanced SQL, No SQL (Cassandra / HBase) and Databricks
• Good to have Machine Learning and R experience or knowledge.
• Experience programming in Java or Python or Scala, etc.
• Expertise in at least two of these technologies: Relational Databases, Analytical Databases, NoSQL databases.
• Experience with Apache Airflow, Cloud Scheduler.
• Familiarity with standard source repositories (GIT, BitBucket)
• Experience and solid knowledge in Agile (Scrum) Methodologies
• Spark / Databricks certification is a major advantage.

Education

• Bachelor s degree in Computer Science, Engineering or equivalent demonstrable experience.