Job Location: Gurgaon/Gurugram
Position Summary
The Data Engineer III plays a Lead technical role in an AgileScrum based Data Science team working in a development environment designing and maintaining data stores that will be used for analytical processing andor data warehouse. The Data Engineer III is a subject matter expert in Hadoop, Vertica and related technologies. This individual is responsible for maintaining data integrity, data modeling, performance tuning, and advanced administration of Big Data, and Data-mart systems.
Key Performance Competencies
– AdaptableFlexible
– Technical Knowledge
– Communication Skills
– Problem Solving
– Initiative
Essential Duties and Responsibilities
– Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
– Implementing ETL process using Talend, and Python scripts
– Monitoring performance and advising any necessary infrastructure changes
– Defining data retention policies
– Data modeling, database optimization, understanding and implementation of schemas, and the ability to interpret and write complex SQL queries
– Responsible for the administration, performance and security of existing databases and the analysis, design, and creation of new databases
– Proactively monitor and tune systems, databases and BI toolssoftware for optimum performance and capacity constraints
– Analyze user requirements and design functional specifications for BI front-end applications
– Produce ETL design guidelines to ensure a manageable ETL infrastructure
– Promote and educate BI usage and benefits across the organization
– Evaluate and make recommendations for databasedata warehouse components, including hardware, database management systems, ETL software, BI products, services, and standards in support of procurement and development efforts.
– Mentor junior database engineers
Knowledge Skills Required :
– Working experience with integration of data from multiple data sources
– Working experience with NoSQL databases, such as HBase, Cassandra, MongoDB
– Working experience with various ETL techniques and frameworks
– Experience with various messaging systems, such as Kafka or RabbitMQ
– Experience with Big Data ML toolkits, such as Mahout, SparkML, or H2O
– Good understanding of Lambda Architecture, along with its advantages and drawbacks
– Proficient understanding of distributed computing principles
– Management of Hadoop cluster, with all included services
– Ability to identifysolve any ongoing issues with Hadoop cluster
– Proficiency with Hadoop v2, MapReduce, HDFS
– Experience with Spark
– Experience with building stream-processing systems, using solutions such as Storm or Spark-Streaming
– Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
– Strong attention to detail
Education, Training and Experience
– Bachelors degree in Computer Science, IT, or related field, or demonstrated equivalent experience
– 6+ years of database design in a development andor production environment
– 4+ years ETL design and implementations
– 2+ years in data cleansing
– One or more of the following certifications highly desirable: MCITP: Database Administrator certification, MCSE: Data Platforms, MCSE: BI
Compensation
– To be decided on a case-by-case basis, pegged with the best in the industry.
Submit CV To All Data Science Job Consultants Across India For Free
