Job Summary

• The Enterprise Data program is developing a fully integrated and intuitive data and analytics platform.
• This platform will enable Lilly team members to quickly view, access, and analyze data sets in statistical environments, advanced analytics environments, and BI/visualization tools. +
• Contributors can easily ingest, prepare, and analyze new data sets (cleanse, enhance, publish) for others to utilize.
• This position is responsible for providing support for ETL / ELT / File Movement of data as part of an enterprise data program.
• The key responsibilities will be to process and move data between different compute and storage services, as well as on-premises data sources at specified intervals.
• The employee will also be responsible for the creation, scheduling, orchestration and management of data pipelines.

Competency Summary

• Data engineers are responsible for ensuring the availability and quality of data needed for analysis and business transactions.
• This includes data integration, acquisition, cleansing, harmonization and transforming raw data into curated datasets for data science, data discovery, and BI/analytics.
• Responsible for developing, constructing, testing and maintaining data sets and scalable data processing systems.
• Data engineers work closest with Data Architects and Data Scientists.
• They also work with business and IT groups beyond the data sphere, understanding the enterprise infrastructure and the many source systems.
• Input is raw datasets. Output is analytics-ready, integrated/curated datasets.

Key Capabilities in this Role Family Include:

• Data Acquisition – is the process of gathering and storing data in a location and format that it can be consumed for data preparation and/or downstream business uses.

• Data Preparation – is an iterative process for exploring, integrating, cleaning, validating and transforming raw data into curated datasets

• Data Publishing – is the act of releasing data in consumable form for (re)use by others.

Key Objectives/Deliverables

• Gather user requirements and translate them into business solutions.

• Design, develop, test, improve, and maintain new and existing solutions for data integration, data prep, ingestion, cleansing, and transforming raw data into curated datasets for business consumption.

• Incorporate deep data management expertise into solutions, including permissions, recovery, security, and monitoring.

• Validate and qualify solutions.

• Identify, troubleshoot, debug, and resolve technical issues.

• Work with business and IT groups to understand source systems and enterprise infrastructure offerings.

• Work with data from multiple data sources to build integrated views that will drive decisions

• Consult with internal customers who own the data to develop information relationships that lead to actionable insights.

• Work in Agile Sprints with business facing project teams

Minimum Position Qualifications

• Bachelor s Degree and 5 years experience in the implementation of modern data ecosystems in AWS/Cloud platforms.

• Strong experience with AWS ETL/File Movement tools ( GLUE, Athena, Lambda, Kenesis and other AWS integration stack)

• Strong experience with Agile Development, Cloud formation Template, AWS CodeBuilt, AWS Code Pipeline

• Strong experience with Two or Three AWS database technologies ( Redshift, Aurora, RDS,S3 & other AWS Data Service ) covering security, policies, access management

• Strong programming Experience with Python and Spark

• Experience with Apache Airflow,Ansible & other automation stack.

• Excellent oral and written communication skills.

• A high level of intellectual curiosity, external perspective, and innovation interest

• Strong analytical, problem solving and investigative skills

• Experience in applying quality and compliance requirements

• Experience with security models and development on large data sets

Additional Preferences

• ITIL Certification

Education Requirements:

• Bachelor s Degree or relevant degree