Description

The Data Engineer is a software engineer specialised in data solutions, including technologies to use and scalability of different solutions. Typically reports to the Chapter Lead.

#_VOIS

Job Responsibility

• Designs, constructs, installs, tests and maintains highly scalable data management systems, to business requirements and industry practices through the application of data modelling, data warehousing and data manipulation techniques and standards;
• Builds and deploys individual datasets for further modeling based on commercial and/or analytical requirements by extracting, loading and transforming data from different sources;
• Supports integration of new data sources (e.g. batch and streaming) into shared data environment (e.g. EDW, Big Data Platform, etc.);
• Builds high-performance algorithms, prototypes, predictive models and proof of concepts, translating business requirements into concrete data engineering tasks;
• Researches opportunities for data acquisition and new uses for existing data;
• Develops data set processes for data modelling, mining and production;
• Integrates new data management technologies and software engineering tools into existing structures;
• Creates custom software components and analytics applications;
• Employs a variety of languages and tools (e.g. scripting languages) to marry systems together;
• Installs and updates disaster recovery procedures;
• Recommends ways to improve data reliability, efficiency and quality;
• Collaborates with data architects, modellers and IT team members on project goals;
• Designs, develops and maintains programmes (e.g. written in Python, Scala, Java);
• Designs, creates and maintains reports and dashboards.

Skills