Data Engineer
Al Nahiya Group
- Abu Dhabi
- Permanent
- Full-time
- Data Engineers are tasked with transforming data into a format that can be easily analyzed and utilized.
- The Data Engineer works closely other stakeholders to ensure the quality of data and the systems implemented is upheld to the highest standards possible
- Support in the development of use cases and build the data models and data flows required to support analytical model development
- Build and monitor the integrations of the company including ETL / ELT
- Build and deploy inference pipelines to productionize the models built and refactor code for new future algorithms as a reusable library
- Set up of CI/CD (continuous integration and continuous deployment) pipelines required for the delivery of the company’s data services (analytics, data mart, etc.) on their own data platform
- Set up model monitoring capabilities and model retraining workflow to ensure continuous improvement of data models
- Work with the data architect and external vendors in the implementation of any technology infrastructure or tools
- Work closely with Data Architect to produce conceptual and logical data architecture design across the platform
- Build, test and maintain data connectors and data pipelines required for data acquisition in the platform
- Set up data management tools to ensure data is handled appropriately on the platform
- Perform the relevant tests required to ensure data quality and perform holistic analysis of the datasets defined for the company
- Maintain the data quality of dashboards and reports consistently with the relevant internal and external stakeholders (e.g., Data Director, etc.)
- Work with data owners, stewards and data governance and compliance team to understand the business rules and to deploy them into the ETL / ELTs as well as integrate them with the existing dashboards
- Test the outcome of ETL integration, and initiate reconciliation between ETL outputs and ingestions inputs
- Build the test automation rules relating to data ingestion, if applicable
- Provide advice on data topics to ecosystem stakeholders whenever required
- Minimum of 5 years of experience in a Data Engineer role
- Preferred degree in Computer Science, Statistics, Informatics, Information Systems or another quantitative field
- Experience in Big Data platforms such as AWS, Azure, Spark, CouchDB, Hive, Pig, etc.
- Experience in integrating data into analytical platforms, ingestion technologies, data profiling, source-target mappings, ETL development, SQL optimization, testing and implementation
- Experience with building processes supporting data transformation, data structures, metadata, dependency, and workload management
- Experience supporting and working with cross-functional teams in a dynamic environment
- Strong analytic skills related to working with unstructured datasets