Job Description

DeepLight AI is a specialist AI and data consultancy dedicated to transforming the regional corporate landscape through bespoke, high-impact intelligent systems. We combine deep expertise in data engineering, cloud architecture, AI/ML platforms, and systems integration with a practical understanding of complex enterprise operations. With deep roots across the financial services and banking sectors, we help major institutions bridge the gap between complex data infrastructure and actionable AI strategy. We don't build temporary fixes—we engineer secure, scalable, and resilient data foundations designed to power production AI at scale. The Opportunity We are seeking a Data Engineer with strong AWS and banking sector experience to join our core engineering squad in the UAE. In this role, you will design, construct, and optimize scalable Lakehouse data solutions that power enterprise analytics and advanced machine learning models. Operating at the intersection of big data, cloud architecture, and MLOps, you will unify data warehousing and data lake capabilities to process complex, high-volume financial data while maintaining strict banking security, ACID compliance, and data governance standards. What You’ll Do.... - Build Next-Gen Lakehouse Architectures: Design, build, and optimize scalable data solutions using Lakehouse architectures (Databricks, Delta Lake) to unify data warehousing and data lake capabilities on AWS. - Develop High-Throughput Pipelines: Build robust batch and real-time ETL/ELT pipelines using PySpark, Apache Kafka, and SQL to ingest and transform complex financial datasets from core banking, market data, and transactional systems. - Optimize for Scale & FinOps: Maximize query performance and storage efficiency across AWS services (S3, Redshift, EMR, Athena, Glue) using advanced partitioning, indexing, and parallel processing to drive down latency and cloud compute costs. - Support Machine Learning & MLOps: Partner with Data Scientists and MLOps Engineers to build automated feature engineering pipelines, track experiments (MLflow), containerize workflows (Docker, EKS), and deploy models to production (AWS SageMaker). - Enforce Banking-Grade Governance: Implement end-to-end data security, encryption, access controls, and audit trails to ensure 100% compliance with Central Bank data sovereignty and privacy mandates. - Automate Everything: Drive software engineering best practices by managing Infrastructure as Code (Terraform), authoring CI/CD pipelines (GitHub Actions/CodePipeline), and writing clean, well-tested Python code. Benefits The benefits you'll enjoy as part of this role include: - Competitive salary - Comprehensive 'Gold Level' personal health insurance - Individual Visa Sponsorship - Professional development and certification support - Subscription reimbursement relating to your role - Opportunity to work on cutting-edge AI projects - Monthly Employee Incentive program - Career advancement opportunities in a rapidly growing AI company This position offers a unique opportunity to shape the future of AI implementation while working with a talented team of professionals at the forefront of technological innovation. The successful candidate will play a crucial role in driving our company's success in delivering transformative AI solutions to our clients. At DeepLight AI, we recognise that diversity drives innovation. We are committed to fostering an inclusive environment where individuals with different thinking styles can thrive and contribute their unique strengths to our specialised AI and data solutions. Our goal is to ensure our application and interview process is accessible, predictable, and fair for all candidates. If you require any specific adjustments to the application process, or if you require any reasonable adjustments should you be successful in being processed to the interview stage, please do let us know. This information will be kept strictly confidential and will not impact hiring decisions.

Requirements

What We’re Looking For

- Hands-on data engineering experience directly within banking, financial services, or fintech environments, with a strong understanding of transactional datasets and regulatory compliance.

- Deep technical command of core AWS data services (S3, Glue, EMR, Redshift) alongside distributed processing engines (Apache Spark / PySpark, Apache Kafka).

- Proven track record building and managing Lakehouse architectures (Databricks, Delta Lake) with expert knowledge of data modeling, schema evolution, and ACID principles.

- High proficiency in Python and SQL (Scala is a plus), alongside practical experience with Docker, Kubernetes (EKS), Terraform, and Git workflows.

- Excellent analytical problem-solving skills with the ability to articulate complex technical concepts clearly to cross-functional teams and client stakeholders.

What Gives You an Edge

- AWS Certified Data Engineer, AWS Solutions Architect, or Databricks Certified Data Engineer.

- Hands-on experience with streaming analytics frameworks (Apache Flink, Spark Streaming) for real-time fraud detection or trade processing pipelines.

- Production experience with modern workflow schedulers like Apache Airflow or Prefect.

As an AI consultancy, our greatest asset is the expertise of our people.

While technical mastery is the foundation of what we do, the ability to bridge the gap between complex data science and actionable business value is what defines your success with Deeplight.

We're looking for individuals who are not only world-class in their fields of specialism, but also compelling communicators and persuasive advocates for their own skills.

You will be the face of our firm, tasked with building trust, articulating the "why" behind your technical decisions, and effectively "selling" your vision to high-level stakeholders.

If you thrive on the challenge of presenting cutting-edge solutions as much as you do on building them, you will fit right in.

Responsibilities

- Build Next-Gen Lakehouse Architectures: Design, build, and optimize scalable data solutions using Lakehouse architectures (Databricks, Delta Lake) to unify data warehousing and data lake capabilities on AWS.

- Develop High-Throughput Pipelines: Build robust batch and real-time ETL/ELT pipelines using PySpark, Apache Kafka, and SQL to ingest and transform complex financial datasets from core banking, market data, and transactional systems.

- Optimize for Scale & FinOps: Maximize query performance and storage efficiency across AWS services (S3, Redshift, EMR, Athena, Glue) using advanced partitioning, indexing, and parallel processing to drive down latency and cloud compute costs.

- Support Machine Learning & MLOps: Partner with Data Scientists and MLOps Engineers to build automated feature engineering pipelines, track experiments (MLflow), containerize workflows (Docker, EKS), and deploy models to production (AWS SageMaker).

- Enforce Banking-Grade Governance: Implement end-to-end data security, encryption, access controls, and audit trails to ensure 100% compliance with Central Bank data sovereignty and privacy mandates.

- Automate Everything: Drive software engineering best practices by managing Infrastructure as Code (Terraform), authoring CI/CD pipelines (GitHub Actions/CodePipeline), and writing clean, well-tested Python code.

Data Engineer

Job Description

Requirements

Responsibilities

Skills Required

Interested in this role?

Similar jobs in the UAE