Databricks Data Quality & Remediation Architect Lead

Databricks Data Quality & Remediation Architect / Lead Location: Remote - UAE | Abu Dhabi preferred Contract: 6 months initially, with strong likelihood of extension Rate: Competitive, depending on experience Start: ASAP Our client is a leading organisation in the UAE delivering a major enterprise data transformation programme, and they are looking for a Databricks Data Quality & Remediation Architect / Lead Developer to help design and lead the delivery of a new enterprise-scale data quality platform. This is a senior, hands-on role combining Databricks architecture, data engineering and technical leadership. What You'll Do • Design and implement a scalable Data Quality & Remediation Engine on Databricks. • Define architecture across data profiling, quality rules, exception management, root-cause analysis and remediation. • Build a metadata-driven data-quality framework. • Define Bronze, Silver and Gold processing layers. • Work with Data Governance teams and Data Stewards on Critical Data Elements and data-quality processes. • Support Microsoft Purview integration, governance, lineage and auditability. • Lead the development team and provide hands-on technical direction. • Own data-quality reporting, including quality KPIs, exceptions and remediation activity. What We're Looking For • 5+ years' experience in data engineering, Databricks or related roles. • Ideally 2+ years in a Lead or Architect position. • Strong hands-on Databricks experience. • Excellent Python and PySpark skills. • Strong knowledge of Databricks Lakehouse, Delta Lake, Unity Catalog, Workflows, Lakeflow and Databricks SQL. • Experience with Azure Databricks and services such as Data Factory, ADLS Gen2 and Azure DevOps. • Experience with data quality, governance or remediation solutions. • Strong technical leadership and stakeholder-management skills. Desirable • Microsoft Purview. • SAP S/4HANA, SAP BW or Oracle Fusion. • MLflow, Mosaic AI or Databricks Asset Bundles. • Enterprise data governance and lineage. Why Apply? You'll have the opportunity to shape the architecture of a major enterprise data platform, lead delivery and work on complex data-quality challenges at scale. The programme is expected to continue beyond the initial six-month contract, offering strong potential for long-term engagement. If you're an experienced Databricks Lead or Architect looking for a technically challenging project where you can make a real impact, we'd love to hear from you. Apply now or get in touch for a confidential discussion. Show more Show less

5+ years' experience in data engineering, Databricks or related roles; ideally 2+ years in a Lead or Architect position; strong hands-on Databricks experience; excellent Python and PySpark skills; strong knowledge of Databricks Lakehouse, Delta Lake, Unity Catalog, Workflows, Lakeflow and Databricks SQL; experience with Azure Databricks and services such as Data Factory, ADLS Gen2 and Azure DevOps; experience with data quality, governance or remediation solutions; strong technical leadership and stakeholder-management skills.

Design and implement a scalable Data Quality & Remediation Engine on Databricks; Define architecture across data profiling, quality rules, exception management, root-cause analysis and remediation; Build a metadata-driven data-quality framework; Define Bronze, Silver and Gold processing layers; Work with Data Governance teams and Data Stewards on Critical Data Elements and data-quality processes; Support Microsoft Purview integration, governance, lineage and auditability; Lead the development team and provide hands-on technical direction; Own data-quality reporting, including quality KPIs, exceptions and remediation activity.