Databricks Data Quality & Remediation Architect Lead

Databricks Data Quality & Remediation Architect / Lead Location: Hybrid - UAE | Abu Dhabi preferred Contract: 6 months initially, with a strong likelihood of extension Rate: Competitive, depending on experience Start: ASAP Our client is a leading organisation in the UAE delivering a major enterprise data transformation programme, and they are looking for a Databricks Data Quality & Remediation Architect / Lead Developer to help design and lead the delivery of a new enterprise-scale data quality platform. This is a senior, hands-on role combining Databricks architecture, data engineering and technical leadership. What You'll Do • Design and implement a scalable Data Quality & Remediation Engine on Databricks. • Define architecture across data profiling, quality rules, exception management, root-cause analysis and remediation. • Build a metadata-driven data-quality framework. • Define Bronze, Silver and Gold processing layers. • Work with Data Governance teams and Data Stewards on Critical Data Elements and data-quality processes. • Support Microsoft Purview integration, governance, lineage and auditability. • Lead the development team and provide hands-on technical direction. • Own data-quality reporting, including quality KPIs, exceptions and remediation activity. What We're Looking For • 5+ years' experience in data engineering, Databricks or related roles. • Ideally 2+ years in a Lead or Architect position. • Strong hands-on Databricks experience. • Excellent Python and PySpark skills. • Strong knowledge of Databricks Lakehouse, Delta Lake, Unity Catalog, Workflows, Lakeflow and Databricks SQL. • Experience with Azure Databricks and services such as Data Factory, ADLS Gen2 and Azure DevOps. • Experience with data quality, governance or remediation solutions. • Strong technical leadership and stakeholder-management skills. Desirable • Microsoft Purview. • SAP S/4HANA, SAP BW or Oracle Fusion. • MLflow, Mosaic AI or Databricks Asset Bundles. • Enterprise data governance and lineage. Why Apply? You'll have the opportunity to shape the architecture of a major enterprise data platform, lead delivery and work on complex data-quality challenges at scale. The programme is expected to continue beyond the initial six-month contract, offering strong potential for long-term engagement. If you're an experienced Databricks Lead or Architect looking for a technically challenging project where you can make a real impact, we'd love to hear from you. Apply now or get in touch for a confidential discussion. Show more Show less

5+ years' experience in data engineering, Databricks or related roles. Ideally 2+ years in a Lead or Architect position. Strong hands-on Databricks experience. Excellent Python and PySpark skills. Strong knowledge of Databricks Lakehouse, Delta Lake, Unity Catalog, Workflows, Lakeflow and Databricks SQL. Experience with Azure Databricks and services such as Data Factory, ADLS Gen2 and Azure DevOps. Experience with data quality, governance or remediation solutions. Strong technical leadership and stakeholder-management skills.

Design and implement a scalable Data Quality & Remediation Engine on Databricks. Define architecture across data profiling, quality rules, exception management, root-cause analysis and remediation. Build a metadata-driven data-quality framework. Define Bronze, Silver and Gold processing layers. Work with Data Governance teams and Data Stewards on Critical Data Elements and data-quality processes. Support Microsoft Purview integration, governance, lineage and auditability. Lead the development team and provide hands-on technical direction. Own data-quality reporting, including quality KPIs, exceptions and remediation activity.