Senior Manager - AI Safety and Evaluation Engineering

Embark on a journey where your unique contributions are celebrated, and your professional growth is embraced. At ADCB, we nurture a diverse, inclusive community where every voice is valued. About the business area GBS is a group of highly skilled and talented professionals who form an essential part of ADCB's continued journey of success. With a proud history of commitment, innovation and delivery, GBS constantly strives for excellence whilst ensuring the highest standards of quality and risk awareness. Each and every member of the GBS family plays an integral role in driving ADCB's strategy, growth and digital evolution by working closely with our valued business partners to achieve exceptional customer experience through our outstanding service and support. We are actively seeking an ambitious professional to join our team at ADCB to work alongside passionate colleagues who share your ambition to redefine excellence in UAE banking In this role, your key responsibilities include : To design, implement, and scale out technical measures to ensure the reliability, safety, and security of the bank’s AI solutions, including rigorous evaluation frameworks for both predictive ML and Generative AI and standardized monitoring protocols, while serving as the technical liaison to second ‑ line risk functions to ensure AI deployments meet the bank’s risk and compliance standards, Awareness of AI Governance, AI Ethics, and Responsible AI principles is required, with a primary focus on technical practitioner responsibilities. Methodology Design: Define standardized evaluation rubrics and scoring systems for AI system performance, reliability, safety and safety. Tailor for different GenAI system archetypes (e.g., Conversational AI, RAG and variants, agents and agentic workflows, AI-enabled data access) Test Cases: Design comprehensive test cases covering deterministic logic, probabilistic outputs, and edge-case scenarios. This should include both benign as well as adversarial scenarios Explainability & Fairness: Implement XAI techniques (e.g., Shapley Values, Integrated Gradients) and unfair bias detection metrics to ensure model transparency and regulatory compliance – e.g., in lending and fraud Embedding into MLOps platform: work with MLOps platform product and engineering leaders to embed above as automated capabilities Integrated Observability Architecture: Partner with MLOps/ LLMOps teams to design and embed telemetry pipelines that capture inputs, outputs, and embeddings, ensuring real-time visibility into model health while maintaining strict PII and data privacy standards. Standardized Metric Frameworks: Establish "Golden Signals" (e.g., latency, drift, hallucination rates, and semantic similarity) and design automated "circuit-breakers" to intercept or reroute model traffic when performance or safety thresholds are breached. Act as a central consultant for individual AI project teams, providing training on evaluation best practices and safety protocols. Standardization: Establish and maintain a unified "Safety Stack" (tools for monitoring, testing, and explainability) and underlying methodology to ensure consistency across the bank’s AI portfolio The ideal candidate should have the following experience: At least: 7 years of experience in ML Engineering, Data Science, or AI Evals/ Testing/ Safety Ideally with experience in financial sector Bachelor’s degree in information technology, Computer Science, Engineering or a related discipline Core Skills: Expert Python proficiency; deep knowledge of statistical testing, XAI libraries, and LLM evaluation frameworks (e.g., Ragas, Giskard, Arize). Systems Design: Proven ability to design observability and monitoring systems at scale Leadership: Experience leading cross-functional initiatives or acting as a technical mentor/consultant. What we offer: Comprehensive Benefits Package: This includes market-leading medical insurance, group life and personal accident insurance, paid leave and leave airfare, employee preferential rates on loans and finance facilities, staff discounts and offers, and children education assistance (for certain job levels). Flexible and Remote Working Options : We understand the importance of work-life balance and offer flexible working arrangements, subject to eligibility and job requirements. Learning and Development Opportunities: We value and facilitate continuous learning and personal development, through a variety of exciting learning opportunities, such as structured instructor-led courses, a comprehensive e-Learning catalog, on-the-job training and professional development programs. At ADCB, we are dedicated to creating a respectful, caring and disciplined work environment that aligns with your career ambitions.

At least 7 years of experience in ML Engineering, Data Science, or AI Evals/ Testing/ Safety; ideally with experience in the financial sector. Bachelor’s degree in information technology, Computer Science, Engineering or a related discipline. Core Skills: Expert Python proficiency; deep knowledge of statistical testing, XAI libraries, and LLM evaluation frameworks (e.g., Ragas, Giskard, Arize). Systems Design: Proven ability to design observability and monitoring systems at scale. Leadership: Experience leading cross-functional initiatives or acting as a technical mentor/consultant.

Design, implement, and scale technical measures to ensure reliability, safety, and security of the bank’s AI solutions, including rigorous evaluation frameworks for predictive ML and Generative AI and standardized monitoring protocols; serve as the technical liaison to second-line risk functions to ensure AI deployments meet risk and compliance standards. Define standardized evaluation rubrics and scoring systems for AI system performance and safety; tailor for different GenAI archetypes. Design comprehensive test cases covering deterministic logic, probabilistic outputs, and edge-case scenarios, including adversarial scenarios. Implement explainability and fairness techniques (XAI) and bias detection metrics to ensure model transparency and regulatory compliance. Embed evaluation into the MLOps platform; collaborate with MLOps and LLMOps teams to design telemetry pipelines for real-time model health visibility while preserving PII and data privacy. Establish standardized metric frameworks (Golden Signals) and automated circuit-breakers to intercept/ reroute model traffic when thresholds are breached. Act as a central consultant for AI projects, providing training on evaluation best practices and safety. Establish and maintain a unified Safety Stack and methodology to ensure consistency across the bank’s AI portfolio.