Data Quality & AI Operations Manager

About Extend: Extend is a Saudi cultural intelligence and advertising company, operating since 2012, serving the Kingdom's leading brands and institutions across experiences, media, and publishing. We are building Extend CIP, the Cultural Intelligence Platform: one place to plan, manage, and measure all media activity, earned and paid, spanning media and influencer buying and a brand monitoring suite, with cultural intelligence and AI at its core. Role Summary The role: Own content and data operations for the Extend platform. Build a leaner, more accurate profiling process and use AI fine tuning to progressively replace manual annotation with automation. You own the evaluation and data feedback loop and lead the content operations team. Success is measured by golden-set accuracy and profile automation rate. What you'll do • Build and maintain golden sets and eval suites; define acceptance criteria for each AI Admin iteration • Own the data side of AI Admin iterations: curate training and evaluation data, run benchmarks, analyze failure modes, and partner with Shanghai engineering on fine tuning and deployment • Design the human-in-the-loop (HITL) mechanism: define the boundary between human review and auto-approval, and feed human corrections back into prompts, eval sets and training data • Author and maintain Arabic/English labeling guidelines; adjudicate edge cases requiring cultural and linguistic judgment • Lead the content operations team with end-to-end ownership of annotation operations: rater training, workflows, throughput and quality SLAs • Establish and track a quality metrics framework: accuracy, automation rate, inter-annotator agreement, cost and turnaround per profile What we're looking for • Fluent in Arabic, with strong knowledge of the Arab world's media, brands, publishers and cultural context • Hands-on experience with AI agent / LLM systems: eval design, golden sets and model feedback loops; conversant in finetuning concepts (SFT, RLHF, prompt iteration) without needing to train models yourself • 2+ years in AI data operations or applied AI quality, from a data labeling company (Scale AI, Surge, Appen, Sama, Invisible, Labelbox), an in-house annotation team at an AI company, or an AI product team • Comfortable collaborating across Riyadh and Shanghai time zones Nice to have: • LLM-assisted labeling automation, taxonomy design, social or media data experience. Show more Show less

2+ years in AI data operations or applied AI quality, from a data labeling company, in-house annotation team, or AI product team. Fluent in Arabic, with strong knowledge of the Arab world's media, brands, publishers and cultural context. Hands-on experience with AI agent / LLM systems: eval design, golden sets and model feedback loops; conversant in finetuning concepts (SFT, RLHF, prompt iteration) without needing to train models yourself. Comfortable collaborating across Riyadh and Shanghai time zones.

Build and maintain golden sets and eval suites; define acceptance criteria for each AI Admin iteration. Own the data side of AI Admin iterations: curate training and evaluation data, run benchmarks, analyze failure modes, and partner with Shanghai engineering on fine tuning and deployment. Design the human-in-the-loop (HITL) mechanism: define the boundary between human review and auto-approval, and feed human corrections back into prompts, eval sets and training data. Author and maintain Arabic/English labeling guidelines; adjudicate edge cases requiring cultural and linguistic judgment. Lead the content operations team with end-to-end ownership of annotation operations: rater training, workflows, throughput and quality SLAs. Establish and track a quality metrics framework: accuracy, automation rate, inter-annotator agreement, cost and turnaround per profile.