Senior Data Platform Engineer

About the role

Requirements

• - Bachelor’s or Master’s degree in Computer Science, Information Technology, or related field.
• - 8+ years of experience in data engineering, with a focus on data platform architecture and development.
• - Strong proficiency in SQL and experience with relational and NoSQL databases.
• - Hands-on experience with cloud data platforms such as AWS, Azure, or Google Cloud.
• - Experience with big data technologies like Hadoop, Spark, and data warehousing solutions.
• - Knowledge of data governance, security, and compliance standards.
• - Excellent problem-solving and communication skills.
• - Proven ability to lead technical projects and mentor team members.
• - Required Skills & Technologies
• - - SAP data extraction: OData services, RFC/BAPI, SAP Data Services, or CDS views — hands-on experience pulling data from SAP FI/CO modules
• - - Python — async data pipelines, background jobs, scheduled tasks
• - - PostgreSQL — schema design, migrations (Alembic), query optimization, partitioning
• - - Azure Data Lake Storage + Synapse Analytics
• - - Microsoft Graph API — SharePoint, M365, organizational data
• - - Apache Airflow or Azure Data Factory
• - - dbt or equivalent for transformation and quality testing
• - - Redis — queue management, TTL, cache invalidation
• - - Observability — structured logging, Azure Monitor or Prometheus/Grafana
• - - Docker — containerized pipeline jobs
• - - Infrastructure-as-code (Terraform or equivalent)

Responsibilities

• Design, build, and maintain scalable data platforms to support diverse data workloads.
• Develop and optimize ETL/ELT pipelines to ensure efficient data movement and transformation.
• Collaborate with cross-functional teams to translate business requirements into technical solutions.
• Ensure data platform security, governance, and compliance with relevant regulations.
• Monitor platform performance, troubleshoot issues, and implement improvements.
• Provide technical leadership and mentorship to junior data engineers.
• Stay updated with the latest trends and technologies in data platform engineering.
• - Platform architecture: pipeline design patterns, orchestration strategy, infrastructure-as-code, monitoring stack
• - Enterprise data extraction: SAP FI/CO (budget, actuals, commitments via OData/RFC), Microsoft Graph API (SharePoint, M365), SuccessFactors OData API — you own the extraction approach and build the first pipelines yourself
• - Staging layer design: PostgreSQL staging schema, raw-to-clean transformation patterns, refresh strategies
• - Ingestion observability: logging standards, alerting on failures, monitoring dashboards for pipeline health
• - Data quality framework: defining quality checks, deciding what pauses a pipeline vs. what triggers a warning
• - Classification enforcement: working with the Governance Lead to embed automated classification tagging into every pipeline
• - Technical mentorship: setting standards and reviewing work for the Senior Data Platform Engineer

Skills

• SAP data extraction: OData services, RFC/BAPI, SAP Data Services, or CDS views
• Python — async data pipelines, background jobs, scheduled tasks
• PostgreSQL — schema design, migrations (Alembic), query optimization, partitioning
• Azure Data Lake Storage
• Synapse Analytics
• Microsoft Graph API — SharePoint, M365
• Apache Airflow
• Azure Data Factory
• dbt or equivalent for transformation and quality testing
• Redis — queue management, TTL, cache invalidation
• Observability — structured logging, Azure Monitor or Prometheus/Grafana
• Docker — containerized pipeline jobs
• Infrastructure-as-code (Terraform)