Modern Data Platforms - Microsoft Databricks Manager

Modern Data Platforms - Microsoft Databricks Manager

Position Overview

The MDP Databricks Architect / Consultant / Developer designs, builds and operates modern analytics and data platforms using the Databricks Lakehouse Platform. The role enables organisations to deliver scalable, high-performance, governed data solutions across analytics, business intelligence (BI), data engineering, data science, and artificial intelligence/machine learning (AI/ML) workloads.

Depending on experience and seniority, the role may be hands-on in solution delivery (Developer / Consultant) or lead platform architecture, standards and data strategy (Architect). The role works closely with business stakeholders, data teams, cloud architects and security teams to ensure the platform is secure, cost-effective and enterprise-ready.

Key Responsibilities

• Design and implement modern data platforms using the Databricks Lakehouse Platform.

• Build ingestion and transformation pipelines using Databricks Workflows, Apache Spark (PySpark / Spark SQL) and Delta Lake, supporting batch, streaming and near real-time processing.

• Architect focus: Define and apply platform standards for data ingestion, modelling, data quality and governance; ensure alignment to cloud, security, identity and cost management requirements.

• Develop and optimise Databricks (Apache Spark) pipelines and Delta Lake models to enable governed data consumption (for example, Power BI/Tableau) and support analytics, data science, and machine learning (ML) use cases.

• Implement monitoring, performance tuning and cluster optimisation; support CI/CD for notebooks, jobs and data pipelines and contribute to operational runbooks.

• Collaborate with analysts, data scientists and application teams; advise stakeholders on Databricks capabilities and design trade-offs, and mentor junior team members (where applicable).

Skills, Experience and Competencies

Technical Skills

• Strong expertise in Databricks Lakehouse Platform.

• Deep experience with Apache Spark, PySpark, and Spark SQL.

• Knowledge of Delta Lake, data modelling, and performance tuning.

• Experience with streaming data (Structured Streaming, Kafka).

• Understanding of data security, governance, and cost optimisation.

Experience (Role Level Determination)

MDP Databricks Architect

• 10–15+ years of data and analytics experience.

• 7–10+ years designing enterprise scale data platforms.

• Proven experience defining lakehouse architectures and data strategies.

MDP Databricks Consultant

• 6–9 years of data and analytics experience.

• 4–6 years delivering modern data platforms using Databricks.

• Strong delivery, design, and stakeholder engagement skills.

MDP Databricks Developer

• 4–6 years of overall data / analytics experience.

• 2–4 years of hands‑on Databricks and Spark experience.

• Strong focus on development and implementation.

Competencies

• Early alignment and shaping clarifying use cases, data domains, success measures and non-functional requirements, then translating them into a practical Databricks delivery plan.

• Governance and controlled change applying standards and change control (with impact assessment) across notebooks, jobs, pipelines, data models and environments.

• Trusted data, quality by design building Delta Lake pipelines and models with validation, testing, lineage and reconciliation to support analytics and artificial intelligence/machine learning (AI/ML).

• Standardise use patterns, templates and accelerators (including CI/CD and infrastructure as code where relevant) to improve speed and consistency.

• Transparent delivery and operational discipline communicating progress and risks clearly, implement monitoring and observability, and optimising cost and performance for stable operations.

Qualifications and Certifications

• Databricks Certified Data Engineer (Associate / Professional)

• Databricks Certified Data Architect (role dependent).

• Cloud platform certifications (Microsoft Azure / Amazon Web Services desirable).

What Success Looks Like

• Databricks is established as a scalable and high-performance modern data platform.

• Data pipelines are reliable, efficient and easy to maintain, and analytics and AI/ML workloads run on governed datasets.

• Data is secure, trusted, and widely adopted across the organisation.

• Platform costs and performance are optimised, and Databricks is positioned as a strategic lakehouse foundation.

Additional Requirements & Eligibility

• Right to work in the Philippines: must hold (and maintain) the appropriate visa/work authorisation for the role and assignment length.

• Hold a current passport suitable for business travel (including the ability to obtain visas as required) and be willing to travel where needed.

• Satisfy company's background screening and any client-specific screening requirements (which may include identity, criminal history, qualification, professional membership and other checks).

• Provide required onboarding documentation as requested by People & Inclusion.

Work Setup:

Shift: Dayshift

Setup: Hybrid

Location: Makati City

By Applying, you give consent to collect, store, and/or process personal and/or sensitive information for the purpose of recruitment and employment may it be internal to Cobden & Carter International and/or to its clients.