AI engineer-Data Databricks Pltform

Databricks Platform EngineerPosition Summary

We are seeking a highly skilled Databricks Platform Engineer to design, build, configure, integrate, and operationalize enterprise Data AI solutions on the Databricks platform. This role will be responsible for establishing scalable, secure, and reliable data and AI capabilities, enabling data engineers, data scientists, AI engineers, and business teams to accelerate innovation and deliver business value.

The ideal candidate will possess deep expertise in Databricks, cloud platforms, data engineering, MLOps, platform automation, and enterprise integration patterns.

Key ResponsibilitiesPlatform Engineering Administration

• Design, build, configure, and maintain Databricks workspaces across development, testing, and production environments.
• Implement platform standards, reusable frameworks, templates, and best practices.
• Manage Unity Catalog, clusters, SQL Warehouses, compute policies, workspace configurations, and access controls.
• Automate platform deployment and configuration using Infrastructure as Code (Terraform, CI/CD pipelines).

Data Engineering Integration

• Build and integrate scalable data pipelines using Databricks Lakehouse architecture.
• Design ingestion frameworks for batch, streaming, API, database, and file-based integrations.
• Implement Delta Lake, Structured Streaming, and medallion architecture patterns.
• Integrate Databricks with enterprise data platforms such as Snowflake, SAP, Oracle, SQL Server, Azure Data Factory, Kafka, and cloud storage services.

AI Machine Learning Enablement

• Enable ML and Generative AI workloads on Databricks.
• Implement MLflow, model lifecycle management, feature stores, and model serving capabilities.
• Support AI engineers and data scientists with scalable development environments.
• Integrate Databricks with LLMs, vector databases, AI gateways, and enterprise AI platforms.

DevOps, DataOps MLOps

• Establish CI/CD pipelines for data and AI workloads.
• Implement automated testing, deployment, monitoring, and rollback mechanisms.
• Create reusable deployment frameworks and engineering accelerators.
• Support release management and environment promotion processes.

Security, Governance Compliance

• Implement enterprise security controls, RBAC, data masking, encryption, and audit logging.
• Configure and manage Unity Catalog governance policies.
• Ensure compliance with enterprise security, privacy, and regulatory requirements.
• Partner with cybersecurity teams to implement platform hardening and vulnerability remediation.