Databricks Architect - ETL/Apache Spark

Role Overview :

We are seeking a highly experienced Databricks Architect to lead the design, implementation, and optimization of enterprise-scale data solutions.

This role requires expertise in cloud-based data engineering, advanced analytics, and scalable architecture design.

The successful candidate will play a pivotal role in enabling data-driven decision-making across the organization.

Key Responsibilities :

• Architect and implement scalable data solutions leveraging Databricks, Apache Spark, and Delta Lake.

• Collaborate with cross-functional teams to define data strategies, pipelines, and integration frameworks.

• Optimize ETL workflows, data lakes, and machine learning environments for performance and reliability.

• Establish and enforce best practices for data governance, security, and compliance.

• Lead migration initiatives from legacy data systems to Databricks and cloud-native platforms.

• Provide technical leadership, mentoring, and architectural guidance to engineering teams.

• Stay current with emerging technologies in big data, AI/ML, and cloud ecosystems.

• Expertise with Databricks Administration performance tuning, optimization, configuration, setup, etc.

• Ability to communicate with stakeholders from Infra, legal & Info Sec.

Qualifications:

• Bachelors or Masters degree in Computer Science, Data Engineering, or related field.

• Proven experience with Databricks, Apache Spark, and cloud platforms (Azure, AWS, or GCP).

• Strong knowledge of data warehousing, ETL, and real-time streaming solutions.

• Hands-on experience with Delta Lake, MLflow, and advanced analytics frameworks.

• Excellent problem-solving, communication, and leadership skills.

Preferred Skills:

• Experience with CI/CD pipelines and DevOps practices for data engineering.

• Knowledge of containerization (Docker, Kubernetes) and microservices architecture.

• Familiarity with BI tools (Power BI, Tableau) for data visualization.