Databricks Data & AI Analytics Architect

Role Description:

We are seeking an experienced Data & AI Analytics Architect / Lead Engineer to modernize enterprise analytics platforms and drive adoption of AI-powered analytical solutions on Databricks. The role focuses on building scalable data ingestion frameworks, metadata-driven architectures, conversational analytic capabilities, and governed data products that improve business decision-making and reduce time to insight.

The organization operates multiple reporting and analytics solutions across Trading, Risk, Finance, and Operations using a mix of legacy data warehouse technologies and custom data pipelines.

Current challenges include:

· Fragmented ingestion and transformation patterns resulting in higher maintenance effort.

· Ongoing migration of analytical workloads from legacy platforms to Databricks.

· Increasing demand for self-service analytics, conversational data access, and AI-assisted decision support.

· Need for improved data quality, lineage, governance, and auditability.

· Requirement to accelerate insight generation and improve analytical scalability, reliability, and performance.

Skills:

· Strong expertise in Databricks, Spark, Delta Lake, Unity Catalog, SQL Warehouses, Databricks Workflows, Databricks Genie, and Genie One.

· Experience designing enterprise-scale data ingestion, ETL/ELT, metadata-driven, and watermark-based processing frameworks.

· Experience modernizing and migrating workloads from legacy data warehouses to Databricks.

· Strong SQL skills including query optimization, semantic modeling, and metadata-driven analytics.

· Experience building AI/LLM-powered analytics solutions, conversational interfaces, and agent-based architectures.

· Experience developing analytical assistants/copilots leveraging business metadata, governance rules, and semantic context.

· Ability to design conversational data exploration with drill-down analysis, context retention, and root-cause investigation.

· Experience with automated insight generation including KPI monitoring, trend analysis, anomaly detection, correlation analysis, and executive summaries.

· Good understanding of cloud data and analytics architectures covering ingestion, transformation, modeling, and serving layers.

· Knowledge of visualization and dashboarding concepts enabling business-friendly analytical experiences.

· Ability to work effectively across business and technology teams to improve data availability, reliability, and performance.

· Familiarity with Trading, Risk, Finance, Operations, or Supply Chain domains is highly desirable.