Solution Architect - Databricks/Data Science

Position Overview :

We are seeking an experienced Databricks Data Science Solution Architect to provide technical leadership and hands-on development support to an established Data Science team. The ideal candidate will have strong Python development expertise and proven experience designing, developing, deploying, and operationalizing production-ready Data Science and Machine Learning solutions on Databricks.

Key Responsibilities :

• Design and architect scalable Data Science, Machine Learning, AI, and advanced analytics solutions using Databricks.

• Develop and review production-quality Python code for Machine Learning models and Data Science applications.

• Build and deploy solutions using Databricks Apps, Jobs, Workflows, Pipelines, MLflow, Unity Catalog, and Model Serving Endpoints.

• Convert experimental notebooks and Machine Learning models into scalable, maintainable production applications.

• Implement MLOps/ModelOps practices, including model versioning, experiment tracking, automated testing, CI/CD, deployment, monitoring, and retraining.

• Optimize application performance, scalability, reliability, and cost.

• Mentor Data Scientists and establish coding standards, reusable frameworks, and development best practices.

• Collaborate with Data Engineering, Platform Engineering, ModelOps, and business teams.

• Troubleshoot production issues and ensure solutions meet enterprise security and governance standards.

Required Skills and Experience :

• 7+ years of experience in Data Science.

• 5+ years of experience with Databricks products.

• Strong hands-on Python development experience.

• Proven experience building and deploying Data Science and Machine Learning solutions on Databricks.

• Hands-on experience with Databricks Apps, Jobs/Workflows, Pipelines, MLflow, Unity Catalog, and Model Serving Endpoints.

• Strong understanding of Machine Learning workflows, MLOps/ModelOps, and production model deployment.

• Experience with Git, CI/CD, APIs, software engineering best practices, and cloud platforms such as AWS, Azure, or GCP.

• Strong technical leadership, mentoring, troubleshooting, and communication skills.

Preferred Skills :

• Databricks certification.

• Experience with PySpark, Spark, R, Posit Workbench, or Posit Connect.

• Knowledge of Generative AI, LLM-based applications, model monitoring, and observability.

• Experience in insurance, financial services, risk, or other regulated industries.

Ideal Candidate Profile :

We are looking for candidates with hands-on experience building Python-based Data Science and Machine Learning solutions within Databricks, rather than profiles focused primarily on Data Engineering. Candidates should be comfortable providing technical guidance to Data Scientists and transforming experimental models into reliable, production-ready solutions.