Databricks AI/ML Engineer

Key Responsibilities :

• Design, develop, and implement scalable AI and machine learning solutions using Databricks.

• Build and maintain end-to-end machine learning pipelines for data preparation, model development, training, validation, and deployment.

• Develop data processing and transformation workflows using PySpark and SQL.

• Perform data exploration, preprocessing, feature engineering, and statistical analysis to prepare datasets for machine learning.

• Develop, train, evaluate, and optimize machine learning models based on business and technical requirements.

• Work with large and complex datasets within the Databricks environment.

• Develop reusable and production-ready Python and PySpark code for data processing and ML workflows.

• Collaborate with data engineers, data scientists, business teams, and technology stakeholders to understand requirements and deliver AI/ML solutions.

• Implement scalable data pipelines and workflows to support machine learning applications.

• Monitor model performance and identify opportunities for model optimization and improvement.

• Troubleshoot data, model, pipeline, and performance-related issues.

• Apply appropriate machine learning algorithms and techniques based on the problem statement and available data.

• Support deployment and operationalization of machine learning models in production environments.

• Follow coding, testing, documentation, and development best practices.

• Contribute to improving the scalability, reliability, and performance of AI/ML solutions.

Technical Skills :

• Strong hands-on experience with Databricks and its capabilities for data engineering and machine learning.

• Strong programming experience in Python.

• Good experience with PySpark for distributed data processing.

• Strong SQL skills for data extraction, transformation, and analysis.

• Solid understanding of Machine Learning and Artificial Intelligence concepts.

• Experience with data preprocessing, feature engineering, model training, evaluation, and optimization.

• Experience working with large-scale datasets and distributed computing environments.

• Exposure to Azure or AWS cloud platforms is preferred.

• Knowledge of MLOps practices and ML model deployment is good to have.

• Understanding of end-to-end ML lifecycle and productionization of models.

MLOps & Cloud Exposure :

• Understanding of practices involved in taking ML models from development to production.

• Exposure to model deployment, monitoring, versioning, and lifecycle management.

• Familiarity with cloud-based AI/ML environments on Azure or AWS.

• Experience implementing automated and scalable ML workflows will be an advantage.

Candidate Profile :

• 10+ years of relevant experience in AI/ML, Data Science, Machine Learning Engineering, or related technology roles.

• Strong problem-solving and analytical skills.

• Ability to work with large datasets and translate complex data problems into scalable solutions.

• Strong understanding of software development and data engineering practices.

• Ability to collaborate effectively with cross-functional teams.

• Good communication skills with the ability to explain technical concepts to business and technical stakeholders.

• Passion for building scalable and production-ready AI/ML solutions using modern data platforms.