Data Scientist

Data Scientist Model Evaluation & MLOps

We are seeking a Data Scientist with strong expertise in machine learning model development, evaluation, and operationalization. The role focuses on designing robust evaluation frameworks, defining and tracking model performance metrics, conducting experimentation using MLflow, improving prediction reliability through calibration techniques, and managing model lifecycle governance through a model registry. The ideal candidate will work closely with business stakeholders, data engineers, and ML engineers to develop, validate, monitor, and continuously improve machine learning models while ensuring reproducibility, explainability, and production readiness.

Key Responsibilities

• Design and implement model evaluation frameworks, validation strategies, and acceptance criteria.
• Define, analyze, and monitor ML performance metrics across classification, regression, and probabilistic models.
• Conduct model experimentation, hyperparameter tuning, and benchmark testing using MLflow experiment tracking.
• Apply model calibration techniques to improve prediction confidence and probability estimates.
• Analyze model performance through backtesting, cross-validation, and champion-challenger comparisons.
• Manage model versioning, governance, and promotion workflows through a model registry.
• Collaborate with data engineers to develop feature engineering and data preparation pipelines.
• Monitor model drift, prediction quality, and business impact in production environments.
• Communicate model insights, assumptions, and recommendations to business stakeholders.
• Contribute to model explainability, observability, and AI governance initiatives.

Required Skills

• Strong knowledge of Machine Learning, Statistics, and Predictive Modeling.
• Experience with Python, Scikit-learn, XGBoost, LightGBM, or similar ML frameworks.
• Hands-on experience with MLflow for experiment tracking and model management.
• Expertise in model evaluation metrics (RMSE, MAE, Precision, Recall, F1, ROC-AUC, Calibration Metrics, etc.).
• Experience with model calibration techniques such as Platt Scaling and Isotonic Regression.
• Understanding of model monitoring, drift detection, and MLOps best practices.
• Experience working with large datasets, feature engineering, and data analysis.

Eval design, ML metrics, MLflow experiment tracking, calibration, model registry

Requirement is immediate.