Data Engineer (Azure Databricks + Python) - Remote Work | REF#295774
Designs and builds scalable data pipelines and Lakehouse architectures on Azure Databricks using Python and SQL, integrating LLM/AI capabilities to automate classification tasks. Fully remote role at BairesDev, an IT services firm delivering projects for clients like Google and Silicon Valley startups.
- Compensation
- Not disclosed
- Location
- Colombia
- Work setup
- Remote
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- 4+ years
- Category
- Data engineering
- Posted
Skills
Must have
- 4+ years of experience in Data Engineering, Machine Learning Engineering, or Software Engineering
- Proven expertise in developing and optimizing data pipelines using Azure Databricks and Python
- Proficiency in integrating ML/AI solutions and LLM APIs into automated data workflows
- Strong background in data manipulation, automation, and supporting classification tasks
- Deep understanding of cloud-native data architectures and AI integration patterns
- Advanced proficiency in English
Full description
At BairesDev® we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley.
Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works remotely on roles that drive significant impact worldwide.
When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success.
As a Data Engineer, you will design and develop scalable data pipelines leveraging Azure Databricks and Python. You will play a key role in integrating AI capabilities into data workflows, enabling the organization to process complex datasets and automate intelligent classification tasks.
What You'll Do
- Design, build, and optimize end-to-end data pipelines and Lakehouse architectures using Azure Databricks.
- Integrate LLM APIs and machine learning models into data processing streams to support automated classification and insights.
- Develop and maintain high-quality Python code and SQL procedures to ensure efficient data transformation and ingestion.
- Collaborate with AI and product teams to implement data-driven solutions that drive significant business value.
- Monitor and troubleshoot pipeline performance, ensuring reliability and scalability across distributed cloud environments.
- Implement data governance and security best practices to maintain the integrity and compliance of the data platform.
What We Are Looking For
- 4+ years of experience in Data Engineering, Machine Learning Engineering, or Software Engineering.
- Proven expertise in developing and optimizing data pipelines using Azure Databricks and Python.
- Proficiency in integrating ML/AI solutions and LLM APIs into automated data workflows.
- Strong background in data manipulation, automation, and supporting classification tasks.
- Deep understanding of cloud-native data architectures and AI integration patterns.
- Advanced proficiency in English.
How we do make your work (and your life) easier:
- 100% remote work (from anywhere).
- Excellent compensation in USD or your local currency if preferred
- Hardware and software setup for you to work from home.
- Flexible hours: create your own schedule.
- Paid parental leaves, vacations, and national holidays.