Junior Mid-Level Machine Learning Data Engineer
Junior/mid-level ML data engineer at PredictX, a London enterprise SaaS company, building and maintaining the data pipelines, warehousing, and infrastructure behind its AI/predictive-analytics product. Day to day: Spark, Python, SQL, ETL/ELT, data modelling, plus exposure to LLMs and ML deployment alongside senior engineers and data scientists.
- Compensation
- Not disclosed
- Location
- Gdańsk, Poland
- Work setup
- Onsite
- Seniority
- Middle
- Employment
- Not specified
- Experience
- 1+ years
- Category
- Data engineering
- Posted
Skills
Must have
- 1-3 years of experience in data engineering or machine learning roles
- Proficiency in Spark (PySpark and/or Scala)
- Proficiency in Python
- Strong skills in at least one other relevant language like Scala or Java
- Command of SQL for complex data querying and manipulation
- Practical experience with a variety of ETL/ELT tools and frameworks
- Understanding of data modelling techniques (e.g. star schema, dimensional modelling) and data warehousing concepts
- Knowledge of data governance, data quality principles, and data security best practices
- Exposure to data integration, cleansing, and transformation on large datasets, including data preparation for ML and LLMs
- Familiarity with data profiling and data lineage tools
- Ability to identify, diagnose, and resolve data issues, performance bottlenecks, and data quality problems
- Analytical and problem-solving skills to translate data into technical solutions
- Excellent written and verbal communication skills for technical and non-technical audiences
- Strong teamwork and collaboration skills across teams
- Proactive, solution-oriented approach with drive to learn new AI/LLM technologies
- Meticulous attention to detail for data accuracy and integrity
- Proven ability to write clear technical documentation
Nice to have
- Solid understanding of ML fundamentals, algorithms, and libraries (scikit-learn, TensorFlow, PyTorch) with interest in NLP and LLMs
- Experience deploying and monitoring ML models, including LLMs, in production
- Experience building and maintaining data pipelines with orchestration tools like Apache Airflow
- Familiarity with big data technologies beyond Spark (Hadoop, Kafka, NoSQL such as MongoDB/Cassandra, streaming platforms)
- Experience with cloud platforms (AWS, Azure, or GCP) and their data/ML services (e.g. SageMaker, Azure AI)
- Understanding of CI/CD pipelines for data and ML deployments
- Exposure to statistical analysis and data visualization tools (R, Tableau, Power BI)
- Experience with prompt engineering and fine-tuning of LLMs
Full description
About PredictX
Make a real difference at one of London’s foremost SaaS scale-ups: Be ready to pioneer the future of AI, data analytics, and technology. Step into PredictX, where we don't just see AI as a fashionable bandwagon but have lived and breathed AI & ML in every aspect of our product for the past decade.
As an Enterprise SaaS provider, we're revolutionising critical decision-making for many of the world’s largest businesses, including 3 FAANGs, seeking empowerment through our integrative AI technology and Predictive Analytics.
We pride ourselves on our commitment to staying at the forefront of technological advancements. You'll be joining a team that actively explores and integrates the latest innovations to maintain our competitive edge.
The Role
As a Junior/Mid-Level ML Data Engineer, you’ll help build and maintain the data pipelines and infrastructure that power our AI solutions. You’ll work closely with senior engineers and data scientists, gaining exposure to LLMs, big data tools, and real-world ML deployment. This is a growth role for someone eager to learn, contribute, and evolve in the AI space
Key Responsibilities
- Assist to develop and maintain scalable and efficient data pipelines using technologies such as Spark, Python, and relevant ETL tools to support our machine learning models, including those leveraging LLMs, and analytical needs.
- Support with the architect and implement robust data warehousing solutions and data models that ensure data quality, integrity, and performance, catering to the specific data requirements of advanced AI models.
- Assist in the development, testing, and deployment of machine learning models, including exploration and integration of Large Language Models (LLMs) and other novel AI architectures, collaborating closely with Data Scientists to productionize innovative solutions.
- Help transform engineer approaches for storing, transforming, transporting, synchronising, archiving, and securing large and complex datasets, including unstructured and semi-structured data crucial for training and deploying advanced AI models.
- Participate in the evaluation and testing of new machine learning models and frameworks, including LLMs, to assess their potential and applicability to our products.
- Assist to identify and resolve performance bottlenecks, data quality issues, and other pain points within our data and ML infrastructure. Proactively recommend and implement solutions for optimisation and improvement, especially in the context of deploying large-scale AI models.
- Define and govern data modelling and design standards, best practices, and development methodologies within the team, considering the unique challenges and opportunities presented by LLMs and other advanced AI.