Expert LLM Engineer (AI Security)
- Compensation
- Not disclosed
- Location
- Gdańsk, PL
- Work setup
- Hybrid
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- Not specified
- Category
- Ai engineering
- Posted
Skills
Nice to have
- At least 7 years of experience in software engineering, machine learning engineering, or security engineering, including at least 2 years of hands-on experience developing and maintaining LLM-based so
- Good knowledge of Python and the ability to build maintainable, testable software
- Experience building backend components and integrating models with applications, APIs, and data sources
- Understanding of model evaluation and validation principles, including data quality and data leakage between datasets
- Experience with prompt engineering and managing the context provided to models
- Experience building AI agents, including tool calling and orchestrating multi-step workflows
- Experience designing evaluation datasets and regression tests to assess the quality, reliability, and safety of LLM-based solutions
- Practical understanding of security risks related to LLMs and AI agents, with experience testing their resistance to manipulation and attempts to bypass safeguards
- Ability to analyse traffic, telemetry, or behavioural patterns to investigate automated activity and identify potentially unsafe or unauthorised actions
- Experience fine-tuning language models, including data preparation, selecting tuning approaches, and evaluating results
- Experience developing fraud or abuse detection mechanisms based on traffic, telemetry, or behavioural patterns
- Experience investigating mobile application security in Android or iOS environments
- Experience in security research, red teaming, or penetration testing beyond LLM-based systems
- Familiarity with tools for LLM security testing, evaluation, and observability, such as garak, promptfoo, or Langfuse
- Familiarity with RAG, embeddings, and vector databases
- Knowledge of LoRA/QLoRA and libraries such as PEFT or TRL
- Familiarity with Docker, CI/CD, and cloud services used to deploy AI solutions
Full description
About Us
At Framna, we partner with ambitious brands to create digital products that make a meaningful impact.
We work across the full product journey, from strategy and UX to engineering and long-term growth, combining technical expertise with a strong product mindset. Together with our partners, we build solutions that users rely on every day.
Our culture is built on trust, ownership, and fearless collaboration. We challenge ideas, support each other, and keep learning-openly sharing feedback, staying flexible in how we work, and approaching each other with empathy. Because the best products are created by teams that move forward together.
About the Role
You will work on AI-powered security solutions for a next-generation, passwordless authentication platform used by millions worldwide, leveraging advanced biometrics to protect digital identities.
Using Python and LLMs, you will develop and evaluate solutions—from data preparation and fine-tuning to prompting, context management, and integration with APIs and product tools. You will build evaluation datasets and regression tests to improve quality, reliability, performance, and cost efficiency.
Through controlled adversarial testing, you will explore how LLMs and AI agents can be manipulated or used to bypass product safeguards. You will investigate threats across Android and iOS environments, analyse agent behaviour in customer applications, and translate findings into product improvements.
We value occasional in-person meetings at our studio to support collaboration and knowledge sharing.
Preferred Skills & Qualifications
- At least 7 years of experience in software engineering, machine learning engineering, or security engineering, including at least 2 years of hands-on experience developing and maintaining LLM-based solutions in production
- Good knowledge of Python and the ability to build maintainable, testable software
- Experience building backend components and integrating models with applications, APIs, and data sources
- Understanding of model evaluation and validation principles, including data quality and data leakage between datasets
- Experience with prompt engineering and managing the context provided to models
- Experience building AI agents, including tool calling and orchestrating multi-step workflows
- Experience designing evaluation datasets and regression tests to assess the quality, reliability, and safety of LLM-based solutions
- Practical understanding of security risks related to LLMs and AI agents, with experience testing their resistance to manipulation and attempts to bypass safeguards