Senior Data Engineer
Senior Data Engineer at RS21 designs, builds, and owns AWS-based data pipelines and platforms powering analytics, AI, and LLM applications across both product and client-facing professional services work — spanning batch/streaming ETL/ELT, embedding and vector-store pipelines, and mentoring of junior engineers.
- Compensation
- Not disclosed
- Location
- US
- Work setup
- Not specified
- Seniority
- Senior
- Employment
- Not specified
- Experience
- 5+ years
- Category
- Data engineering
- Posted
Skills
Must have
- Bachelor's degree or equivalent experience in data engineering, computer science, or related field
- 5+ years hands-on data engineering experience with progression into senior/lead responsibilities
- Hands-on experience with AWS data services: S3, Glue, Redshift, Athena, Kinesis or MSK, Lambda, Lake Formation
- Proven ability to design and deliver production-grade ETL/ELT pipelines and data warehousing solutions
- Solid understanding of streaming architectures and real-time data pipeline design
- Experience supporting AI or LLM-adjacent data workflows, including embedding pipelines and vector store integration
- Clear communication with technical and non-technical stakeholders, including direct client interaction
- Experience decomposing and tracking delivery work in Jira or equivalent tooling
- Strong problem-solving and architectural reasoning at the workstream level
Nice to have
- AWS certifications: Solutions Architect Associate (SAA-C03) or Data Engineer Associate (DEA-C01)
- Databricks Certified Data Engineer Associate and/or dbt Certified Developer
- Experience with infrastructure-as-code tools such as Terraform or CDK
- Background in consulting, professional services, or multi-client delivery environments
- Familiarity with data governance frameworks, data cataloging, and lineage tooling
- Experience with Databricks, dbt, and Airflow or Prefect orchestration
- Exposure to vector databases such as Amazon OpenSearch, Pinecone, or pgvector
- Exposure to DoD, federal, or regulated-sector data environments; FedRAMP-compliant architecture experience a plus
Full description
Position Summary
The Senior Data Engineer is a technically deep, independently operating practitioner who designs, builds, and maintains the data foundations that power RS21's analytics, AI, and LLM-driven applications across both product and professional services work. This role focuses on data readiness, cloud-native data architecture, and AI enablement, ensuring that LLM-powered systems are supported by reliable, scalable, and high-quality data pipelines.
At the Senior level, this person does not require day-to-day direction. They lead pipeline architecture decisions, establish data contracts for their workstreams, mentor junior and core-level engineers, and serve as a reliable technical contributor in client-facing engagements. They translate business and product requirements into well-structured, maintainable technical solutions, and they understand how their work connects to the broader delivery plan.
This role bridges hands-on execution with team-level technical leadership. The Senior Data Engineer is the person a project team depends on to own the data layer end to end, flag problems before they become blockers, and bring enough communication skill to engage directly with clients and cross-functional partners without requiring a translator.
Key Responsibilities
Data Pipelines & Platforms
- Lead pipeline architecture decisions for assigned workstreams, including batch and real-time ingestion, transformation, and delivery to analytics and AI layers.
- Design, build, and maintain scalable ETL/ELT workflows with strong data quality, lineage, monitoring, and observability built in from the start.
- Establish data contracts and pipeline standards for the projects you own, and ensure those standards are followed by other contributors on the workstream.
- Ensure data reliability, performance, and scalability across platforms; proactively surface bottlenecks and recommend improvements before they affect delivery.
- Support both batch and streaming ingestion patterns, including real-time data pipeline design and implementation.
LLM Enablement & AI Data Foundations
- Design and implement data pipelines that support LLM and AI use cases, including:
- Document and unstructured data ingestion
- Data preprocessing, enrichment, and embedding generation
- Vector store integration and retrieval-optimized data structures
- Lead embedding pipeline architecture and vector store configuration in collaboration with developers building LLM features.
- Ensure data freshness, lineage, and governance for AI-powered systems.
- Optimize data structures and retrieval patterns to support efficient LLM context usage.
- Contribute to RAG pipeline design and maintain awareness of foundation model data requirements across active engagements.