Senior Data Engineer, Databricks
Senior Data Engineer builds and maintains secure, scalable data pipelines and platforms for a federal client using Databricks, GCP, BigQuery, and Python to support analytics, AI, and decision-making.
- Compensation
- Not disclosed
- Location
- Not listed
- Work setup
- Remote
- Seniority
- Senior
- Employment
- Full time
- Experience
- 10+ years
- Category
- Data engineering
- Posted
Skills
Must have
- Bachelor's degree and 12+ years of experience, or a Master's degree and 10+ years.
- Led or served as the primary engineer on at least one enterprise Databricks deployment from initial setup, including workspace configuration, Unity Catalog, identity and access management, and cluster
- Hands-on infrastructure as code for Databricks, using Terraform or Databricks Asset Bundles, along with CI/CD promotion across environments.
- Production experience with Delta Lake, Python, SQL, and pipeline orchestration.
- Experience applying security and governance controls in a regulated or federal environment.
Nice to have
- Federal security frameworks such as FedRAMP and NIST 800-53.
- Databricks certification, such as Data Engineer Professional or Platform Administrator.
Full description
Location: Fully Remote, United States
Position Type: Full-Time
Clearance: U.S. citizenship is required. The candidate must be eligible to obtain and maintain federal suitability, a government-issued PIV credential, and required system access.
The Opportunity
TeamWorx Security is seeking a Senior Data Engineer to design, build, and maintain enterprise data platforms and pipelines for a federal government client. The Senior Data Engineer will work with technical teams, program leadership, and government stakeholders to integrate complex data sources, improve data quality, and deliver secure, scalable data products that support analytics, automation, artificial intelligence, and operational decision-making. The ideal candidate has stood up and governed an enterprise Databricks platform from initial deployment, including Unity Catalog, identity and access, and environment promotion.
The work rewards engineers who can move between projects, learn a new business area quickly, and apply their skills wherever the need is greatest. The ideal candidate brings strong data engineering fundamentals, hands-on experience delivering production solutions in an enterprise Databricks environment, and familiarity with Google Cloud Platform.
Success in this role requires initiative, ownership, and clear communication. The Senior Data Engineer is expected to work through ambiguity, identify needs or risks before they become problems, and move assignments forward without waiting for detailed direction. This person must communicate progress, decisions, and obstacles clearly while remaining accountable for work through delivery and ongoing support.
What You’ll Be Doing
- Deploy and configure enterprise Databricks workspaces, including Unity Catalog metastore and catalog structure, cluster policies, network and private connectivity settings, and environment separation across development, test, and production.
- Implement data governance in Unity Catalog, including role-based access, group and identity federation, data classification and tagging, row- and column-level security, lineage, and audit logging aligned to federal security requirements.
- Build CI/CD and infrastructure-as-code pipelines that deploy Databricks resources, jobs, and code consistently across environments.
- Build and support production data pipelines and layered data models in Databricks, Python, and SQL that feed analytics, reporting, and AI/ML.
- Document platform architecture, access models, and runbooks, and brief federal stakeholders on design decisions, risks, and status.
Required:
- Bachelor's degree and 12+ years of experience, or a Master's degree and 10+ years.