Lead AI Engineer (San Francisco, CA)
Lead AI Enterprise Data Engineer to architect scalable data platforms and production-grade AI solutions using Databricks, AWS, and LLMs, powering real-time monitoring dashboards and engineering productivity tools for a global consumer technology company. Requires deep expertise in Databricks/PySpark, AI/ML model deployment, and building data pipelines across eCommerce and MarTech domains.
- Compensation
- Not disclosed
- Location
- Not listed
- Work setup
- Hybrid
- Seniority
- Lead
- Employment
- Not specified
- Experience
- 5+ years
- Category
- Ai engineering
- Posted
Skills
Full description
About our client
Our client is a global leader in consumer cybersecurity — protecting people, not just devices — with integrated, intuitive products that keep families and communities safe online. Their eCommerce platform is the commercial engine behind that mission, serving millions of customers worldwide across purchase, checkout, subscription, and auto-renewal journeys.
The eCommerce Engineering Intelligence team is being built to answer a deceptively hard question: how is eCommerce actually performing, technically and commercially, right now? This role is the hands-on technical builder behind that answer.
The role
You will own the build of two strategic initiatives:
-
A unified operational intelligence platform — an eCommerce Health view that combines system signals (latency, errors, integration and data-flow health) with commercial performance (conversion, checkout, auto-renewal, feature adoption) into one trusted source of truth.
-
AI enablement across engineering — shipping agents and productivity tools that engineering teams actually adopt and use every day.
This is a builder's role, not a platform administration role. You will be writing Spark, designing schemas, tuning jobs, and shipping applications that stakeholders open and rely on.
What you'll doBuild the data foundation
- Bring data from eCommerce, MarTech, observability, and operational datastores into an analytics-ready and AI-ready state through production ETL/ELT pipelines, expert SQL, and deliberate data modeling — with automated quality validation built in.
- Design dimensional models (star/snowflake schemas, sensible partitioning and grain) purpose-built to power analytics, KPI reporting, and dashboards.
- Build gold-layer, summarized datasets — collapsing millions of raw events into one clean summary record per entity per day — the aggregation layer that makes natural-language querying and executive dashboards trustworthy rather than approximately correct.
Make performance visible
- Design and ship the eCommerce Health view: one place where system health and commercial outcomes sit side by side.
- Surface standardized engineering metrics across teams — delivery velocity (DORA-style), quality, security posture, incident trends, and service ownership.
Ship AI that reduces toil
- Build AI agents that detect anomalies, surface critical issues with actionable context, notify the right stakeholders, and drive intelligent triage that accelerates root-cause analysis.
- Build AI productivity tools that help teams ship faster and operate more reliably — RCA assistance, intelligent search across telemetry and code, runbook copilots.