Mid-Level/Lead DATA ENGINEER-Python, AWS, Spark (Hybrid-TX & IL)
Experienced data engineer on State Farm's Property & Casualty Data Pipeline team, building and scaling automated data products and ETL pipelines that power business decisions and analytics. Core stack is Python/PySpark with AWS services (Glue, EMR, Lambda, Redshift, S3), Databricks/Spark, Airflow CI/CD, and OpenTofu IaC, in a hybrid Richardson, TX role.
- Compensation
- $85.5K–$130K / yr
- Location
- Richardson, TX
- Work setup
- Hybrid
- Seniority
- Lead
- Employment
- Not specified
- Experience
- Not specified
- Category
- Data engineering
- Posted
Skills
Must have
- Professional experience as a Data Engineer
- Proficiency in programming languages such as Python, Spark SQL (or PySpark), R, Java, Bash
- Hands-on experience with AWS services including Glue, EMR Serverless, Lambda, Step Functions, EventBridge, S3, DynamoDB, Kinesis Firehose, Redshift, Iceberg, and SageMaker
- Experience with distributed data processing frameworks such as Apache Spark, Databricks
- Experience with infrastructure as code tools such as OpenTofu (formerly Terraform)
- Familiarity with CI/CD pipelines including automated testing, security scans, and tools like Airflow
Nice to have
- Experience or ability to rapidly gain P&C data domain knowledge, including rating, underwriting, and/or claims
- Experience with relational databases such as DB2, Postgres, Redshift
- Experience with version control systems such as GitHub or GitLab
- Data access skills using SQL and Athena
- Experience designing, building, and maintaining data pipelines for automated data processing
- Knowledge of data modeling techniques such as star schema and snowflake schema, with an understanding of data architecture
Full description
Overview
Being good neighbors – helping people, investing in our communities, and making the world a better place – is who we are at State Farm. It is at the core of how we operate and the reason for our success. Come join a #1 team and do some good!
Hybrid Work Arrangement: A “Hybrid” position means selected candidates will be assigned to a facility and will spend some time working both in the office and from home. For Enterprise Technology, a hybrid work arrangement will require approximately 40% in-office time each month, beginning in late 2027.**
HUB LOCATIONS: Richardson, TX; Bloomington, ILSPONSORSHIP: ** Applicants for this position are required to be eligible to lawfully work in the U.S. immediately; employer will not sponsor applicants for U.S. work authorization (e.g. H-1B visa) for this opportunity.
Responsibilities
Our Property & Casualty Data Pipeline team is hiring a Data Engineer to help advance one of the company’s highest priorities. In this role, you will build and scale the data products and pipelines that power better business decision, profitable growth, and stronger customer retention. You will develop automated, reliable analytical data assets that help transform how we operate and how we serve our customers.
This position sits within Enterprise Technology and offers the opportunity to work in a highly collaborative environment focused on delivering meaningful business impact. You will partner across teams, solve complex data challenges, and help shape modern data solutions aligned to enterprise goals. It is a strong fit for someone who wants to expand their technical skills, work on high-visibility initiatives, and contribute in a fast-moving, innovative setting.
Your Responsibilities May Include:
- Utilizes industry-adopted languages and frameworks in coding, testing, security, DevOps, DataOps and data engineering practices
- Develops and maintains reusable, scalable, and compliant data solutions across multiple platforms and compute environments
- Responsible for the identification, acquisition, cleansing, profiling, and ETL (extracting, transformation, and loading) of data used in analytic discovery and production solution deployment across multiple platforms
- Establishes business domain knowledge for existing State Farm data sources and investigates, recommends, and initiates acquisition of data resources, both internal and external
- Identifies and consults on emerging technologies and critical core systems, including techniques, tools, data sources, and platforms in the data engineering field
- Familiar with handling datasets containing mixes of structured and unstructured data
- Exhibits DataOps mindset where team is accountable for ensuring data aligns to enterprise needs and leveraging automation to deliver quality data solutions