Lead Software Engineer - Data Engineering | Data Technology
The Lead Software Engineer will design and optimize large-scale ETL data pipelines and cloud-native data solutions using Python, PySpark, and AWS. This role involves managing modern data lake architectures and Snowflake environments while championing engineering best practices and AI-assisted development within a financial services context.
- Compensation
- Not disclosed
- Location
- Columbus, OH, United States
- Work setup
- Not specified
- Seniority
- Lead
- Employment
- Not specified
- Experience
- 5+ years
- Category
- Data engineering
- Posted
Skills
Must have
- 5+ years of applied experience in data engineering
- Proficiency in Python and Java
- Hands-on experience with PySpark or distributed data processing frameworks
- Expertise in DBT and modern ETL practices
- Experience with workflow orchestration platforms like Control-M or Apache Airflow
- Expertise with AWS data services including Glue, Athena, CloudWatch, and Lambda
- Knowledge of Iceberg or Delta Lake
- Experience with Snowflake administration and development
- Strong SQL skills
Nice to have
- Experience with Kafka or Flink
- Familiarity with AI/ML technologies including LLMs and vector search
- Experience with AI-assisted development tools like GitHub Copilot
- Financial services industry experience
- Experience mentoring engineers and leading technical delivery
Full description
This is your chance to change the path of your career and work at one of the world's leading financial institutions.
As a Lead Software Engineer – Data Engineering at JPMorgan Chase within the Consumer & Community Banking/Data Products team, you serve as a seasoned member of an agile team to design and deliver trusted market-leading technology products in a secure, stable, and scalable way. You are responsible for carrying out critical technology solutions across multiple technical areas within various business functions in support of the firm’s business objectives.
Job Responsibilities:
-
Design, develop, and optimize large-scale ETL (Extract Transform Load) data pipelines.
-
Build high-quality Python applications using modular code, reusable components, logging, and automated testing.
-
Develop and maintain distributed data processing solutions using PySpark.
-
Large scale end-to-end testing design and validation.
-
Implement and support workflow orchestration using Control-M or Apache Airflow (MWAA).
-
Develop cloud-native solutions leveraging AWS services, including Glue, Athena, Lambda, and CloudWatch.
-
Design and manage modern data lake architectures utilizing Iceberg and/or Delta Lake.
-
Administer and optimize Snowflake environments, including streams, tasks, roles, and warehouses.
-
Participate in code reviews and champion engineering best practices, testing standards, and CI/CD processes.
-
Leverage approved AI-assisted development tools while ensuring secure, responsible, and compliant software delivery.
Required qualifications, capabilities, and skills
-
Formal training or certification on software engineering concepts and 5+ years applied experience with a strong focus on data engineering.
-
Experience in developing, debugging, and maintaining code in a large corporate environment with one or more modern programming languages – Python (primary) & Java (secondary)
-
Hands-on experience with PySpark or other distributed data processing frameworks.
-
Strong expertise in DBT (Data Build Tool) and modern ETL (Extract Transform Load) development practices.
-
Experience with workflow orchestration platforms such as Control-M or Apache Airflow (MWAA).