Senior Lead Software Engineer - Databricks

We have an opportunity to impact your career and provide an adventure where you can push the limits of what's possible.

The Chief Data & Analytics Office (CDAO) at JPMorgan Chase is responsible for accelerating the firm’s data and analytics journey. This includes ensuring the quality, integrity, and security of the company's data, as well as leveraging this data to generate insights and drive decision-making. The CDAO is also responsible for developing and implementing solutions that support the firm’s commercial goals by harnessing artificial intelligence and machine learning technologies to develop new products, improve productivity, and enhance risk management effectively and responsibly.

As a Senior Lead Software Engineer at JPMorgan Chase within the Chief Data Analytics Office - AIML Data Platforms Team, you will play a pivotal role in developing and delivering cutting-edge technology products that are secure, stable, and scalable. You will work within an agile team to solve complex cloud data platform challenges, leveraging your expertise in software engineering, cloud technologies, and data platforms.

Job responsibilities

• Executes creative software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or break down technical problems
• Solves the companies most challenging cloud data platform problems by building innovative technical solutions around Data Lake Tools.
• Designs, implements, and maintains a managed AWS Databricks platform, and provides engineering and operational support for the platform to SRE and app teams.
• Performs platform design, set-up and configuration, workspace administration, resource monitoring, providing engineering support to data engineering teams, Data Science/ML, and Application/integration teams.
• Develops secure high-quality production code, and reviews and debugs code written by others
• Identifies opportunities to eliminate or automate remediation of recurring issues to improve overall operational stability of software applications and systems
• Leads evaluation sessions with external vendors, startups, and internal teams to drive outcomes-oriented probing of architectural designs, technical credentials, and applicability for use within existing systems and information architecture
• Leads communities of practice across Software Engineering to drive awareness and use of new and leading-edge technologies
• Adds to team culture of diversity, opportunity, inclusion, and respect

Required qualifications, capabilities, and skills

• Formal training or certification on software engineering concepts and 5+ years applied experience
• Hands-on experience with Python and/or Java application program development with use of automated unit testing
• Hands-on practical experience delivering system design, application development, testing, and operational stability
• Hands-on practical experience with terraform development and understanding of terraform enterprise
• Hands-on experience with GitHub / Bitbucket code versioning tool, Jenkins build tool and pypi / maven artifactory integrations
• Knowledge of Big Data distributed compute frameworks like Spark

Preferred qualifications, capabilities, and skills – 4 bullets max

• Exposure to AWS & Databricks Platform administration
• Experience with Agile development processes, as needed (SCRUM/KANBAN) using JIRA.
• Experience in Data pipelines using Spark
• Experience in managing product release lifecycle at enterprise level.