Data Engineer
- Compensation
- Not disclosed
- Location
- Bengaluru
- Work setup
- Onsite
- Seniority
- Not specified
- Employment
- Full time
- Experience
- 3+ years
- Category
- Data engineering
- Posted
Skills
Must have
- Hands-on experience with PySpark for data processing and transformation.
- Understanding of data pipelines, distributed processing, and data engineering concepts.
- Ability to troubleshoot and optimize data processing workflows.
Nice to have
- Experience with Palantir Foundry and code repository management.
- Experience working with Snowflake in data engineering or analytics projects.
- Familiarity with Git or similar version control systems.
- Understanding of data warehousing, ETL/ELT processes, and business intelligence workflows.
Full description
Job Title: Data EngineerCompany: Jnanasethu IT Labs Private Limited
Location: Bengaluru, Karnataka, India
Experience: 3-5 Years
Employment Type: Full-time
Job Description
We are looking for skilled Data Engineers with strong hands-on expertise in PySpark and SQL to support data engineering initiatives, KPI development, and enterprise reporting solutions.
The ideal candidates will have experience in data processing, transformation, and pipeline development. Familiarity with Palantir Foundry, particularly working with code repositories, is a significant advantage for roles supporting HCQ KPIs. For the Quality Dashboard initiative, strong SQL skills and experience working with Snowflake are essential.
The selected candidates will contribute to building reliable data solutions, developing analytical datasets, and supporting business intelligence and dashboarding requirements.
Key Responsibilities
- Develop, maintain, and optimize data processing workflows using PySpark and SQL.
- Build and manage data transformation pipelines to support business KPIs and analytical reporting.
- Work on HCQ KPI-related data engineering activities using PySpark.
- Utilize Palantir Foundry to develop and manage data workflows, where applicable.
- Work with code repositories to manage, maintain, and collaborate on data engineering projects.
- Develop and optimize SQL queries for data extraction, transformation, validation, and analysis.
- Work with Snowflake to query, process, and prepare datasets for the Quality Dashboard initiative.
- Support the development of a new Quality Dashboard to replace the existing dashboard.
- Ensure data accuracy, consistency, quality, and reliability across reporting workflows.
- Troubleshoot data processing issues and optimize query and pipeline performance.
- Collaborate with data engineers, analysts, BI developers, and business stakeholders to deliver effective data solutions.
Required Skills and Qualifications
- PySpark and Data Engineering
- Hands-on experience with PySpark for data processing and transformation.
- Understanding of data pipelines, distributed processing, and data engineering concepts.
- Ability to troubleshoot and optimize data processing workflows.
- SQL and Snowflake
- Strong proficiency in SQL, including complex queries, joins, aggregations, and data transformations.
- Experience querying and processing data using Snowflake.
- Understanding of data validation, query optimization, and performance tuning.