Rosen's Diversified, Inc - American Foods Group, LLC
Senior Cloud Data Engineer
Designs, builds, and optimizes cloud data pipelines and lakehouse solutions on Azure Databricks (PySpark, Python, SQL, Delta Lake) to support manufacturing operations, BI, and analytics at American Foods Group. Integrates ERP, MES, and Industrial IoT data, governs access via Unity Catalog, and partners with business stakeholders in an Agile team.
- Compensation
- Not disclosed
- Location
- Hansen Road, Brown County
- Work setup
- Not specified
- Seniority
- Senior
- Employment
- Not specified
- Experience
- Not specified
- Category
- Data engineering
- Posted
Skills
Must have
- Bachelor's degree in computer science, information systems, data engineering, or related field (or equivalent experience)
- 3–5 years of experience in data engineering, cloud engineering, or analytics platform development
- Minimum 2 years of hands-on experience with Azure Databricks
- Experience supporting enterprise data solutions in a manufacturing or industrial environment
- Strong knowledge of Azure Databricks, Apache Spark, PySpark, Python, SQL, Delta Lake, lakehouse architecture, data modeling, and data warehousing
- Experience with Azure Data Lake Storage Gen2, Azure Data Factory, Azure DevOps, Git, and CI/CD for data platforms
- Experience with data quality, security, monitoring, troubleshooting, and performance tuning
- Knowledge of manufacturing data domains such as ERP, MES, production planning, inventory, supply chain, quality, or Industrial IoT
- Ability to translate manufacturing processes into reliable data requirements with operational stakeholders
Nice to have
- Experience with Microsoft Fabric, Power BI, or real-time/streaming data workloads
- Hands-on experience with Unity Catalog and governed lakehouse implementations
- Exposure to machine learning or AI data preparation and deployment patterns
- Experience with Infrastructure as Code using Terraform, Bicep, or ARM templates
- Knowledge of Infor M3 application design, business data model, and core table structures
- Understanding of production, planning, inventory, quality, maintenance, and supply chain processes
- Microsoft Certified: Azure Data Engineer Associate
- Databricks Certified Data Engineer Associate or Professional
- Microsoft Certified: Fabric Analytics Engineer Associate
Full description
JOB SUMMARY
The Senior Cloud Data Engineer designs, develops, and optimizes scalable data solutions that support manufacturing operations, business intelligence, advanced analytics, and digital transformation. This role builds and maintains cloud-based data pipelines using Azure Databricks and related Azure services to integrate enterprise operational and business data. The ideal candidate combines hands-on cloud data engineering experience with collaboration across business stakeholders, analysts, architects, and data engineers to deliver reliable, secure, and high-performing data products that enable data-driven decisions across the organization.
Essential Functions and Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines using Azure Databricks, PySpark, Python, and SQL.
- Build and optimize ingestion frameworks for ERP, MES, Industrial IoT, quality, supply chain, inventory, finance, and related operational systems.
- Develop reusable engineering patterns and standards that improve delivery speed, supportability, and reliability.
- Implement and support lakehouse solutions using Delta Lake and Medallion Architecture across Bronze, Silver, and Gold layers.
- Design and optimize data models for reporting, analytics, AI, and machine learning use cases.
- Maintain data lineage, metadata, technical documentation, and operational runbooks.
- Integrate Azure Databricks with Azure Data Factory, Azure Data Lake Storage Gen2, Azure Key Vault, Microsoft Fabric, Power BI, and Azure DevOps.
- Support orchestration and movement of data between enterprise applications and cloud platforms.
- Monitor and optimize Databricks clusters, jobs, notebooks, and workflows for performance, scalability, and cost efficiency.
- Troubleshoot production issues, identify root causes, and implement sustainable corrective actions.
- Contribute to monitoring, alerting, testing, and operational support practices.
- Apply organizational standards for data quality, governance, privacy, security, and compliance.
- Implement role-based access controls and governed data access using Unity Catalog and Azure security services.
- Build automated validation, auditing, and data quality controls into engineering workflows.
- Partner with business users, analysts, architects, and IT teams to translate requirements into maintainable technical solutions.
- Participate in Agile ceremonies, sprint planning, code reviews, and technical design reviews.
- Mentor junior engineers and promote coding, testing, documentation, and deployment best practices.
QUALIFICATIONS, KNOWLEDGE, SKILLS, AND EXPERIENCE
- Bachelor's degree in computer science, information systems, data engineering, or a related field; an equivalent combination of education and experience may be considered.