Ian Martin Group
DataOps/Cloud Data Engineer - Senior
12-month onsite contract in Toronto for a senior DataOps/cloud data engineer who designs, builds and optimizes end-to-end data pipelines and automates metadata collection and data integration. Broad cloud data platform experience is expected: DaaS/DBaaS/DWaaS, ETL/ELT, data warehouse/lake/lakehouse, DataOps, CI/CD and data governance.
- Compensation
- Not disclosed
- Location
- Toronto, Canada
- Work setup
- Not specified
- Seniority
- Senior
- Employment
- Contract
- Experience
- Not specified
- Category
- Data engineering
- Posted
Skills
Must have
- Experience with cloud data platforms, data management and data exchange tools and technologies
- Experience with commercial and open-source data/database development and management, incl. DaaS, DBaaS, DWaaS and other storage platforms (cloud and on-premise)
- Experience with data pipeline and workflow development, orchestration, deployment and automation
- Proficiency in multiple programming languages to build pipelines, automate tasks and write scripts
- Experience with DataOps principles, best practices, implementation and Agile project development/deployment
- Experience with CI/CD and data provisioning automation
- Experience with digital product, data analysis, data exchange, data provisioning and data security
- Expert experience in data conversion and migration of very large OLAP/OLTP environments to SaaS, PaaS and IaaS
- Experience designing and implementing fact/dimension models, data mapping, data warehouse, data lake and data lakehouse
- Experience managing cloud data services: storage, repositories, data lake, data lakehouse, key vault, virtual machine, disk
- Experience with structured, semi-structured and unstructured data collection, ingestion, provisioning and exchange
- Experience with DataOps performance monitoring and tuning
- Excellent analytical, problem-solving, decision-making, communication and presentation skills
- Team player with a track record for meeting deadlines
- Demonstrated experience designing, implementing and supporting cloud-based data platforms and services
- Experience developing scalable ETL/ELT processes, data pipelines and integration solutions
- Experience implementing DataOps practices: deployment automation, testing, monitoring, operational support
- Experience with cloud-native data engineering and modern data platform architectures
- Experience implementing metadata, lineage and data quality capabilities in cloud data environments
- Experience developing integration solutions using APIs, event-driven architectures and batch or streaming technologies
- Knowledge of data governance, security and operational best practices for cloud platforms
- Experience automating metadata collection, data movement and operational processes
- Experience supporting cloud modernization and data migration initiatives
- Experience collaborating with architects, modellers and governance teams to operationalize data standards and common data models
- Knowledge of performance optimization and scalability for cloud data platforms
- Ability to troubleshoot complex technical problems and develop practical solutions
Full description
DataOps/Cloud Data Engineer - Senior
- Pay Rate: Flexible- Onsite
- Location: Toronto, Ontario
- Contract Length: 12 Months
Responsibilities
- Designing and developing data pipelines from source to end user
- Optimizing data pipelines
General Skills
- Experience with Cloud data platforms, data management and data exchange tools and technologies
- Experience with Commercial and opensource data and database development and management and specializing in data storage setting and managing cloud Data As a Service (DaaS). application Database as a Service (DBaaS), Data Warehouses as a Service (DWaaS), and other storage platforms (both in the cloud and on-premise).
- Experience with Data pipeline and workflow development, orchestration, deployment, and automation and specializing in programming and pipelines to create and managing the dataflow and movement of data.
- Experience with Cloud data engineer must be familiar and experienced with different programming languages and be able to integrate with many different platforms to create data pipelines, automate tasks, and write scripts.
- Experience in DataOPS principle, best practice, and implementation and Agile project development and deployment
- Experience in Continuous Integration/Continuous Development/Deployment (CI/CD) and Data provisioning Automation
- Experience with digital product, data analysis, data exchange, data provisioning, and data security
- Extensive expert experience in designing/developing and implementing data conversation and migration of VLD (Very large Data) of Online analytical processing (OLAP) and online transaction processing (OLTP) environments to Cloud Software-as-a-service (SaaS), Platform-as-a-service (PaaS) and Infrastructure-as-a-service (IaaS) environments.
- Experience in design, development and implementation of fact/dimension model, data mapping, data warehouse, data lake and data lakehouse for enterprise
- Experience managing Cloud Data services for project delivery, including storage, repositories, Data Lake, Data Lakehouse, key vault, virtual machine, disk, etc.
- Experience with structured, semi-structured, unstructured data collection, ingestion, provisioning and exchange technological development of enterprise data warehouse and data lake and data lakehouse solutions and operational support.
- Experience with DataOPS performance monitoring and tuning
- Excellent analytical, problem-solving and decision-making skills; verbal and written communication skills; presentation skills; interpersonal and negotiation skills