Data Integration Lead
Apply on company site View on the company's website
Posted on the company's own website.
Can I apply?
- Work permit not stated
Data Integration Lead role focused on real-time data pipelines on Azure Databricks, PySpark, Kafka, ADF and ADLS, handling 10M+ records daily.
Responsibilities include building medallion-style ETL workflows in SQL and PySpark and fixing upstream data quality and requirement gaps.
The posting does not state employment type, workplace, location within Canada, salary, or work authorization requirements, so eligibility is unclear.
From the posting
- locations
ATS location: Canada
- job function
Data Integration Lead
- role family
Data Integration Lead
- skills
Azure Databricks, PySpark, Kafka, Azure Data Factory (ADF), and Azure Data Lake Storage (ADLS)
- actionable
Developed and optimized SQL and PySpark transformations within medallion-style ETL workflows
Job description
Real-time data pipelines using Azure Databricks, PySpark, Kafka, Azure Data Factory (ADF), and Azure Data Lake Storage (ADLS), enabling reliable ingestion and transformation of 10M+ records daily. • Developed and optimized SQL and PySpark transformations within medallion-style ETL workflows, improving processing efficiency, maintainability, and data quality for downstream analytics. • Identified requirement gaps and data quality issues stemming from upstream data sources and solutioned root-cause fixes by implementing validation rules, refining transformation logic, and collaborating with stakeholders to improve downstream data reliability.
Seen on
- oraclecloud — fa-etqo-saasfaprod1.fa.ocs.oraclecloud.com