Address
Form of workJob Description
The Data Engineering Consultant has the following responsibilities and duties:
- Create and maintain optimal data architecture pipeline.
- Perform root cause analysis on internal and external data, processes and share insights with different stakeholders using reporting and visualizations tools.
- Design infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources into a data warehouse
- Understand existing implementation and should enhance it as needed.
- Work with other Data Engineers, data ingestion specialists, and experts across cross-functional teams.
- Work on large datasets to meet functional and business requirements
Requirements
The Data Engineering Consultant should have the following skills, education, and experience:
Skills
Required:
- Excellent SQL and Python skills
- Experience with Spark programming language(s)
- Experience with Docker/containerization
- Expertise in designing efficient Data Models
- Experience with ETL/ELT and familiarity of Data Warehouse concepts
- Strong knowledge across Microsoft Azure Tools.
- Data pipeline and workflow management tools: Databricks (Spark + Python), ADF, Dataflow
- Knowledge of API-based integration and security (Restful and SOAP)
- Experience with Data Visualization tools such as Power BI etc.
- Ability to learn fast and translate data into actionable results
- Excellent written and communication skills with client stakeholders
Preferred:
- Experience with JSON Programming
- Experience developing and supporting scalable data pipelines
- Experience with Snowflake Data Warehouse
- Solid Linux programming skills
- Cosmos DB, Teradata, SQL Server (all nice to have)
- Stream-processing systems: Streaming-Analytics, IoT Hub, Event Hub, Kusto Queries (all nice to have)
Education
Required:
- Bachelors in a quantitative field such as computer science, computer engineering other related disciplines.
Preferred:
- Masters or PhD in a quantitative field such as computer science, computer engineering other related disciplines.
Experience
- An experienced professional will have at least 5+ years of professional experience in Data Engineering role with a Masters Degree or 5+ years of experience in a Data Engineering related role with a Bachelors degree
Location: Anywhere in Canada
#LI-REMOTE
