Azure Data Engineer

The Role

Senior Data Engineer to contribute towards the success of our Data and Analytics Technology initiatives. This is a hands-on technical role, and this person will be responsible for the development and build of scalable solutions as it relates to the architecture strategies, data standards, digital data management, data integration, tools, and technology. The right candidate will play a deep dive hands-on critical development role in the digital transformation and in shaping how we acquire, ingest, transform, and deliver data through the digital consumer channels.

Key Responsibilities

• Architect, design, develop and engineering end-to-end data pipelines across multiple data sources and systems of record.
• Ensure data quality, integrity, security, and completeness throughout the data lifecycle.
• Develop, design data models, data structures and ETL jobs for data acquisition and transformation purposes.
• Develop deep understanding of the data sources, implement data standards, maintain data quality and master data management.
• Developing data services and API
• Work closely with the cloud service providers to ensure completeness and alignment with the service offerings.
• Manage and maintain cloud-based data and analytics platform.
• Deep understanding of the cloud offerings and engage in quick proof of concepts and proof of value in prototyping data and analytics solutions and derive viability.
• Understanding of Unified Data governance and Data Quality Management solutions for managing and governing enterprise data.
• Ability to interact with the business stakeholders to understand requirements and translating into technology solutions.

Technical Qualifications

• Bachelor's degree in computer science, computer engineering, or other related field of study with 10+ years of relevant experience
• Experience in Azure Cloud platform service offerings
• Expertise in Azure SQL with strong knowledge on writing complex SQL queries, Stored procedures, Tuning and Query optimizations.
• Experience with Azure API Management Service (APIMS) and developing Azure Functions for exposing API / RESTful data services.
• Experience working on real-time data capture, processing, transformation and storing using technologies like Azure Event Hubs, Azure Service Bus, and Azure Stream Analytics
• Knowledge of creating pipelines and complex data transformations and manipulations using one of the languages .Net/C#, Python, Java, R, or Scala with Databricks/Spark
• Experience in NoSQL Databases and Big data technologies including Azure CosmosDB or other Document stores.
• Experience with Azure Purview for automating and managing Data catalog, classification, lineage, and metadata management across enterprise data.
• Experience with configuring rules, exceptions and approvals workflows for Data quality management using Profisee.
• Experience working with different data storage options including ADSLS Gen2 and Azure BLOB storage.
• Understanding of different data formats including Parquet, Avro, CSV, ORC etc.
• Understanding of MPP databases and maintain large amount of data processing is a plus.
• Experience in creating, scheduling, and monitoring Azure Data Factory (ADF) pipelines.
• Experience with Azure Synapse Analytics and Azure Analysis Services, Power BI and business Intelligence technologies is a big plus but not mandatory.
• Past working experience on a fast paced and agile environment
• Experience working with an on-shore / off-shore model.

Build and meet project timelines and manage delivery commitments with proper communication to management.