Lead Data Engineer

Summary : Looking for a Lead Data Engineer for its Cloud and Big Data projects in the Data Warehousing, Data Analytics domain.

Location: C2H Working Remotely/Office in NJ/PA/DE state

The Role:

Develop data-driven solutions with Cloud based technologies to meet evolving business needs. Deliver impactful, scalable, and highly performant Data solution to our customers

based in US, UK, Canada and India based customers

Provide technical design and develop Extract/Transform/Load (ETL), ELT applications that interface with Datametica's customer applications.

Lead and implement data driven solutions and collaborate with larger development teams

Design scalable components, frameworks, & libraries for structured and unstructured data ingestion pipelines

Lead and Develop Data Quality, Test Strategy, Test Plan, Test Cases for cloud based solutions

Core Responsibilities

Drive Analysis to understand and meet all technical specifications and business requirements according to the established designs.

Analyze Issues identified during Testing phase, propose solutions to fix the issues.

Provide senior-level technical consulting to peer Data Engineers during design, development and testing for highly complex and critical data projects.

Work extensively within the Cloud ecosystem and migrate data from Teradata/Netezza/ similar on-Premise Big Data platforms to Cloud based platforms like Big Query, SQL

Key Role Requirements

Approx 8+ years of experience with a proven experience in data platform, big Data Engineering and ETL optimization of scalable and distributed systems

Hands-on Experience in SQL is mandatory. Quickly learning skills to adapt GCP cloud native SQL (Bigquery).

Hands-on experience in Python and Scripting (Unix Shell) is mandatory. Experience in using a Jupyter notebook is a plus.

Understanding of fundamentals of Git and Git workflows. Must have used at least one of the SAAS-based Git tools like GitHub, GitLab, Bitbucket.

Hands on and deep experience working with Google Data Products (e.g. BigQuery, Dataflow, Cloud Composer-Airflow, GCS, Compute, Pub-Sub etc.).

Troubleshoot issues and perform administrative tasks like setting up projects, users, authorizations, Migrations and job scheduling.

Willingness to continuously learn & share learnings with others

Capability to collaborate with stakeholders and project leaders to understand requirements, deliverables, and set expectations on tasks that you will be responsible for

Ability to work in a fast-paced, rapidly changing environment

Ability to work with business stakeholders and IT team to understand the requirements and quickly translate them into workable solutions using Python and SQL (Bigquery)

Experience working in an agile and collaborative team environment

Excellent written and verbal communication, presentation and professional speaking skills

Proven problem-solving skills and attention to detail with a commitment to excellence and high standards