Lead Data Engineer

Key Skills: Big data, AWS, ETL

3+ years' experience as a lead . The lead should have experience leading a scrum team of 5 or more engineers .

We are looking for a skilled and motivated Spark Data Engineer, to lead a small team of talented Data Engineers. In this role, you'll bring advanced subject matter knowledge to solve complex business issues, and we'll look to your Spark Data Engineering subject matter expertise! In this role, you will frequently contribute to the development of new ideas and methods. You will also get to work on complex, interesting problems, where analysis of situations or data requires an in-depth evaluation of multiple factors. We'd love for you to provide expertise to functional project teams, and you may also participate in cross-functional initiatives.

Responsibilities

• Lead a team of 5-8 talented, motivated engineers and help them deliver while remaining hands on throughout the time.
• In this role, you'll bring advanced subject matter knowledge to solve complex business issues, and we'll look to your Spark Data Engineering subject matter expertise!
• You will also get to work on complex, interesting problems, where analysis of situations or data requires an in-depth evaluation of multiple factors
• We'll rely on your experience and expertise to provide direction and guidance to process improvements, including in helping to establish/advise on policies
• Leads one or more project teams of other Data Engineers for all stages of design and development for complex, secure and performant data solutions and models, including design, analysis, coding, testing, and integration of structured/unstructured data
• Ability to work as a Big Data Engineer as an individual contributor or team player
• Mines data using modern tools and programming languages
• Collaborates and communicates with project team regarding project progress and issue resolution
• Communicates and drives data insights/innovation into the business
• Builds and manages relationships throughout the organization
• Ties insights into effective visualizations communicating business value and innovation potential
• Manages relationships with business partners to evaluate and foster data driven innovation, provide domain-specific expertise in cross-organization projects/initiatives
• Maintains proficiency within the data science domain by keeping up with technology and trend shifts
• Contributes to industry data science domain initiatives
• Represents the data science team for all phases of larger and more-complex development projects
• Defines and implements models to uncover patterns and predictions creating business value and innovation
• Provides guidance, training and mentoring to team members
• Collaborates with peers, junior engineers, data scientists and project team
• Typically interacts with high- level Individual Contributors, Managers, Directors and Program Core Teams
• Leads multiple projects requiring Data Engineering solutions development
• Drives design innovation resolution.

Knowledge & Skills

• Extensive Hands-on experience on Big Data frameworks like Spark, Hive using Scala or Python
• Extensive experience in AWS services - Redshift, Athena, EMR, DocumentDB, S3
• Knowledge of AI & Data Science
• Exposure to CI/CD pipeline & Github.
• Experience in ETL, Data Lake and Data warehouse pipeline.
• Experience using statistics, mathematics, algorithms and programming languages to solve big data challenges.
• Fluent in structured and unstructured data, its management, and modern data transformation methodologies.
• Ability to define and create complex models to pull valuable insights, predictions and innovation from data.
• Effectively and creatively tell stories and create visualizations to describe and communicate data insights.
• Strong analytical and problem-solving skills.
• Excellent written and verbal communication skills; mastery in English and local language.
• Ability to effectively communicate data insights and negotiate options at senior management levels.

Scope & Impact

• Collaborates with peers, junior engineers, data scientists and project team.
• Typically interacts with high- level Individual Contributors, Managers, Directors and Program Core Teams.
• Leads multiple projects requiring Data Engineering solutions development.
• Drives design innovation.

Education & Experience

• Bachelor's, Master's or PHD degree in Mathematics, Economics, Physics, Computer Science, or equivalent.
• 6-10 years' professional experience