Data Engineer

Technical/Functional Skills

Frameworks: Spark, PySpark, Python, Apache Airflow and Cassandra, Sprint Boot Micro Services, Apache Camel, Angular 6 UI Framework

DB: Postgres DB, Oracle

DevOps: GITLAB, Jenkins, Maven, Confluence, Jfrog Artifactory, SONARQUBE, JIRA

Experience Required

• Must have strong experience in Spark, PySpark, Python, Apache Airflow and Cassandra.

• Must have strong experience in Apache Flink, Python, Apache Kafka and Cassandra

.

• Build near real time data pipeline with Apache Spark, Pyspark & Python to extract data from NoSQL DBs and Load it into Cassandra Data Enterprise Graph

• Build real time data pipeline with Apache Flink, Apache Kafka and Python to extract data from Relational and NoSQL DBs and Load it into Cassandra Data Enterprise Graph

Roles & Responsibilities

o Ability to translate application storyboards and use cases into functional applications

o Build near real time data pipeline with Apache Spark, Pyspark & Python to extract data from NoSQL DBs and Load it into Cassandra Data Enterprise Graph

o Build real time data pipeline with Apache Flink, Apache Kafka and Python to extract data from Relational and NoSQL DBs and Load it into Cassandra Data Enterprise Graph

o Design, build, and maintain efficient, reusable, and reliable Java code

o Ensure the best possible performance, quality, and responsiveness of the applications

o Help maintain code quality, organization, and automatization

o Strong debugging, problem solving and investigative skills. Ability to assimilate disparate information (log files, error messages etc.) and pursue leads to find root cause of problems

o Experience in Agile/Scrum methodology

o Proven ability to create innovative solutions.

o Strong agile/scrum development experience

o Expertise developing web applications and APIs using Micro Services, with demonstrable production-scale experience.

o Web applications development using Java, HTML5, CSS3, JavaScript, jQuery, Angular JS, Spring Boot.

o Skilled n Version control tools e.g., GIT, SVN, etc. to maintain common source Create testing and validation scripts for the features developed.

o Develop the Automation Script for end-to-end verifications

o Create, enhance, debug and execute the Test cases

o Writing unit / Integration tests using JUnit / Mockito for code coverage as per the standards and Log4j 2 for logging.

o Expert in database scripts, stored procedures, functions, triggers etc