Big Data Engineer

• Hands on experience on Big Data ecosystem
• Mastering/Leading in the development of applications/tools using Sqoop, Spark, Python, Scala packages
• Good understanding of BI tools (Tableau/Power BI, MicroStrategy) for Big Data 
• Knowledge on Big suites Hive, Oozie, Kafka, good experience in Hadoop (Native) Big Data processing
• Experience of writing Hive queries
• Familiarity with development best practices such as code reviews and unit testing.
• Experience in designing the automation framework using Python, Perl and Shell scripting.
• Leading, guiding and helping the team members with the challenges faced during the project development process and problem solving.
• Have flexibility and ability to learn and use new technologies and to work in team environment as wells as independently to get things done
• Excellent communications skill
• Experience in working with client and handle the pressure 
• Diagnosing software issues and engineering workarounds
• Experience in working with users, user roles using Ranger
• Good understanding on security and administration

Technology – 

• Big data (Native) Ecosystem 
• OS: CentOS/Linux/Unix
• Tool/Programs: Sqoop, Spark, Impala, Python, Scala
• Database: Hive, HBase, NoSQL

Qualification – 

              Bachelor's Degree in Computer Science or equivalent

Primary Responsibilities

• Act as a Big data subject/solution matter expert and liaison to Big Data users and IT
• Serve as an escalation point for production and/or platform issues and acts as lead liaison with Technical Support to resolve outstanding issues within the Big Data environment
• Execute actions necessary to achieve and maintain optimum Big Data platform performance, reliability and availability Identify bottlenecks in production systems and provide solutions 
• Provide Big Data Platform capacity planning, metrics, and performance tuning recommendations
• Use Big Data Management tools (Ranger)

• • Creating Users/Groups/Roles
  • Creating and using Access Control Templates
  • Creating, deployment and scheduling jobs
  • Defining connections to data sources, libraries, metadata objects, and folder structures

Qualifications

• 5+ years of experience working with and maintaining IT Application support.
• 3+ years of experience with Big Data Engineer
• Experience in production data loads and data/storage management for multiple data sources.
• Knowledge of change management procedures and version control is required
• Experience/Knowledge of application security, performance and tuning
• Passion about technology and continuous learning and mastering computer-related technologies 
• Strong communication skills and ability to work as a team

Good to have

• Strong knowledge of Big Data open-source communities.