W2 | Spark Developer | Hybrid (Charlotte, NC, Dallas, TX)

Hello,

My name is Chandra Bhushan, and I am a Staffing Specialist at Softsages

LLC.

I am reaching out to you on an exciting job opportunity with one of our clients.

Job Title: Spark Developer (W2)

Location: Hybrid (Charlotte, NC, Dallas, TX)

Visa: USC, GC.

Need LinkedIn (Mandatory)

Job Description:

1. Lead complex technology initiatives including those that are company wide with broad impact.
2. Act as a key participant in developing standards and companywide best practices for engineering complex and large scale technology solutions for technology engineering disciplines.
3. Design, code, test, debug, and document for projects and programs.
4. Review and analyze complex, large scale technology solutions for tactical and strategic business objectives, enterprise technological environment, and technical challenges that require in depth evaluation of multiple factors, including intangibles or unprecedented technical factors.
5. Make decisions in developing standard and companywide best practices for engineering and technology solutions requiring understanding of industry best practices and new technologies, influencing and leading technology teams to meet deliverables and drive new initiatives.
6. Collaborate and consult with key technical experts, senior technology team, and external industry groups to resolve complex technical issues and achieve goals.
7. Lead projects, teams, or serve as a peer mentor.

Required Qualifications:

1. 5 plus years of Software Engineering experience, or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, or education.

Job Expectations:

1. Design and implement automated sparkbased framework to facilitate data ingestion, transformation and consumption.
2. Implement security protocols such as Kerberos Authentication, Encryption of data at rest, data authorization mechanism such as role based access control using Apache ranger.
3. Design and develop automated testing framework to perform data validation.
4. Enhance existing spark based frameworks to overcome tool limitations, and/or to add more features based on consumer expectations.
5. Design and build a high performing and scalable data pipeline platform using Hadoop, Apache Spark, MongoDB, Kafka and object storage architecture.
6. Work with Infrastructure Engineers and System Administrators as appropriate in designing the big data infrastructure
7. Collaborate with application partners, Architects, Data Analysts and Modelers to build scalable and performant data solutions.
8. Effectively work in a hybrid environment where legacy ETL and Data Warehouse applications and new big data applications coexist
9. Work with Infrastructure Engineers and System Administrators as appropriate in designing the big data infrastructure.
10. Support ongoing data management efforts for Development, QA and Production environments
11. Provide tool support, help consumers troubleshooting pipeline issues.
12. Utilizes a thorough understanding of available technology, tools, and existing designs.
13. Leverage knowledge of industry trends to build best in class technology to provide competitive advantage.

Required Qualification:

1. 5 plus years of experience of software engineering experience
2. 5 plus years of experience delivering complex enterprise wide information technology solutions
3. 5 plus years of experience delivering ETL, data warehouse and data analytics capabilities on big data architecture such as Hadoop
4. 5 plus years of Apache Spark design and development experience using Scala, Java, Python or Data Frames with Resilient Distributed Datasets (RDDs), Parquet or ORC file formats
5. 6 plus years of ETL (Extract, Transform, Load) Programming experience
6. 2 plus years of Kafka or equivalent experience
7. 2 plus years of NoSQL DB like Couchbase/MongoDB experience.
8. 5 plus experience working with complex SQLs and performance tuning

Desired Qualification:

1. 3 plus years of Agile experience
2. 2 plus years of reporting experience, analytics experience or a combination of both
3. 2 plus years of operational risk or credit risk or compliance domain experience
4. 2 plus years of experience integrating with RESTful API
5. 2 plus years of experience with CICD tools.

Manager Notes:

1. Team Duties and business impact: The Risk Data Services is a horizontal function within Risk Technology organization and is responsible for delivering data consistently across Risk.
2. The Risk Data Services team is seeking a Lead Software Engineer. Driving strategy for the team for the entire platform and help with migration to the cloud.
3. Big Data Hub, the tools are used purely for ETL functions, moving and conforming data, not for building web applications.
4. The use of Python and Java is for Data Engineering, we are not seeking Web Developers, back and front end etc.
5. Functional Industry Title: Big Data Hadoop Engineer

Requirements:

1. This is a need for someone to build and write the capabilities.
2. Experience building and enhance scala framework and spark experience
3. Java
4. Python

--

Best Regards,

Chandra Bhushan

Desk No: (305) 428-7458

Mail: Chandra.bhushan@softsages.com