Data Engineering Team Lead

Career Area:
Business Technologies, Digital and Data
Job Description:
Your Work Shapes the World at Caterpillar Inc.
When you join Caterpillar, you're joining a global team who cares not just about the work we do - but also about each other. We are the makers, problem solvers, and future world builders who are creating stronger, more sustainable communities. We don't just talk about progress and innovation here - we make it happen, with our customers, where we work and live. Together, we are building a better world, so we can all enjoy living in it.
The Opportunity:
This position will be responsible for building scalable, high performance infrastructure and data driven analytics applications that provide actionable insights. The position will be part of Caterpillar's fast-moving Global Parts Pricing organization, driving action and tackling challenges and problems that are critical to realizing superior business outcomes. The Data Engineering Team Lead will mentor a team of data engineers building efficiency and taking key initiatives to successful completion. The Data Engineering Team Lead will work with data scientists, business intelligence analysts, and others as part of a team that assembles large, complex data sets, pipelines, apps, and data infrastructure that provide competitive advantage.
Job Duties:

• Build infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using AWS tools
• Design, develop, and maintain performant and scalable applications
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability
• Perform debugging, troubleshooting, modifications and testing of integration solutions
• Lead pull request and sprint reviews
• Operationalize developed jobs and processes
• Create/Maintain database infrastructure to process data at scale
• Create solutions and methods to monitor systems and processes
• Automate code testing and pipelines
• Engage directly with business partners to participate in design and development of data integration/transformation solutions.
• Engage and actively seek industry best practice for continuous improvement

Required Skills:

• BS in Computer Science, Data Science, Computer Engineering, or related quantitative field
• 4+ years development experience, preferably using Python and/or PySpark
• Experience with AWS cloud services (e.g. EMR, EC2, Lambda)
• Experience administering and/or developing in Snowflake
• Understanding of data structures, algorithms, profiling & optimization
• Understanding of SQL, ETL/ELT design, and data modeling techniques
• Great verbal and written communication skills to collaborate cross functionally and drive action
• Thrive in a fast-paced environment that delivers results
• Passion for acquiring, analyzing, and transforming data to generate insights
• Strong analytical ability, judgment and problem analysis techniques

Desired Skills:

• Masters Degree in Computer Science, Data Science, Computer Engineering, or related quantitative field
• Advanced experience in AWS cloud services (e.g. Glue, CloudFormation, CloudWatch/EventBridge, ECR, ECS, Athena, Fargate, etc.)
• Strong background working with version control systems (Git, etc.)
• Experience managing continuous integrations systems, Azure pipelines is a plus
• Experience with cybersecurity, compliance, and/or IT general controls
• Advanced level of experience with programming, data structures and algorithms
• Working knowledge of Agile Software development methodology
• Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement
• A successful history of manipulating, processing and extracting value from large disconnected datasets

No sponsorship available
Location: Peoria, IL, USA
Additional Information:
This employer is not currently hiring foreign national applicants that require or will require sponsorship tied to a specific employer, such as H, L, TN, F, J, E, O. As a global company, Caterpillar offers many job opportunities outside of the U.S. which can be found through our employment website at www.Caterpillar.com/Careers.
Employee benefit details:
Our goal at Caterpillar is for you to have a rewarding career. Our teams are critical to the success of our customers who build a better world.
Here you earn more than just an hourly wage, because we value your performance, we offer a total rewards package that provides day one benefits (medical, dental, vision, RX, and 401K) along with a yearly STIP bonus.
Additional benefits include paid vacation days and paid holidays (prorated based upon hire date).
Final details:
Please frequently check the email associated with your application, including the junk/spam folder, as this is the primary correspondence method. If you wish to know the status of your application - please use the candidate log-in on our career website as it will reflect any updates to your status.
EEO/AA Employer. All qualified individuals - Including minorities, females, veterans and individuals with disabilities - are encouraged to apply.
#LI
Relocation is available for this position.
Visa Sponsorship is not available for this position. This employer is not currently hiring foreign national applicants that require or will require sponsorship tied to a specific employer, such as, H, L, TN, F, J, E, O. As a global company, Caterpillar offers many job opportunities outside of the U.S which can be found through our employment website at www.caterpillar.com/careers.
Posting Dates:
March 13, 2024 - March 27, 2024
Any offer of employment is conditioned upon the successful completion of a drug screen.
EEO/AA Employer. All qualified individuals - Including minorities, females, veterans and individuals with disabilities - are encouraged to apply.
Not ready to apply? Join our Talent Community.