Address
Form of workJob Description
- Attractive Salary $160 - 200,000
- Comprehensive Benefit and Retirement Programs.
- Paid Relocation, Lots of upward mobility, Sign On Bonuses
Data Pipeline engineers play a crucial role in ensuring that our customers work with updated and accurate data in Foundry. As the first responders to data health failures on key pipelines, they bring technical expertise and a passion for system support. Reading and understanding code to identify fixes, making changes to enhance performance, and ensuring the stability of production Data Pipelines are central to their responsibilities. Collaborating with Pipeline Engineers from various customer accounts, they provide valuable insights to our product engineering team and other user and product support teams.
Core Responsibilities:
- Maintaining and building schedules for pipeline operation.
- Establishing and managing health checks on different pipelines.
- Responding to, triaging, and debugging pipelines in case of failures, including limited out-of-hours support for critical issues.
- Reading and writing code changes, as well as modifying monitoring setups when necessary.
- Navigating the pipelines and documentation effectively.
- Following SOPs to engage other teams and data providers when data issues arise.
- Communicating outages with end users of a pipeline.
- Contributing to and monitoring tooling improvements when feasible.
- Active work is not required on weekends or out of hours, but the engineer must be available to respond to critical outages during their assigned "on-call" weeks. After an on-call weekend, the engineer will receive 2 days off.
Who Will Be a Great Fit?
- TS clearance or higher.
- Comfort with reading and writing code in SQL, Python, Pyspark, and Java.
- Basic understanding of Spark and a willingness to learn the basics of tuning Spark jobs.
- Practical experience in performing root cause analysis and documenting lessons learned from production incidents (e.g., creating post-mortem reports).
- Ability to work within an agile team.
- Strong written and verbal communication skills, with the ability to engage skillfully with customers on complex, sensitive topics.
- Strong organizational skills and attention to detail through effective prioritization.
