Data Engineer Series (Level I, Ii, Iii)

Job ID: 5299
Business Unit: MTA Headquarters
Location: New York, NY, United States
Regular/Temporary: Regular
Department:
Date Posted: Jan 16, 2024
Description
JOB TITLE: Data Engineer Series (Level I, II, III)
SALARY RANGES: Level I: $67,172 - $88,163
Level II: $75,484 - $99,073
Level III: $82,411 - $108,165
HAY POINTS: Level I: 406
Level II: 496
Level III: 571
DEPT/DIV: Chief Strategic Initiatives Officer/ Data & Analytics
SUPERVISOR: Manager, Data Engineering
LOCATION: 2 Broadway New York, NY 10004
HOURS OF WORK: 9:00 am - 5:30 pm (7 1/2 hours/day)
DEADLINE: Open Until Filled
SUMMARY:
The MTA delivers subway, bus and commuter rail service and operates tolled bridges and tunnels in the New York City metro area. It is the largest public transportation provider in the Western Hemisphere, with millions of daily riders and a multi-billion-dollar budget. Delivering service and maintaining our infrastructure is increasingly data-driven, so data scientists are now central to the MTA's operations and decision-making. As a member of the Data & Analytics team, you will work on a dynamic, growing team tasked with helping to define and achieve the MTA's strategic priorities and address its most pressing challenges. Your work will become public data and influence critical decisions affecting millions of New Yorkers.
Our work combines new approaches and algorithms with a deep knowledge of our operations to deliver deeper insights into our performance. Our outputs are used by a wide array of stakeholders: from the MTA Board and senior execs to front-line managers to our open data customers. If you want to use your programming skills to solve complex problems and improve public transit in NYC, then we'd like to talk to you.
Data Engineers design, build, test, and deliver end-to-end, automated data pipelines over complex on-premises and off-premises platforms. They extract data from multiple source systems containing structured, semi-structured and unstructured data , refine it to make it consistent, reliable, and available to colleagues on the Data & Analytics team, operations and management teams across the MTA , and, in support of the agency's and New York State's Open Data goals, to external stakeholders and the general public .
They will use languages such as SQL and Python and tools such as Oracle, Postgres, and SQL Server to analyze large datasets, build new ones, and design overall data architectures. They will carefully document all work and work closely with colleagues to define needs, problem-solve, support the overall team agenda, and build relationships throughout and at all levels of the agency. They will have to be able to quickly learn the unique features, data constraints and business needs of any part of the MTA. In addition, unlike other Data Engineering roles, they will support the entire downstream pipeline process and, occasionally, end-users of the data products. In general, they will have to support the MTA's strategic goals to build data systems and processes that are well-structured and sustainable.
RESPONSIBILITIES :
All Data Engineers must be able to:

• Develop and operationalize data pipelines to collect, structure, and integrate a wide range of data sources and make information available to staff and decision-makers for analysis and consumption.
• Support the development of new data sets, data access, extraction methodologies, algorithms, and processes to improve performance.
• Help e valuate legacy data sources and implement plans for migrat ion to modern data infrastructure to improve efficiency, timeliness, quality, robustness, and scalability.
• T est data sets and pipelines for data quality and usability , conduct i n g root cause analyses in response to issues , and implement cost effective resolutions for data anomalies.
• Prepare data architecture diagrams and system documentation as well as summaries of business rules, data flow maps, and other design artifacts to ensure understanding of all users and audiences, both technical and non-technical.
• Keep skills current and m aintain data systems in the latest technologies and cloud platforms for collect ion , integrat ion , ingest, transform ation , and organiz ation to support analyses, reporting, a nd all business needs .
• C arry out assigned project workstreams and communicate with teammates, clients and all relevant parties on project status.
• Support a professional environment that respects individual differences, enables all colleagues to develop and contribute to their full potential at the MTA and beyond .
• Perform other duties as assigned.

Level 2 Data Engineers must also be able to:

• Project manage new data pipeline development and legacy data system conversion to create data system assets that are efficien t , timel y , reliable and accurate , robust, and scalab le .
• Develop new methodologies , algorithms , and scripts as needed for extraction , transformation , and quality assurance .
• Help develop standards and c reate actual data architecture diagrams, system documentation, data models, business decision-making maps, and other design related artifacts to ensure consistency and improve understanding of both technical and non-technical users of Data & Analytic products .
• Provide input into data governance and sustainable data management policies and practices across the agency.
• Research and recommend the best pipeline technologies , toolsets, and applications to support the data science and reporting teams' efforts .
• Proactively support the design of workstreams and project plans, update project plans as needed, and communicate with clients and all relevant parties on project status.
• Support efforts to recruit new engineers and develop existing teammates . As requested by manager, r eview team member code for efficiency, clarity, and adherence to best practices.

Level 3 Data Engineers must also be able to:

• Operationalize data pipelines, data warehouses, data marts, multi-dimensional cubes, and data lakes.
• Find solutions to project challenges using enterprise-class solutions .
• Lead project planning and support manager in overall work planning, team and client communications.
• Ensure data governance and sustainability needs are met in project designs and systems maintenance.
• Identify and propose new data-analysis projects that support the agency strategic goals, service quality and efficiency. Advance data-driven decision-making across the company through stakeholder partnering, tool design, and effective communications.
• Support the selection and development of teammates within the department. Mentor less-experienced members of the team to build their business and coding/analytic knowledge and skills.

KNOWLEDGE, SKILLS, AND ABILITIES
All Data Engineers must have:

• Strong skills in software engineering , database designs, and data pipeline orchestration .
• Ability to work with structured, semi-structured, unstructured data from numerous disciplines , such as transportation, finance, HR, and asset maintenance .
• Experience with Data Engineering orchestration tools (e.g., Airflow) and DevOps tools (e.g., Git) and familiarity with related concepts.
• Ability to manage workstreams and work on team-based projects.
• Ability to think at a policy and strategic level.
• Strong written communication skills for both non-technical and technical documents .

Level 2 Data Engineers must also have:

• Ability to collaborate with data scientists, data reporters, and end users on project design and business goals.
• Extensive e xperience work ing un structured, as well as semi-structured and more structured data from a variety of policy areas - as well as from distinct disciplines - transportation, finance, HR, asset maintenance .
• Experience managing with the operating and organization constraints of the public management sphere .
• Familiarity with common algorithms used to calculate KPIs and an u nderstanding of how data are fed into data visualization/ business intelligence tools such as PowerBI .
• Experience in quality assurance techniques and automated testing practices.
• Knowledge of current innovations and techniques in Data Engineering.
• Ability to present work and engage constructively with agency colleagues at all levels, from frontline managers to executives , who use data and data products for decision-making .
• Ability to project manage and help lead team-based projects.

Level 3 Data Engineers must also have:

• Ability to play a lead role on the Data & Analytics team to set priorities and overall program planning in support of business goals.
• Excellent judgment on how to manag e or overcome technical , organizational, or governance constraints to meet business goals .
• Extensive experience with data visualization and/or business intelligence tools such as PowerBI .
• Extensive e xperience and ability to design and implement q uality assurance and automated testing systems .
• Extensive experience in project design and management and the ability to lead successful team-based projects.
• Proactive interest in identifying strategic, policy , and business issues in all proposed and ongoing projects .
• Ability to present and engage on complex work products with executive audiences and to review and make decisions on project design options with clients and stakeholders.

EDUCATION AND EXPERIENCE :
All Data Engineers must have:

• A bachelor's degree in Computer Science, Engineering, Information Management, Statistics, Mathematics, or an equivalent combination of education and experience. Master's degree preferred.
• A minimum of (1) year of experience working in Data Engineering or a position with similar programing and data management content.
• A minimum of (1) year of experience building data pipelines, automating tasks through scripts, writing database queries and debugging/ maintaining code.
• A minimum of (2) years of experience with Python and SQL programming.
• A minimum of (2) years of experience with relational databases (e.g., Oracle, Postgres, SQL Server), including writing queries to obtain and manipulate data.

Level 2 Data Engineers must also have:

• At least (2) years' experience working in a Data Engineering or a position with similar programing and data management content.
• A minimum of (2) years of experience building pipelines, automating tasks through scripts, writing database queries and debugging/ maintaining code.
• A minimum of (3) years of experience with Python and SQL programming
• A minimum of (3) years of experience with relational databases (e.g., Oracle, Postgres, SQL Server), including writing queries to obtain and manipulate data.
• A master's degree may substitute for one year of experience in each area.

Level 3 Data Engineers must also have:

• At least (3) years' experience working in a Data Engineering or a position with similar programing and data management content.
• A minimum of (3) years of experience building pipelines, automating tasks through scripts, writing database queries and debugging/ maintaining code.
• A minimum of (4) years of experience with Python and SQL programming
• A minimum of (4) years of experience with relational databases (e.g., Oracle, Postgres, SQL Server), including writing queries to obtain and manipulate data.
• A master's degree may substitute for one year of experience in each area.

OTHER INFORMATION:
Pursuant to the New York State Public Officers Law & the MTA Code of Ethics, all employees who hold a policymaking position must file an Annual Statement of Financial Disclosure (FDS) with the NYS Commission on Ethics and Lobbying in Government (the "Commission").
EQUAL EMPLOYMENT OPPORTUNITY
MTA and its subsidiary and affiliated agencies are Equal Opportunity Employers, including with respect to veteran status and individuals with disabilities.
The MTA encourages qualified applicants from diverse backgrounds, experiences, and abilities, including military service members, to apply.