Address
Form of workAs a Data Engineer, you will be responsible for designing and building a quality cloud-based data pipeline and its peripheral tooling. You must be self-driven and comfortable working in a highly collaborative environment while interfacing with both technical and non-technical persons. In this role, you will join forces with other teams of software engineers and data scientists to support our business initiatives.
Core Attributes:
The ideal candidate will possess...
- Design and analytical capabilities needed to build cloud-based data platforms
- Collaboration and communication skills are required to support our software engineering practices including agile development, CI/CD, code review, etc.
- Strong software engineering and computer science fundamentals along with software testing experience including unit tests, integration tests, etc.
- A well-developed sense of ownership in their work, driving decisions from inception through completion
- Passion for delivering value to the business with a relentless drive to improve process
- A keen desire to understand the domain knowledge around the products and how the products are developed along with knowledge of industry trends
- A strong sense of ethical responsibility as big data touches the privacy of our clients
- The ability to develop innovative solutions in a greenfield environment
Principal Responsibilities:
- Create and maintain optimal data pipeline architecture
- Identify, design, and implement internal process improvements focused on: automating manual processes, optimizing data delivery, re-designing infrastructure for better scalability, etc.
- Build the infrastructure required for extraction, transformation, and loading of data from various data sources
- Build analytic tools to provide actionable insights into key business performance metrics
- Work with stakeholders across the business and technology to solve their data needs
- Be responsible for key aspects of project delivery at a high level of quality
Required Technical Experience:
Experience in...
- Building and optimizing "Big Data" data pipelines, architectures, and data sets
- Object-oriented or functional languages (Java8+, Python, Scala, etc.)
- CI/CD Tools (Git, Docker)
- AWS cloud services (EC2, ECS, EMR, S3, RDS, etc.)
- ETL Orchestration Tools (Airflow, Glue, Luigi, NiFi, etc.)
- SQL and NoSQL data store (PostgreSQL, MySQL, DynamoDB, Elasticsearch, etc.)
Knowledge of...
- Big data tools and vendors (Hadoop, Spark, Kafka, Databricks, etc.)
- HashiCorp Tools (Terraform, Vault, Consul, etc.)
- Big data concepts (message queuing, streaming processing, high availability, scalability, etc.)
- Data security concepts (data governance, access control, etc.)
- The workflow of data wrangling, feature engineering, model development and model deployment along with necessary architectural requirements
Soft Skills:
- Strong interpersonal skills and ability to forge relationships resulting in successful partnerships
- Ability to convey ideas effectively to help drive solutions for the business and tech partners
- Enthusiasm for learning and is results-oriented
Required Education:
Strong academic record and a BA/BS degree with high mathematical and computing content (e.g., Computer Science, Statistics, Mathematics, Engineering, or Physics)
