Address
Form of workJob Description
Title: Cloud Data Engineer
Location: Greenville, SC Hybrid
Duration: 12 months
Visa: USC, GC, GC EAD, or H4 EAD (Only W2)
Job Description
At-a-Glance:
Are you ready to build your career by joining an international automotive company? If so, our client is hiring a Cloud Data Engineer!
What You'll Do:
Are you ready to build your career by joining an international automotive company? If so, our client is hiring a Cloud Data Engineer!
What You'll Do:
- Provides complete application lifecycle development, deployment, and operations support for large-scale batch and real-time data processing pipelines using cloud technologies.
- Collaborates with product owners, data scientists, business analysts and software engineers to design and build solutions to ingest, transform, store, and export data in a cloud environment while maintaining security, scalability, and personal data protection.
- Responsibilities:
- Implements and enhances complex data processing pipelines with a focus on collecting, parsing, cleaning, managing and analyzing large data sets that produce valuable business insights and discoveries.
- Determines the required infrastructure, services, and software required to build advanced data ingestion & transformation pipelines and solutions in the cloud.
- Assists data scientists and data analysts with data preparation, exploration, and analysis activities.
- Applies problem solving experience and knowledge of advanced algorithms to build high-performance, parallel, and distributed solutions.
- Performs code and solution review activities and recommends enhancements that improve efficiency, performance, stability, and decreased support costs.
- Applies the latest DevOps and Agile methodologies to improve delivery time.
- Works with SCRUM teams in daily stand-up, providing progress updates on a frequent basis.
- Supports application, including incident and problem management.
- Performs debugging and triage of incident or problem and deployment of fix to restore services.
- Documents requirements and configurations and clarifies ambiguous specs.
- Performs other duties as assigned by management.
What You Bring:
- Bachelor's Degree in Business, Computer Science or Electrical Engineering preferred or the equivalent of 4 years professional IT related experience. MS degree (preferred).
- 3+ years of Experience:
- Enterprise software engineering experience with object oriented
- Design, coding and testing patterns, as well as, experience in engineering (commercial or open source) software platforms and large-scale data infrastructure solutions.
- Software engineering and architecture experience within a cloud environment (Azure, AWS).
- Enterprise Data Engineering experience within any "Big Data" environment (preferred).
- Software development experience using Python.
- Experience working in large-scale data integration and analytics projects, including using cloud (e.g. AWS Redshift, S3, EC2, Glue, Kinesis, EMR) and data-orchestration (e.g. Oozie, Apache Airflow) technologies
- Implementing distributed data processing pipelines using Apache Spark
- Designing relational/NoSQL databases and data warehouse solutions
- 2+ years of Experience:
- Writing and optimizing SQL queries in a business environment with large-scale, complex datasets
- Experience working in an Agile environment (Scrum, Lean or Kanban).
- Unix/Linux operating system knowledge (including shell programming).
- 1+ years of Experience
- Experience in automation/configuration management tools such as Terraform, Puppet or Chef.
- Experience in container development and management using Docker.
- Languages: SQL, Python, Spark
- Basic knowledge of continuous integration tools (e.g. Jenkins).
- Basic knowledge of machine learning algorithms and data visualization tools such as Microsoft Power BI and Tableau
