Big Data Engineer

Title: Data Engineer

Location: McLean, VA (3 days hybrid)

Duration: 6 + Months (with possible extension)

Position Overview:

They are seeking a Senior Data Engineer who enjoys data and building data storage platforms from ground up. The ideal candidate has a passion for data analysis, technology and helping people leverage the technology to transform their business processes and analytics. As a Data Engineer, you will be part of a team responsible for supporting a wide range of internal customers. You will draw on all the skills in your toolkit to analyze, design, and develop data storage and data analytic solutions using data lake patterns, that help our customers run more effective operations and make better business decisions.

Must Haves:

• Object oriented programming experience using Python.
• SQL
• API experience is mandatory with a preference for familiarity with Boto3.
• Experience with Pyspark with a solid understanding of Big data.

Preferred Skills:

• Understanding of Apache Hadoop and the Hadoop ecosystem. Experience with one or more relevant tools (Sqoop, Flume, Kafka, Oozie, Hue, Zookeeper, HCatalog, Solr, Avro).
• Deep knowledge on Extract, Transform, Load (ETL) and distributed processing techniques such as Map-Reduce
• Experience with Columnar databases like Snowflake, Redshift
• Experience in building and deploying applications in AWS (EC2, S3, Hive, Glue, EMR, RDS, ELB, Lambda, etc.)
• Experience with building production web services
• Experience with cloud computing and storage services
• Knowledge of Mortgage industry

Qualifications

• At least 5 years of experience developing in Python, SQL (postgres/snowflake preferred)
• Bachelor's degree with equivalent work experience in computer science, data science or a related field.
• Experience working with different Databases and understanding of data concepts (including data warehousing, data lake patterns, structured and unstructured data)
• 3+ years' experience of Data Storage/Hadoop platform implementation, including 3+ years of hands-on experience in implementation and performance tuning Hadoop/Spark implementations.
• Implementation and tuning experience specifically using Amazon Elastic Map Reduce (EMR).
• Implementing AWS services in a variety of distributed computing, enterprise environments.
• Experience writing automated unit, integration, regression, performance, and acceptance tests.
• Solid understanding of software design principles

Strategy Development and Implementation

• Develop data filtering, transformational and loading requirements.
• Define and execute ETLs using Apache Sparks on Hadoop among other Data technologies.
• Determine appropriate translations and validations between source data and target databases.
• Implement business logic to cleanse & transform data.
• Design and implement appropriate error handling procedures.
• Develop project, documentation, and storage standards in conjunction with data architects.
• Monitor performance, troubleshoot and tune ETL processes as appropriate using tools like in the AWS ecosystem.
• Create and automate ETL mappings to consume loan level data source applications to target applications.
• Execution of end-to-end implementation of underlying data ingestion workflow.

Operations and Technology

• Leverage and align work to appropriate resources across the team to ensure work is completed in the most efficient and impactful way.
• Understand capabilities of and current trends in Data Engineering domain

Dexian is a leading provider of staffing, IT, and workforce solutions with over 12,000 employees and 70 locations worldwide. As one of the largest IT staffing companies and the 2nd largest minority-owned staffing company in the U.S., Dexian was formed in 2023 through the merger of DISYS and Signature Consultants. Combining the best elements of its core companies, Dexian's platform connects talent, technology, and organizations to produce game-changing results that help everyone achieve their ambitions and goals.

Dexian's brands include Dexian DISYS, Dexian Signature Consultants, Dexian Government Solutions, Dexian Talent Development and Dexian IT Solutions. Visit https://dexian.com/ to learn more.

Dexian is an Equal Opportunity Employer that recruits and hires qualified candidates without regard to race, religion, sex, sexual orientation, gender identity, age, national origin, ancestry, citizenship, disability, or veteran status.