Address
Form of workJob Description
Data engineer who can work on migrating the Spark (Pyspark / Scalaspark) jobs to Cloud based environments, Data pipeline building using Spark,Biquery, Deploy the jobs to multiple environments and monitor the Production jobs. Co-ordinate with offshore & onsite team, any reports generation. Migrating the scripts from Oozie to GCP Composer based scripts
Tech Stack :
- Spark
- GCP Composer (Dags creation)
- GCP Dataproc
- GCP BigQuery, GCS
- Oozie
- HiveQL
- Terradata SQL
