Address
Form of work Job Description:
Data engineer who can work on migrating the Spark (Pyspark / Scalaspark) jobs to Cloud based environments, Data pipeline building using Spark,Biquery, Deploy the jobs to multiple environments and monitor the Production jobs. Co-ordinate with offshore & onsite team, any reports generation. Migrating the scripts from Oozie to GCP Composer based scripts.
Tech Stack :
Spark
GCP Composer (Dags creation)
GCP Dataproc
GCP BigQuery, GCS
Oozie
HiveQL
Terradata SQL
Data engineer who can work on migrating the Spark (Pyspark / Scalaspark) jobs to Cloud based environments, Data pipeline building using Spark,Biquery, Deploy the jobs to multiple environments and monitor the Production jobs. Co-ordinate with offshore & onsite team, any reports generation. Migrating the scripts from Oozie to GCP Composer based scripts.
Tech Stack :
Spark
GCP Composer (Dags creation)
GCP Dataproc
GCP BigQuery, GCS
Oozie
HiveQL
Terradata SQL
