Job Title: GCP Data Engineer
Location: Cary, North Carolina (on-site)
Type: Full Time
Job Description:
- Data Engineer will be responsible for providing expertise in implementation of data ingestion pipeline using
- GCP native managed services.
- Strong experience with relational and non-relational databases in Cloud with billions of records (structured &
- unstructured data)
- Ability to design & develop data flow pipelines from scratch
- Excellent problem solving & debugging skills along with designing skills
- Must have experience of GCP services mainly Cloud Storage, Dataflow, BigQuery, Pub/Sub, Cloud Composer
- etc and BigQuery optimisation techniques
- Sound experience on creating data flow scalable modules using Apache Bean Java pipeline
- End to end automation experience for pipeline automation
- Good to have DataProc, DataFusion experience
- Exposure to Bigdata platform
- Exposure to processing of Avro and Parquet file formats ( good to have ) using serverless architecture
- Good exposure and hands on knowledge on Data Warehouse / Data Lake solutions both on premise and in
- cloud.
- Involvement in cloud migration projects and experience on data platform modernization
- Comprehensive experience on programming language - Java
- Excellent communication skills / ability to articulate to customers
- Experience of CI/CD pipelines (preferred Jenkins, GitHub, GitHub Actions etc.) for automated build, test and
- deployment of code.
- Knowledge of Terraform ( good to have )