Role: AWS Data Infrastructure Engineer/AWS Data Platform Infrastructure Engineer
Full Time / FTE
Location: Alameda CA 94502 (Onsite)
Salary Market/Flexible
In the process of adopting Data Mesh concepts to build a data platform which can support federated data products with governance and security. Data Platform provides all the data infrastructure needs of data engineers in developing data products.
Key Responsibilities
Architect design and implement unified data access for various personas like data engineer data analyst data steward by simplified access control mechanism which supports governance on AWS and nonAWS assets Design build & optimize CI/CD pipelines for data engineers to deploy and push code across multiple environments Develop and maintain infrastructure as code (IaaC) with Terraform for reproducible and scalable deployments. Design & implementation of AWS Cloud infrastructure using best practices and industry standards by working closely with internal & external stakeholders like Information Security Cloud Infrastructure Data Engineering teams etc. Work closely with data governance teams in implementing data governance policies periodic review and optimization of repository from security & governance perspective Continuous improvement of Cloud Data platform to enhance speed agility and cost efficiency Skills & Experience
- Expertise in reducing costs and increasing speed and efficiency in large scale data platform deployments
- Proficiency in Terraform Python Scripting along with GitHub Actions for deployment of infrastructure as code (IaC)
- Deep understanding of data governance policies implementation best practices and handson experience in implementing Policies as Code
- Architecture design & implementation of unified data access using AWS Lake formation AWS Data Zone IAM OKTA and others
- Experience in architecture & security of AWS EC2 EBS S3 EKS Athena Redshift RDS Kafka Glue and other data management tools
- Experience in in design and implementation of granular access to data without losing the speed & agility of delivering the access provisioning
- Expertise in CI/CD process and deployment technologies using tools like GitHub GitHub Actions
- Expertise in logging monitoring reliability engineering
- Strong knowledge of networking concepts and experience managing networkrelated issues