Data Engineer

Job description & requirements

Job Description :

We are seeking a skilled Data Engineer to be a part of our dynamic team. As a Data Engineer you will play a crucial role in designing implementing and maintaining our data infrastructure. You will collaborate closely with data scientists analysts and other stakeholders to ensure efficient data processing and analytics.

Responsibilities :

Optimize Data Workflow : Identify and implement optimization techniques to improve data processing efficiency performance and reliability.

Data Modeling : Design and implement data models for efficient storage and retrieval of data ensuring data integrity and consistency.

Implement Data Security Measures : Implement security measures to safeguard sensitive data and ensure compliance with relevant regulations (e.g. GDPR HIPAA).

Collaborate with CrossFunctional Teams : Work closely with data scientists analysts and other stakeholders to understand their data requirements and provide technical solutions to meet their needs.

Troubleshooting and Debugging : Identify and resolve issues related to data quality performance and scalability in a timely manner.

Documentation : Document data pipelines workflows and processes to ensure knowledge sharing and maintainability.

Requirements :

Bachelor's Degree in Computer Science Engineering or related field.

35 years of experience as a Data Engineer or similar role.

Proficiency in Python for data processing and scripting.

Experience with Apache Spark or PySpark for distributed data processing.

Solid understanding of AWS services particularly AWS Glue S3 EC2 and EMR.

Experience with SQL and NoSQL databases. Strong problemsolving skills and attention to detail.

Excellent communication and collaboration skills. Ability to work effectively in a fastpaced dynamic environment.