Python Software Engineer

Job Title: Python Software Engineer
Job Location: Dallas TX(Remote)
Job Duration: 100% Remote
Interview: Virtual

We are seeking a skilled Software Engineer with expertise in AWS Glue and Apache Hudi to join our team. The ideal candidate will have strong proficiency in Python and handson experience with Change Data Capture (CDC) methodologies.

Responsibilities:

• Develop and implement ETL processes using AWS Glue to extract transform and load data from various sources into data lakes and data warehouses.
• Design and optimize data pipelines to ensure efficient data ingestion processing and integration.
• Utilize Apache Hudi for incremental data processing and realtime data updates.
• Implement Change Data Capture (CDC) strategies to capture and propagate data changes across systems in near realtime.
• Collaborate closely with data engineers data scientists and business stakeholders to understand data requirements and deliver scalable solutions.
• Ensure data quality and integrity throughout the ETL process and provide troubleshooting and support as needed.
• Document technical specifications processes and procedures related to ETL development and deployment.

Required Skills:

• Extensive experience (Senior level) in ETL development using AWS Glue.
• Strong proficiency in Python programming for data processing and automation tasks.
• Intermediatelevel proficiency in Apache Hudi with handson experience in implementing data ingestion and processing workflows.
• Solid understanding and practical experience with Change Data Capture (CDC) methodologies.
• Experience working in cloud environments particularly AWS.
• Ability to work independently and as part of a collaborative team.
• Excellent problemsolving skills and attention to detail.
• Effective communication skills to interact with technical and nontechnical stakeholders.