Job Role : ETL/Spark Developer
Location : Mclean VA Day 1 onsite in hybrid set up
Job Description:
- Your expertise in Python PySpark ETL processes CI/CD (Jenkins or GitHub) and experience with both streaming and batch workflows will be essential in ensuring the efficient flow and processing of data to support our clients
- Collaborate with crossfunctional teams to understand data requirements and design robust data architecture solutions
- Implement ETL processes to extract transform and load data from various sources
- Ensure data quality integrity and consistency throughout the ETL pipeline
- Utilize your expertise in Python and PySpark to develop efficient data processing and analysis scripts
- Optimize code for performance and scalability keeping uptodate with the latest industry best practices
- Integrate data from different systems and sources to provide a unified view for analytical purposes
- Collaborate with data analysts to implement solutions that meet their data integration needs
- Design and implement streaming workflows using PySpark Streaming or other relevant technologies
- Develop batch processing workflows for largescale data processing and analysis.
Python,PySpark,ETL