Sr Data Engineer

Sr. Data Engineer
Location: Warren NJ (Onsite)
Duration: 12 months.
Position type: W2 contract
Visa: Any visa independent candidate.

Mandatory skills: Data Engineer Databricks SQL Python Spark Azure Data Factory Snowflake Amazon Redshift Google BigQuery Azure Synapse AWS GCP CI/CD pipelines P & C domain experience

Summary:

We are seeking a highly skilled and experienced Data Engineer to join our team. The ideal candidate will have a strong background in data processing and transformation using Databricks proficiency in SQL and Python experience with Azure Data Factory and a solid understanding of data warehouse migration strategies.

Responsibilities:

• Design develop and maintain scalable data pipelines using Databricks and integrate with Azure Data Factory (ADF) workflows.
• Develop and optimize SQL queries and data models for performance and scalability.
• Lead data warehouse migration projects including planning execution and validation of data transfer from legacy systems to modern platforms.
• Familiarity with cloud data warehouses such as Snowflake Amazon Redshift Google BigQuery or Azure Synapse.
• Collaborate with crossfunctional teams to gather requirements define data architecture and implement solutions that adhere to business needs and data governance standards.
• Write complex data transformation logic using Python and Spark.
• P & C domain experience preffered
• Monitor and ensure the stability of data processing jobs troubleshoot issues and perform root cause analysis.
• Implement data quality checks data cleansing procedures and ensure data integrity.
• Document all data engineering processes and create standard operating procedures.
• Stay current with industry trends and best practices in data engineering and contribute to continuous improvement of the data platform.

Qualifications:

Bachelors degree in Computer Science Engineering or a related field or equivalent work experience.
Minimum of 3 years of experience in a data engineering role with a focus on Databricks data pipelines and data warehouse technologies.
Proven experience with SQL Python and Spark.
Handson experience with Azure Data Factory and familiarity with other Azure services.
Experience with data warehouse migration including schema conversion data replication and ETL processes.
Strong understanding of data modeling data warehousing concepts and big data architectures.
Excellent problemsolving skills and the ability to work independently or as part of a team.
Strong communication skills and the ability to collaborate effectively with both technical and nontechnical stakeholders.

Preferred Skills:

Certifications in Databricks and Azure Data Factory.
Experience with additional cloud platforms (e.g. AWS GCP) and data integration tools.
Knowledge of data governance security and compliance best practices.
Experience with CI/CD pipelines and version control systems like Git.