Azure Data Engineer
Posted on :
23-08-2024
Employer Active
1 Vacancy
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Share
Send me jobs like this
Job Alert
You will be updated with latest job alerts via emailValid email field required
Send jobs
Job Description
Vidpro Consultancy Services is seeking a Senior Databricks Engineer / Databricks Technical Lead / Data Architect with 510 years of experience.
Location: Bangalore Chennai Delhi Pune Kolkata and Gurgaon
Work Type: Hybrid
Mandatory Skills: Python/ PySpark / Spark with Azure/ Azure Databricks / Azure DataFactory
Primary Roles and Responsibilities:
Creating Modern Data Warehouse solutions using Databricks and AWS/Azure Stack
Providing forwardthinking solutions in data engineering and analytics
Collaborating with DW/BI leads for new ETL pipeline development requirements
Troubleshooting issues in existing pipelines and addressing gaps
Understanding reporting layer needs and developing data models
Assisting team members with problem resolution and technical challenges
Leading technical discussions with client architects and team members
Orchestrating data pipelines in a scheduler via Airflow
Skills and Qualifications:
Bachelors and/or master s degree in computer science or equivalent experience
Minimum 3 years of experience in Data warehouse/ETL projects
Deep understanding of Star and Snowflake dimensional modelling
Strong knowledge of Data Management principles
Proficiency in Databricks Data & AI platform and Databricks Delta Lake Architecture
Handson experience in SQL Python and Spark (PySpark)
Experience with AWS/Azure stack and ETL with batch and streaming (Kinesis)
Building ETL/data warehouse transformation processes
Familiarity with Apache Kafka for use with streaming data/eventbased data
Working knowledge of Hadoop (incl. Hive Pig Impala) and nonrelational/NoSQL data repositories (incl. MongoDB Cassandra Neo4J)
Experience with structured and unstructured data including imaging & geospatial data
Proficiency in RDBMS complex SQL PL/SQL Unix Shell Scripting performance tuning and troubleshooting
Databricks Certified Data Engineer Associate/Professional Certification (Desirable)
Comfortable working in a dynamic fastpaced innovative environment with several ongoing concurrent projects
Experience working in Agile methodology
Strong verbal and written communication skills
Strong analytical and problemsolving skills with a high attention to detail
circleci,pl/sql,python,azure datafactory,microsoft azure,sql,terraform,aws,cassandra,hadoop,extract, transform, load (etl),azure,hive,agile methodology,pipelines,rdbms,mongodb,data management,git,azure databricks,apache kafka,unix shell scripting,neo4j,data warehouse,databricks,databricks delta lake architecture,pyspark,etl,impala,databricks data & ai platform,pig,spark
Employment Type
Full Time
Key Skills
Report This Job
Disclaimer: Drjobpro.com is only a platform that connects job seekers and employers. Applicants are advised to conduct their own independent research into the credentials of the prospective employer.We always make certain that our clients do not endorse any request for money payments, thus we advise against sharing any personal or bank-related information with any third party. If you suspect fraud or malpractice, please contact us via contact us page.
