Lead Data Engineer - E-Commerce Technologies

This is a remote position.

• Architect and scale a stateoftheart data infrastructure capable of handling batch and realtime data processing needs with unparalleled performance.
  
• Collaborate closely with the data science team to oversee data systems ensuring accurate monitoring and insightful analysis of business processes.
  
• Design and implement robust ETL (Extract Transform Load) data pipelines optimizing data flow and accessibility.
  
• Develop comprehensive backend data solutions to bolster microservices architecture ensuring seamless data integration and management.
  
• Engineer and manage integrations with thirdparty ecommerce platforms expanding data ecosystem and capabilities.
  

Requirements

• A background at a productcentric software company directly contributing to building products (vs providing data/analytics to business stakeholders)
  
• Software development experience with a focus on data engineering.
  
• Extensive experience in building ETL pipelines using tools such as Apache Spark Databricks or Hadoop.
  
• Proficiency in Python or Java with a deep understanding of software engineering best practices.
  
• Expertise in distributed computing and data modeling capable of designing scalable data systems.
  
• Proficiency with NoSQL databases including MongoDB Cassandra DynamoDB and CosmosDB.
  
• Proficiency in realtime stream processing systems such as Kafka AWS Kinesis or GCP Data Flow.
  
• Skilled in utilizing caching and search technologies like Redis Elasticsearch or Solr.
  
• Familiarity with message queuing systems including RabbitMQ AWS SQS or GCP Cloud Tasks.
  
• Experience with Delta Lake Parquet files and AWS GCP or Azure cloud services.
  
• A strong advocate for Test Driven Development (TDD) and experienced in version control using Git platforms like GitHub or Bitbucket.
  
• Experience at a startup is preferred.
  
• Experience with consumer ecommerce data/technologies would be a bonus.
  

Benefits

• Work Location: Remote
  
• 5 days working