Machine Learning QA Engineer

Job Duties/Responsibilities

• Verify the quality of input data, ensuring it is clean, relevant, and representative.
• Define appropriate evaluation metrics based on the specific problem domain & align with the project objectives and stakeholder requirements.
• Implement robust cross-validation techniques to assess model performance and compare model output with expected results.
• Evaluate the model for biases that may lead to unfair outcomes for certain groups.
• Foster collaboration between data scientists, engineers, domain experts, and stakeholders throughout the QA process.
• Communicate findings, insights, and potential limitations of the ML system effectively to the team.
• Gather project schedules, software functional and non-functional requirements from our project teams, and turn them into test plans, test designs, and test cases
• Work closely with stakeholders to setup QA processes and establish best practices
• Create and execute manual & automated tests
• Perform regression testing to ensure no bugs are introduced in new builds
• Estimate, plan and prioritize test activities
• Track quality assurance metrics
• Write test reports for project stakeholders
• Be an integral member of the team that follows the Agile software development lifecycle process
• Debug, troubleshoot, and improve live cloud-based applications
  

Minimum Qualifications

• 8+ years of QA experience.
• Experience in AI (ML/DS) models testing.
• BS in Computer Science or a related degree
• Experience writing test plans, cases, and designs from scratch for large scale distributed applications, web services or RESTful APIs
• Experience creating, executing and reviewing results of functional, integration, and regression testing
• Experience with performance, security and/or stress.
• Ability to design and implement test automation & unit testing frameworks using Cypress, Selenium, Cucumber or Robot Framework.
• Experience working with JavaScript, Java, Python or a similar programming language
  

Preferred Qualifications

• MS In Computer Science or Related Field
• Comfortable using a variety of tools & technologies to investigate and resolve issues (e.g. SQL, RabbitMQ, Elastic Search, Postgres, Postman, JMeter)
• Knowledge of Cloud Technologies and Distributed Systems.
• Experience with AWS/Azure, Docker and container technologies is a solid plus
• Experience with continuous integration/continuous deployment operations and toolsets (e.g. Jenkins).