QA AI Testing and Automation Engineer at Resilinc

Imagine being at the forefront of the artificial intelligence revolution, where your work directly shapes the reliability and ethics of the next generation of intelligent systems. We are a team dedicated to building robust AI agents and generative applications that people can truly trust. We believe that as AI becomes more integrated into our daily lives, the quality of these models is just as important as the innovation behind them. We are looking for a passionate engineer to join us in this mission, helping us bridge the gap between cutting-edge research and dependable, real-world software.

The opportunity

We are searching for a Senior QA AI Testing and Automation Engineer to lead the charge in validating our AI-driven systems. In this role, you will be the guardian of our model performance, ensuring that our AI agents and Large Language Models are not only accurate but also fair, robust, and explainable. You will work in a fully remote environment, collaborating with cross-functional teams to integrate rigorous testing standards into every stage of our development lifecycle. This is a unique chance to influence the quality of complex AI products while building sophisticated automation frameworks from the ground up.

A day in the life

• You will design and execute comprehensive test strategies for AI models, focusing on critical metrics like precision, recall, hallucination detection, and adversarial robustness.
• You will build and maintain high-quality automation scripts using Python, Selenium, and Playwright to ensure our web platforms and APIs remain stable and performant.
• You will collaborate closely with our data engineers to validate complex data pipelines and feature engineering processes, ensuring that the data fueling our models is accurate and reliable.

Who you are

You are a seasoned engineer who thrives when solving complex problems with minimal supervision. You possess a deep technical background and a keen eye for detail, and you are comfortable communicating technical findings to a variety of stakeholders. Your toolkit includes:

• Proven experience testing AI models, LLMs, and Generative AI applications, including familiarity with evaluation tools like Arize, MAIHEM, and LangTest.
• Strong proficiency in Python, with a focus on automating model validation and testing for bias and explainability.
• Expertise in SQL for backend validation, specifically with PostgreSQL and MySQL.
• A proactive mindset and a strong sense of ownership over the quality of the products you build.
• Fluency in English to effectively document processes and collaborate with our global team.

Why you'll love it here

We value the autonomy of our team members and believe that great work can happen from anywhere. By joining us, you will enjoy the flexibility of a remote work arrangement, allowing you to balance your professional and personal life effectively while working on some of the most exciting challenges in the field of artificial intelligence.