Resilinc

QA AI Testing and Automation Engineer

Remote

Full Time

#Engineering

#AI

#Automation

#Python

#SQL

#Selenium

#Playwright

#Testing

Resilinc is looking for a QA AI Testing and Automation Engineer

Full Time Senior Remote Engineering AI Automation Python SQL Selenium Playwright Testing English Remote work

We are seeking a highly skilled Quality Assurance Engineer with expertise in testing AI models, Large Language Models (LLMs), AI agents, and Generative AI applications. The ideal candidate will have experience in validating AI-driven systems, ensuring model accuracy, performance, fairness, explainability, and robustness. Additionally, the candidate will be responsible for developing automation tests alongside their AI testing responsibilities.

The successful candidate will be self-motivated and proactive, demonstrating a strong sense of ownership and accountability while requiring minimal supervision.

What You Will Do

Develop and implement QA strategies for AI-powered applications, focusing on accuracy, bias, fairness, robustness, and performance.

Design and execute automated and manual test cases to validate AI Agents/LLM models, APIs, and data pipelines and good understanding of data integrity, data models, etc

Assess AI models using quality metrics such as precision/recall, and hallucination detection.

Test AI models for bias, fairness, explainability (XAI), drift, and adversarial robustness.

Validate prompt engineering, fine-tuning techniques, and model-generated responses for accuracy and ethical AI considerations.

Conduct scalability, latency, and performance testing for AI-driven applications.

Collaborate with data engineers to validate data pipelines, feature engineering processes, and model outputs.

Design, develop, and maintain automation scripts using Selenium and Playwright for API and web testing

Work closely with cross-functional teams to integrate automation best practices into the development lifecycle.

Identify, document, and track bugs while conducting detailed regression testing to ensure product quality.

What You Will Bring

Proven expertise in testing AI models, LLMs, and Generative AI applications, with hands-on experience in AI evaluation metrics and testing tools like Arize, MAIHEM, and LangTest.

Strong proficiency in Python for writing test scripts and automating model validation, along with a deep understanding of AI bias detection, adversarial testing, model explainability (XAI), and AI robustness.

Demonstrate strong SQL expertise for validating data integrity and backend processes, particularly in PostgreSQL and MySQL.

Strong analytical and problem-solving skills with keen attention to detail, along with excellent communication and documentation abilities to convey complex testing processes and results.

Resilinc

16 views

2 applied

Social Media

Visit Resilinc

Share this job

Copy Permalink

Open roles at Resilinc

Resilinc

Director Senior Manager Engineering

Remote

Full Time

#Engineering

#Engineering Leadership

#Python

#Cloud Platforms

#Microservices Architecture

#Data Engineering

#AI

#Enterprise Software

#Agile Methodologies

#Kubernetes

Resilinc

AI Software Development Engineer in Test

Remote

Full Time

#Supply Chain

#Artificial Intelligence

#Engineering

#AI

#Python

#SQL

#Selenium

#Playwright

#Testing

#Performance Testing

#Data Validation

Resilinc

Lead Senior FullStack Engineer

Remote

Full Time

#Engineering

#Java

#Python

#Angular

#API Design

#Azure

#DevOps

#CI CD

#Machine Learning

#Microservices

Resilinc

Senior Manager - Product Marketing

Remote

Full Time

#Marketing

#SaaS

#Supply Chain

#Product Marketing

#Market Analysis

#Sales Enablement

#Go To Market Strategy

#Content Strategy

Resilinc

Senior DevOps Engineer

Remote

Full Time

#Engineering

#Terraform

#Azure Cloud

#GitHub Actions

#Infrastructure as Code

Discover similar jobs

LeanData

GTM Engineer (Data & AI)

Remote

Full Time

#Sales

#Revenue Operations

#Data Engineering

#AI

#Workflow Automation

#Data Warehousing

#API Integration

#Orchestration

Truv

Lead Product Manager, Consumer Product

Remote

Full Time

#Product

#Fintech

#Product Management

#User Testing

#Testing

#Data Analysis

#Stakeholder Management

#Compliance

#Technology

Dijital-team-pty-ltd

Automation Engineer

Remote

Full Time

#IT

#Managed Services

#PowerShell

#Python

#Jinja

#BASH

#REST API

#JSON

#Git

#JavaScript

Emi Labs

Back-end Engineer

Argentina

Remote

Full Time

#Backend Engineering

#AI Tools

#Systems

#Node.Js

#Backend Development

#Architecture

#AI

#LLM

#Testing

#Monitoring

#Workflow Automation

Wrapbook

Senior Product Manager

Remote

Full Time

#Product Management

#Fintech

#SaaS

#Product Strategy

#Roadmap Planning

#Collaboration

#Data Analysis

#SQL

#AI Tools

#Customer Insights

#Technical Systems

Planner 5D

AI Moderator

Remote

Contractor

#Product

#Support

#AI

#Attention To Detail

#Research

#English

Candidly

Senior Infrastructure Engineer

Remote

Full Time

#Infrastructure Engineering

#Cloud Computing

#DevOps

#AWS

#Azure

#Kubernetes

#Docker

#IaC

#Python

#Linux

#Monitoring

#Security

PelotonInc

Senior Software Engineer

Remote

Full Time

#Engineering

#Full Stack

#DevOps

#Docker

#Kubernetes

#AWS

#GCP

#Azure

#Flux

#Rancher

#Continuous Delivery

#Infrastructure

#Microservices

Ivanti

Associate Site Reliability Engineer

Remote

Full Time

#Site Reliability

#Cloud Operations

#DevOps

#Linux

#Windows

#Networking

#Kubernetes

#Docker

#Python

#Java

#AWS

#Azure

#Ansible

BioIntelliSense

DevOps Engineer

Remote

Full Time

#Cloud

#DevOps

#Healthcare

#Terraform

#AWS

#Datadog

#Bitbucket Pipelines

#CircleCi

#Databricks

#Python

#Flutter

Hyperhug

QA Engineer

Remote

Full Time

#Game Development

#QA Testing

#Mobile

#Manual Testing

#TestRail

#Jira

#Android Studio

#XCode

#Unity

#Git

#Firebase

#Python

#C#

Tarmac Technologies

Python Django Backend Engineer

Remote

Full Time

#Technology

#Backend Development

#Tech

#Python

#Django

#RESTful API

#AWS

#Backend Engineering

HeyJobs

Graphic Design Creative Technology

Remote

Part Time

#Technology

#AI Tools

#Digital Marketing

#Engineering

#JavaScript

#Python

#Landing Pages

#Content

Inspiren

Director of Product Management

230k - 270k USD

Remote

Full Time

#Product Management

#Health Tech

#AI

#Hardware

#Machine Learning

#Product Strategy

#Collaboration

#Data Analysis

#Discovery

Niche

Senior Designer, Design Systems

AR, BR

Remote

Full Time

#Design Systems

#Product Design

#Figma

#Design

#Storybook

#AI

#Accessibility

#Responsive Design

ProktaHRSolutions

Senior Software Engineer - Network Services Orchestration

India

Remote

Full Time

#Automation

#Orchestration

#Technology

#Cisco

#Python

#Java

#Linux

#DevOps

Northflank.com

Backend Software Engineer

57k - 127k USD

Remote

Full Time

#Backend Engineering

#Cloud

#Microservices

#Go

#Python

#Node.Js

#SQL

#NoSQL

#RESTful APIs

#Docker

#Kubernetes

#AWS

NewPageSolutionsInc

Python Developer

Remote

Contractor

#Technology

#Digital Health

#Software Development

#Python

#AWS Lambda

#AWS ECS

#Automated Testing

#Agile Methodologies

#Terraform

#Drupal

#PHP

Teach For All

Head of AI Solutions & Engineering

Remote

Contractor

#AI

#Education

#Technology

#TypeScript

#Python

#REST APIs

#Git

#Design

#Google Cloud

#Business Analysis

InterNations

Online Marketing Intern

Remote

Internship

#Technology

#SEO

#Online Marketing

#YouTube

#Google Analytics

#WordPress

#AI

Your dream job awaits.

Explore exciting opportunities, connect with top employers, and ignite your career.

Explore Jobs

Related Resources

Engineering Salary Guide

Compensation data for Engineering roles

Engineering Job Market

Hiring trends and demand for Engineering

Engineering Interview Prep

Practice questions for Engineering interviews

AI Salary Guide

Compensation data for AI roles

AI Job Market

Hiring trends and demand for AI

AI Interview Prep

Practice questions for AI interviews

Automation Salary Guide

Compensation data for Automation roles

Automation Job Market

Hiring trends and demand for Automation

QA AI Testing and Automation Engineer at Resilinc