QA AI Testing and Automation Engineer at Resilinc

Resilinc logo
Resilinc

QA AI Testing and Automation Engineer

Remote

Full Time

#Engineering

#AI

#Automation

#Python

#SQL

#Selenium

#Playwright

#Testing

Resilinc is looking for a QA AI Testing and Automation Engineer

Sign up to unlock quick summaries and profile fit assessments

We are seeking a highly skilled Quality Assurance Engineer with expertise in testing AI models, Large Language Models (LLMs), AI agents, and Generative AI applications. The ideal candidate will have experience in validating AI-driven systems, ensuring model accuracy, performance, fairness, explainability, and robustness. Additionally, the candidate will be responsible for developing automation tests alongside their AI testing responsibilities.
The successful candidate will be self-motivated and proactive, demonstrating a strong sense of ownership and accountability while requiring minimal supervision.

What You Will Do
  • Develop and implement QA strategies for AI-powered applications, focusing on accuracy, bias, fairness, robustness, and performance. 
  • Design and execute automated and manual test cases to validate AI Agents/LLM models, APIs, and data pipelines and good understanding of data integrity, data models, etc 
  • Assess AI models using quality metrics such as precision/recall, and hallucination detection. 
  • Test AI models for bias, fairness, explainability (XAI), drift, and adversarial robustness. 
  • Validate prompt engineering, fine-tuning techniques, and model-generated responses for accuracy and ethical AI considerations. 
  • Conduct scalability, latency, and performance testing for AI-driven applications. 
  • Collaborate with data engineers to validate data pipelines, feature engineering processes, and model outputs. 
  • Design, develop, and maintain automation scripts using Selenium and Playwright for API and web testing 
  • Work closely with cross-functional teams to integrate automation best practices into the development lifecycle. 
  • Identify, document, and track bugs while conducting detailed regression testing to ensure product quality. 


  • What You Will Bring
  • Proven expertise in testing AI models, LLMs, and Generative AI applications, with hands-on experience in AI evaluation metrics and testing tools like Arize, MAIHEM, and LangTest
  • Strong proficiency in Python for writing test scripts and automating model validation, along with a deep understanding of AI bias detection, adversarial testing, model explainability (XAI), and AI robustness
  • Demonstrate strong SQL expertise for validating data integrity and backend processes, particularly in PostgreSQL and MySQL. 
  • Strong analytical and problem-solving skills with keen attention to detail, along with excellent communication and documentation abilities to convey complex testing processes and results. 
  • Resilinc logo

    Resilinc

    16 views

    2 applied

    Social Media

    Visit Resilinc
    Share this job
    Copy Permalink
    Open roles at Resilinc
    Resilinc logo
    Resilinc

    Director Senior Manager Engineering

    Remote

    Full Time

    #Engineering

    #Engineering Leadership

    #Python

    #Cloud Platforms

    #Microservices Architecture

    #Data Engineering

    #AI

    #Enterprise Software

    #Agile Methodologies

    #Kubernetes

    Resilinc logo
    Resilinc

    AI Software Development Engineer in Test

    Remote

    Full Time

    #Supply Chain

    #Artificial Intelligence

    #Engineering

    #AI

    #Python

    #SQL

    #Selenium

    #Playwright

    #Testing

    #Performance Testing

    #Data Validation

    Resilinc logo
    Resilinc

    Lead Senior FullStack Engineer

    Remote

    Full Time

    #Engineering

    #Java

    #Python

    #Angular

    #API Design

    #Azure

    #DevOps

    #CI CD

    #Machine Learning

    #Microservices

    Resilinc logo
    Resilinc

    Senior Manager - Product Marketing

    Remote

    Full Time

    #Marketing

    #SaaS

    #Supply Chain

    #Product Marketing

    #Market Analysis

    #Sales Enablement

    #Go To Market Strategy

    #Content Strategy

    Resilinc logo
    Resilinc

    Senior DevOps Engineer

    Remote

    Full Time

    #Engineering

    #Terraform

    #Azure Cloud

    #GitHub Actions

    #Infrastructure as Code

    Discover similar jobs
    N
    NewPageSolutionsInc

    Python Developer

    Remote

    Contractor

    #Technology

    #Digital Health

    #Software Development

    #Python

    #AWS Lambda

    #AWS ECS

    #Automated Testing

    #Agile Methodologies

    #Terraform

    #Drupal

    #PHP

    T
    Teach For All

    Head of AI Solutions & Engineering

    Remote

    Contractor

    #AI

    #Education

    #Technology

    #TypeScript

    #Python

    #REST APIs

    #Git

    #Design

    #Google Cloud

    #Business Analysis

    InterNations logo
    InterNations

    Online Marketing Intern

    Remote

    Internship

    #Technology

    #SEO

    #Online Marketing

    #YouTube

    #Google Analytics

    #WordPress

    #AI

    Q
    Quora

    Staff Machine Learning Engineer

    220k - 321k USD

    Remote

    Full Time

    #Machine Learning

    #Recommendation Systems

    #Engineering

    #Python

    #C++

    #Data Pipelines

    #Model Training

    #Algorithms

    Ramp logo
    Ramp

    Security Engineer, Cloud

    Remote

    Full Time

    #Cloud Security

    #Security Engineering

    #Fintech

    #AWS

    #Terraform

    #Python

    #Flask

    #Infrastructure

    #DevOps

    DroneDeploy logo
    DroneDeploy

    Senior DevOps Engineer

    Remote

    Full Time

    #DevOps

    #Cloud Infrastructure

    #MLOps

    #Kubernetes

    #Terraform

    #Python

    #Golang

    #AWS

    #Linux

    #Observability

    #GitHub Actions

    Civis Analytics logo
    Civis Analytics

    Chief Technology Officer

    225k - 325k USD

    Remote

    Full Time

    #AI

    #Data Platform

    #Technical Leadership

    #AI Systems

    #Data Engineering

    #Cloud Platforms

    #Distributed Systems

    #Technical Architecture

    #Team Leadership

    #Product

    Palantir logo
    Palantir

    Tech Fellowship - Frontier

    Remote

    Other

    #AI

    #Data Analysis

    #Foundry

    #AI Tools

    #Workflow Automation

    #Problem Solving

    #Communication

    Sevaa Group logo
    Sevaa Group

    Senior Drupal Developer

    Remote

    Contractor

    #Drupal

    #Engineering

    #DevOps

    #PHP

    #Testing

    #Responsive Design

    #Security

    H
    HeyJobs

    Graphic Design / Creative Technology

    29k - 29k USD

    Remote

    Internship

    #Technology

    #AI Tools

    #Digital Marketing

    #Engineering

    #Prototyping

    #Automation

    #JavaScript

    #Python

    GoodRec logo
    GoodRec

    Head of Growth

    Remote

    Full Time

    #Growth

    #Data Analysis

    #Testing

    #SQL

    #Customer Research

    #Project Management

    #Google Analytics

    #Amplitude

    #User Journeys

    #Market Research

    N
    Nomic Foundation

    Senior Engineering Manager, Dev Tools

    Remote

    Full Time

    #Ethereum

    #Engineering

    #People Management

    #Technical Leadership

    #Code Reviews

    #Roadmap Planning

    #Rust

    #TypeScript

    #Solidity

    #Software Architecture

    Backstory logo
    Backstory

    Director of IT

    Remote

    Full Time

    #IT Operations

    #Team Leadership

    #Infrastructure

    #MacOS Management

    #SaaS Administration

    #Systems Engineering

    #Automation

    #Tooling

    #Architecture

    #Vendor Management

    A
    Able

    Data Engineer

    Remote

    Full Time

    #Data Engineering

    #AI

    #Software Development

    #SQL

    #Python

    #Data Warehouses

    #Airflow

    #Data Modeling

    #BI Tools

    #Data Governance

    Prosper logo
    Prosper

    Infrastructure Security Engineer

    Remote

    Full Time

    #Fintech

    #Cloud Security

    #Information Security

    #GCP

    #Azure

    #Terraform

    #Wiz

    #SIEM

    #Python

    Agerpoint logo
    Agerpoint

    General Interest

    Remote

    Full Time

    #Business Development

    #AI

    #Artificial Intelligence

    #Data

    #Computer Vision

    #Cloud Computing

    #Analytics

    #Machine Learning

    #Remote Sensing

    Relay Commerce logo
    Relay Commerce

    Full-stack Software Engineer

    Remote

    Full Time

    #Subscriptions

    #Engineering

    #TypeScript

    #Node.Js

    #React

    #PostgreSQL

    #REST APIs

    #Docker

    #Express

    #TypeORM

    #Shopify

    C
    Character.AI

    Staff Data Scientist

    Remote

    Full Time

    #Data Science

    #Monetization

    #AI

    #Experimentation

    #Metrics

    #Analytics

    #Subscriptions

    #Advertising

    #Product

    MyTime logo
    MyTime

    Senior Angular Frontend Developer

    Remote

    Full Time

    #Engineering

    #Frontend Development

    #Angular

    #React

    #Unit Testing

    #UI Development

    #Performance Optimization

    #Scalability

    #Code Review

    L
    Lightning Labs

    Assets Protocol Engineer

    Remote

    Full Time

    #Bitcoin

    #Cryptocurrency

    #Engineering

    #Go

    #Distributed Systems

    #Networks

    #Cryptography

    #Open Source

    Your dream job awaits.

    Explore exciting opportunities, connect with top employers, and ignite your career.