QA Engineer - AI Testing and Automation

About the Role

Test and validate AI-driven products including LLMs, RAG systems, and chatbots for accuracy, consistency, hallucination, and bias.

Responsibilities

• Test AI-driven products including LLMs, RAG systems, AI assistants, chatbots
• Validate accuracy, relevance, consistency, hallucination, and bias in AI outputs
• Create test datasets and automation scripts
• Perform functional, regression, integration, and end-to-end testing

Requirements

• Bachelor's degree in Computer Science or IT
• 3+ years manual and automation testing experience
• 1 year AI/ML/LLM/chatbot testing experience (preferred)
• Knowledge of RAG systems and embeddings

Skills

`Selenium` · `Playwright` · `Cypress` · `REST Assured` · `PyTest` · `Java` · `Python` · `JavaScript` · `API Testing` · `SQL` · `AI/ML/LLM Testing` · `Vector Databases`

How to apply

Or apply online: https://itpro.lk/job/14077/