QA Engineer – AI Testing and Automation

About the Role

Test and validate cutting-edge AI solutions — LLMs, RAG systems, and AI assistants — within the healthcare domain using automation and AI-specific testing methodologies.

Responsibilities

• Validate AI outputs for accuracy, relevance, consistency, hallucination, bias, safety, and business correctness
• Develop test datasets and automation scripts
• Perform functional, regression, and end-to-end testing
• Work with local LLM tools (Ollama, LM Studio, LocalAI)
• Collaborate cross-functionally with developers and product owners

Requirements

• Bachelor's degree in Computer Science or related field
• 3+ years hands-on testing experience
• 1+ year AI/ML/LLM product testing (preferred)
• ISTQB or AI testing certification (advantageous)

Skills

`Manual Testing` · `Automation Testing` · `API Testing` · `SQL` · `Selenium` · `Playwright` · `Cypress` · `REST Assured` · `PyTest` · `Java` · `Python` · `JavaScript` · `LLM Testing` · `Vector Databases`

How to apply

Or apply online: https://itpro.lk/job/14077/