ELEKS Quality Assurance Office is looking for a Senior Automation Test Engineer (AI/LLM) in Argentina

ABOUT PROJECT

Our customer is among the top 5 health insurance companies in the US. They are dedicated to expanding access to high-quality, cost-effective healthcare.

We are looking for an experienced Automation Test Engineer to join the project engagement and work embedded within the client’s delivery team, focusing on testing AI-powered applications, including Large Language Models (LLMs) and Retrieval-Augmented Generation (RAG) solutions.

REQUIREMENTS

Experience testing LLM/RAG behavior, including intent routing, retrieval quality, prompt safety, and evaluation/regression testing
Experience in test automation using Python and PyTest
Experience testing REST APIs and FastAPI-based applications
Experience with SQL and Databricks data validation
Experience testing Azure AI Search
Experience testing Azure OpenAI APIs
Experience with LLM intent-routing validation
Experience with RAG retrieval-quality testing
Experience with prompt-injection and PII-leakage testing
Experience with groundedness, hallucination, and citation validation
Experience with RAGAS or DeepEval evaluation
Experience with automated regression testing using JSONL datasets
Experience with Postman and Newman
Experience with Jenkins CI/CD integration
Experience with Docker and Git
Experience with Application Insights or OpenTelemetry monitoring
At least Upper-Intermediate level of English

RESPONSIBILITIES

Develop, execute, and maintain automated tests using Python and PyTest
Perform REST API and FastAPI testing
Validate data using SQL and Databricks
Test Azure AI Search and Azure OpenAI APIs
Validate LLM intent routing and RAG retrieval quality
Perform prompt-injection and PII-leakage testing
Validate groundedness, hallucinations, and citations in LLM-generated responses
Conduct LLM evaluations using RAGAS or DeepEval
Perform automated regression testing using JSONL datasets
Use Postman and Newman for API testing
Integrate automated tests with Jenkins CI/CD
Use Docker and Git in test automation activities
Monitor applications using Application Insights or OpenTelemetry

WHAT YOU WILL GET WITH ELEKS

Close cooperation with a customer
Challenging tasks
Competence development
Ability to influence project technologies
Team of professionals
Dynamic environment with low level of bureaucracy

ABOUT ELEKS

ELEKS is a custom software development company. We deliver value to our clients, thanks to our expertise and experience gained from working as a software innovation partner since 1991.

Our 2000+ professionals located in the Delivery Centers across Eastern Europe and sales offices in Europe and North America, provide our clients with a full range of software engineering services. These include product development, QA, R&D, design, technology consulting and dedicated teams.