- Company Name
- Apexon
- Job Title
- AI QE Engineer
- Job Description
-
**Job Title**
AI QE Engineer
**Role Summary**
Design, develop, and maintain automated testing frameworks for large language models, generative AI workflows, and AI‑driven applications. Validate AI output quality, detect hallucinations, and ensure system reliability across functional, integration, API, and performance dimensions. Contribute to quality strategy for next‑generation AI features and collaborate across development, data science, and quality teams.
**Expectations**
- 4–10 years of experience as a software engineer, SDET, or automation engineer.
- Proven ability to write robust code in Python, TypeScript, or Java.
- Hands‑on experience building automation scripts, tools, or frameworks.
- Practical knowledge of LLMs, prompt engineering, and AI output evaluation.
- Exposure to agentic AI systems and tools such as LangGraph, AutoGen, or CrewAI.
- Understanding of Model Context Protocol or context‑aware workflow automation (desirable).
**Key Responsibilities**
1. Build and evolve automated test suites for functional, integration, API, and basic performance testing of AI models and services.
2. Validate LLM outputs: quality checks, hallucination detection, prompt validation, and regression tests for model updates.
3. Develop and maintain RAG pipelines, embedding similarity searches, and content evaluation metrics.
4. Create automation for RESTful APIs using PyTest, Selenium, Playwright, or other API testing libraries.
5. Work with AWS or equivalent cloud platforms to deploy AI or automation solutions.
6. Integrate CI/CD pipelines to include automated testing, model validation, versioning, and artifact management.
7. Collaborate with developers, data scientists, and QE peers to capture requirements, document defects, and participate in triage.
8. Communicate progress, risks, and results clearly to stakeholders.
**Required Skills**
- Programming: Python, TypeScript, Java (proficient).
- Automation: PyTest, Selenium, Playwright, API testing libraries.
- AI/ML: LangChain, Hugging Face, GPT models, vector databases, RAG pipelines.
- ML/DL frameworks: Scikit‑learn, PyTorch, TensorFlow, Keras, Transformers, OpenCV.
- AI‑specific: embeddings, similarity search, content evaluation, hallucination detection.
- DevOps: AWS, CI/CD (GitLab CI, GitHub Actions, Jenkins), version control, artifact repositories.
- SDLC: requirements analysis, defect tracking, documentation, stakeholder communication.
**Required Education & Certifications**
- Bachelor’s degree in Computer Science, Software Engineering, or related field (or equivalent experience).
- Certifications in QA/automation or AI/ML (e.g., ISTQB, AWS Certified DevOps Engineer, or relevant AI courses) are a plus.