Job VC
Senior QA Engineer ID66279
Technologies
Description
Hi there! AgileEngine is an Inc. 5000 company that creates award-winning software for Fortune 500 brands and trailblazing startups across 17+ industries. We rank among the leaders in areas like application development and AI/ML, and our people-first culture has earned us multiple Best Place to Work awards.
Why join us
If you’re looking for a place to grow, make an impact, and work with people who care, we’d love to meet you! :)
About the role
We are looking for a
Senior QA Engineer
to perform manual testing of web applications and validate AI-generated outputs as part of a product team building
LLM-integrated
features. You will design and execute test cases across UI, API, and AI behavior scenarios using TestRail, Postman, and JIRA, working within an Agile environment. The role requires a detail-oriented engineer who can evaluate model quality metrics and ensure accuracy, completeness, and alignment across prompt—response workflows.
What you will do
● Perform manual testing of web applications to identify bugs, glitches, and inconsistencies
● Develop, maintain, and execute detailed manual test cases based on product requirements and specifications
● Perform thorough cross-browser UI testing to ensure consistent behavior across multiple browsers
● Validate AI-generated test cases, summaries, and recommendations produced by integrated LLMs
● Design and execute test scenarios to verify AI behavior, including prompt-response consistency and contextual relevance
● Evaluate AI model quality using defined acceptance metrics such as precision, recall, hallucination rate, and duplication rate
● Conduct manual API testing to validate request and response accuracy, error handling, and integration
● Use TestRail to create, manage, and track test plans, test cases, and test execution results
● Use Jenkins CI/CD pipelines
● Report and document defects clearly using JIRA
● Participate in daily stand-ups, sprint planning, and other Agile team meetings
● Review user stories and acceptance criteria to understand the scope of testing
● Estimate own tasks and report risks to the team lead
● Collaborate with developers and other QA team members to reproduce issues and verify fixes
● Maintain and update test documentation as needed
Must haves
●
4+ years of experience
with manual testing
● Basic understanding of methodologies and SDLC and STLC models
● Knowledge of testing and quality assurance processes
● Strong experience with test design activities to cover a set of product requirements
● Knowledge of test design techniques and testing methodologies
● Defect analysis skills and the ability to write detailed and useful defect reports
● Experience with
Postman
for API testing and browser developer tools
● Experience with UI testing
● Experience with
TestRail
as a test management system
● Basic understanding of CI/CD processes
● Intermediate English level
Nice to haves
● Understanding of LLM evaluation principles, including prompt engineering, output validation, and model fine-tuning cycles
● Familiarity with AI and ML concepts, APIs, and frameworks such as
OpenAI API
,
Azure OpenAI
, or
LangChain
● Knowledge of data protection and compliance standards for AI systems, including GDPR, zero-data retention, and PII anonymization
● Experience with SSO testing
● Experience in performance or load testing
Perks and benefits
●
Professional growth: Accelerate your professional journey with mentorship, TechTalks, and personalized growth roadmaps
●
Competitive compensation: We match your ever-growing skills, talent, and contributions with competitive USD-based compensation and budgets for education, fitness, and team activities
●
A selection of exciting projects: Join projects with modern solutions development and top-tier clients that include Fortune 500 enterprises and leading product brands
●
Flextime: Tailor your schedule for an optimal work-life balance, by having the options of working from home and going to the office — whatever makes you the happiest and most productive.
Meet Our Recruitment Process
Asynchronous stage
— An automated, self-paced track that helps us move faster and give you quicker feedback:
● Short online form to confirm basic requirements
●
30–60
minute skills assessment
●
5-minute
introduction video
Synchronous stage
— Live interviews
● Technical interview with our engineering team (scheduled at your convenience)
● Final interview with your future teammates
If it’s a match—you’ll get an offer!
Why join us
If you’re looking for a place to grow, make an impact, and work with people who care, we’d love to meet you! :)
About the role
We are looking for a
Senior QA Engineer
to perform manual testing of web applications and validate AI-generated outputs as part of a product team building
LLM-integrated
features. You will design and execute test cases across UI, API, and AI behavior scenarios using TestRail, Postman, and JIRA, working within an Agile environment. The role requires a detail-oriented engineer who can evaluate model quality metrics and ensure accuracy, completeness, and alignment across prompt—response workflows.
What you will do
● Perform manual testing of web applications to identify bugs, glitches, and inconsistencies
● Develop, maintain, and execute detailed manual test cases based on product requirements and specifications
● Perform thorough cross-browser UI testing to ensure consistent behavior across multiple browsers
● Validate AI-generated test cases, summaries, and recommendations produced by integrated LLMs
● Design and execute test scenarios to verify AI behavior, including prompt-response consistency and contextual relevance
● Evaluate AI model quality using defined acceptance metrics such as precision, recall, hallucination rate, and duplication rate
● Conduct manual API testing to validate request and response accuracy, error handling, and integration
● Use TestRail to create, manage, and track test plans, test cases, and test execution results
● Use Jenkins CI/CD pipelines
● Report and document defects clearly using JIRA
● Participate in daily stand-ups, sprint planning, and other Agile team meetings
● Review user stories and acceptance criteria to understand the scope of testing
● Estimate own tasks and report risks to the team lead
● Collaborate with developers and other QA team members to reproduce issues and verify fixes
● Maintain and update test documentation as needed
Must haves
●
4+ years of experience
with manual testing
● Basic understanding of methodologies and SDLC and STLC models
● Knowledge of testing and quality assurance processes
● Strong experience with test design activities to cover a set of product requirements
● Knowledge of test design techniques and testing methodologies
● Defect analysis skills and the ability to write detailed and useful defect reports
● Experience with
Postman
for API testing and browser developer tools
● Experience with UI testing
● Experience with
TestRail
as a test management system
● Basic understanding of CI/CD processes
● Intermediate English level
Nice to haves
● Understanding of LLM evaluation principles, including prompt engineering, output validation, and model fine-tuning cycles
● Familiarity with AI and ML concepts, APIs, and frameworks such as
OpenAI API
,
Azure OpenAI
, or
LangChain
● Knowledge of data protection and compliance standards for AI systems, including GDPR, zero-data retention, and PII anonymization
● Experience with SSO testing
● Experience in performance or load testing
Perks and benefits
●
Professional growth: Accelerate your professional journey with mentorship, TechTalks, and personalized growth roadmaps
●
Competitive compensation: We match your ever-growing skills, talent, and contributions with competitive USD-based compensation and budgets for education, fitness, and team activities
●
A selection of exciting projects: Join projects with modern solutions development and top-tier clients that include Fortune 500 enterprises and leading product brands
●
Flextime: Tailor your schedule for an optimal work-life balance, by having the options of working from home and going to the office — whatever makes you the happiest and most productive.
Meet Our Recruitment Process
Asynchronous stage
— An automated, self-paced track that helps us move faster and give you quicker feedback:
● Short online form to confirm basic requirements
●
30–60
minute skills assessment
●
5-minute
introduction video
Synchronous stage
— Live interviews
● Technical interview with our engineering team (scheduled at your convenience)
● Final interview with your future teammates
If it’s a match—you’ll get an offer!