Python, FastAPI, Qdrant, Playwright, React, MCP, Docker
Business Services
AI QA Agent/Agentic AI
India
About The Project
Creating test cases for a single Jira ticket required reviewing the requirements, checking linked Figma designs, locating the relevant source code, validating business rules in the documentation, and writing manual test cases. The process was slow, repetitive, and often inconsistent between testers.
Generic AI tools did not solve the issue. The test cases seemed accurate, but they often included outdated fields, missing APIs, or flows unrelated to the actual product, forcing QA engineers to verify everything manually.
To solve this, we built a retrieval-augmented generation AI QA agent that works with the project’s Jira tickets, Figma designs, source code, and requirement documents. Every test case is linked to its source, and approved cases are automatically converted into Playwright scripts, executed in a real browser, and used to generate bug reports for failed scenarios.
Key Project Deliverables
We delivered a QA workflow designed for end-to-end traceability, from the initial test plan to automated test execution.
Ticket-Driven Test Planning
The agent reads the Jira ticket and its Figma design to draft a structured test plan covering scope, acceptance criteria, user flows, and edge cases.
Source-Cited Test Cases
Each generated test case includes the exact file or requirement reference it was derived from, allowing testers to verify the output quickly.
Live Playwright Authoring
Approved test cases are converted into executable Playwright scripts, and the steps are validated in a real browser before the script is finalized.
Automatic Bug Reporting
Failed test runs are converted into ready-to-file Jira bug reports that include screenshots, failure details, logs, and a link to the parent ticket.
Private Code Retrieval
All source code search occurs within the local environment, ensuring that client code is never uploaded to any external AI or vector databases.
Flexible Model Support
Teams can use different AI models from a shared catalog while keeping API keys stored locally on their machines rather than in a shared database.
Project Challenges
The biggest challenge was accuracy. AI-generated test cases can appear professional even when they contain subtle mistakes, and QA teams cannot depend on outputs that require line-by-line fact-checking.
Confidentiality was another critical concern. Client source code could not leave the organization’s environment, which ruled out most standard approaches to code search and shaped the entire retrieval design.
Generating browser automation was also more difficult than generating the test cases themselves. Scripts written from screenshots or static page information often failed when executed against the live application.
The team also had to handle CAPTCHA-protected logins, manage long-running AI conversations, and prevent unnecessary token usage during extended QA sessions.
Solutions & Impact
We built a retrieval-augmented generation architecture that searches the project’s own code and documentation before generating any QA output. Requirement documents are indexed for semantic search, while source code is retrieved locally to maintain confidentiality.
- Integrated Jira and Figma through MCP for automatic requirement and design retrieval.
- Built hybrid local code search using both keyword and semantic retrieval.
- Indexed business and requirement documents for semantic search across all generations.
- Added source citation to every generated test case.
- Implemented live Playwright authoring that validates steps in a real browser.
- Deployed reusable login sessions for CAPTCHA-protected applications.
- Isolated generated scripts in a restricted subprocess using test-only credentials.
- Added automatic bug drafting with one-click Jira filing.
- Built a shared model catalog for multiple AI providers with locally stored API keys.
The agent became part of the team’s daily QA routine. Testers spend less time writing repetitive test cases and more time exploring complex and edge-case scenarios. Test coverage is more consistent, every test case remains traceable to the original code, and failed runs become ready-to-review bug reports.
Ship Faster with an AI-Enabled Quality Assurance Agent
Create traceable test cases, validate browser automation, and speed up releases without sharing your source code outside your local environment.
Project in Figures
3
Months
800
Estimated Man-Hours
4
Members
Applied Technologies
More Screens
