Home Software Testing

Testing should guide AI, not just check its work

Connecting AI to test artifacts and execution evidence turns individual coding tasks into a continuous, context-aware SDLC workflow.

software-testing-audit

Engineering teams are using AI to generate test cases, convert manual tests into automated scripts, and prioritize tests based on relevance and risk. Experts are even predicting that AI could potentially drive productivity gains of 30% to 35% across the software development lifecycle (SDLC). The issue is that, as of now, AI is largely being used to improve individual testing tasks. These capabilities make testing faster, but they don’t yet connect AI to the broader testing context needed to understand how a product should behave.

While AI is helpful for reducing repetitive work, there are still very relevant underlying challenges that this “solution” does not address. The first is behavior. When AI generates code, it struggles to map a specific product to how it should function in live business environments. This is because it doesn’t have access to the business rules, edge cases, historical failures, and expected behaviors that regression tests capture. The second is traceability. As AI coding accelerates exponentially, tracking exactly what the technology touches can become difficult, if not impossible, to do accurately.

For AI to move beyond individual testing tasks and into the next phase of the SDLC, testing itself needs to become part of how AI builds, evaluates, and acts. Transitioning to this development environment works best over a three-stage process.

Stage One: Connect

The first stage is to establish a single layer that safely connects AI to the test artifacts and execution evidence AI requires to build correctly. A Model Context Protocol (MCP), for example, can provide AI with only the relevant test cases, requirements, execution results, and other necessary information to support accurate code development.

If AI has this access, developers can leverage their AI agents to pull relevant testing context into their processes. This helps prevent another disconnected workflow in which developers must manually search for and select relevant test cases or scripts to build against. A developer could ask AI what coverage already exists for a requirement, identify gaps in test scope, and create and record tests within the same sequence.

Stage Two: Contextualize

Successfully operationalizing the access provided by the MCP layer requires tackling the behavior challenge. This entails converting test artifacts and execution evidence into a live context layer. At its core, this step is about preventing errors before they occur.

In practice, this means allowing AI to use the testing knowledge organizations have already built over time, including test cases, scripts, data, logs, requirements, and error messages to define how code should actually behave before and during development. Execution results can keep the context up to date, ensuring the behavioral baseline remains consistent with the actual product.

The layer also provides some relief for the traceability pain point we established earlier. Having a context layer allows organizations to trace the connection between requirements, tests, and code. In turn, it lays the foundation for understanding how one AI-generated change puts another existing behavior at risk.

Stage Three: Act

Once AI has a foundation of connection and context, the final stage is putting that into action. Long-term, this step should turn into AI agents executing workflows continuously.

While an old operating model might have had AI agents generating code or drafting tests, with newer context, AI could evaluate a proposed change using the context layer, pinpoint affected behaviors, and decide where the change requires additional validation. It could then update the test suites accordingly.

Over time, multiple agents could implement code using established tests as a guide and connect the entire process into a continuous workflow, with minimal manual intervention. This would enable development teams to focus on higher-value work and reserve their judgment for the approvals that matter most, such as analyzing test case drafts, expected results, or edge cases.

Putting the Stack into Practice

Consider a hotel booking platform that wants to add a “modify my stay” feature to allow guests to change their booking without necessitating a cancellation. In the traditional AI SDLC, asking an AI agent to create a feature like this would likely expose a documented feature, such as the cancellation widget, as AI wouldn’t understand how to build toward “correct” behavior in a live business environment.

With an integrated test context approach, the agent could build the “modify widget” based on existing test cases, scripts, and requirements. It would understand the impact changing one feature has on the others. It could run continuous tests to ensure effectiveness and feed execution results back to the test suite. Developers would only have to step in to review the process at checkpoints and make the final shipping decision.

While individual task completion is useful, autonomous workflows like the one in this example have the potential to reimagine the SDLC from the ground up. By employing an MCP and a live context layer to put AI agents into action, software development could operate efficiently and nearly autonomously.


Hélder Ferreira

Hélder Ferreira is Director of Product Management at Sembi. As a Product Director with a background in Mathematics & Computer Science, Hélder brings over a decade of experience building impactful software products in the internet and enterprise space. His journey has taken him from hands-on product ownership to leading cross-functional teams and driving strategic vision at scale. Today, Hélder focuses on shaping product strategy with a strong emphasis on innovation, user-centric design, and the transformative potential of AI. Hélder is particularly passionate about leveraging AI to deliver smarter, more efficient, and deeply intuitive experiences that solve real customer problems.

×