Your company is only as strong as the software that runs it. Today your software delivery team is as strategic as your sales, marketing, and service teams.
In order for this statement to stand true, software testing must be a critical component and priority for the business. Yet, testing is still one of the least valued jobs in the software industry. The lack of sexiness is real; it does not create direct value to the customer, it includes a lot of repetitive tasks, and many consider it a lesser profession. But the reality is that great software is always carefully tested.
Its boring reputation has become its detriment. Software organizations rarely invest enough time and intelligence in testing. If one is given a choice of spending budget on creating new features in a product or testing the existing ones, the former option is often chosen without much thought. Or, if a meaningful testing budget is available, does it really get spent wisely?
The best motivator for software organizations to test lands on one crucial component – the customer. When customer satisfaction sees a visible decline, quality and testing are considered seriously.
But why be reactive when the proactive stance saves time, money, and reallocates efforts in places that can contribute to the business’ bottom line?
For those who seek to benefit from automated testing, here are tips and tactics to make the most out of the testing experience.
Do not pass go… until you’ve read this first
If you want to find something to improve in a software organization, testing is a sure bet. But before initiating any major improvement actions, it is worthwhile to figure out where the organization is with its testing today. The following are the typical scenarios. Does your organization fall under a category?
- There is no professional testing at all and it should be put in place. Many development teams attempt to be agile and implement DevOps into their organizations but often fall into the trap of not truly understanding these critical processes. How professional is the software development process? If there is no professional testing in place, chances are the development process requires serious optimization.
- An old-fashioned waterfall development organization that has a large testing department. Do your math first. If the organization still applies the best practices of the early 1990s, chances are you’re better off replacing the teams instead of trying to change their practices. If that is not a viable option, prepare for a long change management project with a lot of training and un-learning.
- More testing is necessary. This is typical in organizations that have received negative feedback from their customers. Again, assess the development process first. These are typically very challenging assignments because you’ll have to tell the (business) leader that quality is not going to happen by just testing more and it is not going to solve their problems in a short time.
- Testing productivity is lacking. This should be an impactful assignment. There is professional testing in place. It costs too much and is too slow. Almost all testing organizations are too expensive and too slow and almost all of them have great improvement potential.
- Improve transparency. Testing is supposed to make quality visible and actionable but quite often it fails to do so, even in quite professional testing teams. Business leaders usually see the symptoms of this deficiency in poor predictability of the software releases. Like in any transparency initiative, the challenge is human resistance: transparency is welcome and desired only when it is applied to everyone else but the receiver of feedback.
Build automation incrementally
Test automation is notoriously expensive to do and even more expensive to maintain, even with the most advanced tools. Therefore, automation should first be applied to tests and testing activities that are frequently repeated and infrequently changed.
A proper sequence of automating test execution works as follows.
First, run the automated tests you already have – if any. A representative subset of automated tests at the very beginning of a test cycle tells whether the old functionality is still working after having added new functionality and whether it makes sense to proceed to test the new functionality.
Second, apply exploratory testing to new functionality. Exploratory testing, when done by a skilled tester, is the most productive means for detecting defects in new software. Exploratory testing means that someone with a tester’s mindset spends a few hours with the software and tries to break it. The feedback is thus very fast and the effort spent is optimal. Because there is no other documentation, it makes sense to record (video capture) the session.
Third, document and augment relevant tests for re-testing purposes. Some of the work of the exploratory tester is throw-away stuff but, for the most part, it is reusable for regression tests. For that purpose, it must be documented so that it can be repeated. Note, however, that if you plan to automate those tests, the test automation script usually serves as adequate documentation, too. Because the exploratory test was meant to be a quick but effective snapshot, the regression tests usually require some further development and variation to improve test coverage.
Ultimately, automate when you can, then repeat by starting at step one.
The four ways to improve the productivity of testing
If you measure what gets tested, how time is spent on different testing activities, and where defects are found, you’ll eventually find significant improvement opportunities, even beyond testing itself. Below are some simple ideas.
Focus testing on functions and features that have had the most defects in the past; they are likely to have the most defects in the future.
Study when errors were detected and when they were made. Anything you can do to shorten the time between making an error and finding it will improve both speed and quality.
Watch out for re-correction rates. Certain defects get corrected several times – and typically always end up wrong. Find out why by tracing back to the origins of such defects, which often reveals improvement opportunities in the software process.
Automate wisely. If your software is of any value for business it will be tested more frequently than you originally thought. Automation tools and methods matter: maintaining poorly built automation may be more expensive than testing manually but robust automation will save enormous amounts of time and money while helping improve quality.
