To err is human. That’s never going to change. And that’s also why we need software testing. Your team will occasionally make mistakes, overlook conflicts, or have biases that perpetuate flaws.
Software testing reveals errors that occurred during development and is vital to prevent subpar products from getting into customers’ hands. Along those lines, software testing also contributes to the reputation you are building in your market.
Furthermore, software testing is vital to the efficiency of your operation, allowing you to identify errors or bugs as they occur rather than in the late stages of development when they will be much more difficult and costly to fix.
The elephant in the room, however, is whether ISVs and software development teams are following software testing best practices. SmartBear released its Third Annual Testing Community Survey to provide a picture of the state of software testing moving into 2020. SmartBear conducted the survey online, gathering responses from more than 2,500 development, testing, and software delivery lifecycle professionals.
The research report answers a variety of questions about how your peers and competitors are approaching this essential part of development:
Who is testing?
SmartBear notes that QA engineers and manual testers spend the most time on testing compared to people in other roles.
Notably, software testing isn’t only a priority for larger companies. Over the three years that SmartBear has conducted the survey, there has been a shift from responses from enterprises of 10,000 employees or more to greater participation from companies with fewer than 500 employees.
What are teams testing?
SmartBear found that most respondents to its survey, 79 percent, tested web applications, followed by API or web services, which 77 percent tested. SmartBear points out that from 2017 to 2019, the number of people testing API/web services and mobile applications has significantly increased while testing desktop applications, hybrid or progressive web applications and packaged applications have decreased.
When do teams test?
The amount of time that software developers and ISVs dedicate to software testing has remained steady over the past few years, with people responsible for testing using about 60 percent of their time to complete it.
Of the people responding to the survey, 23 percent say executing tests takes the most of their time, followed, in order, by creating automated tests, writing manual test cases, and reporting and analyzing results.
Interestingly, respondents who spent less than 20 percent of their time each week on testing were “significantly less satisfied” with their testing processes than those who spent 80 percent of their time. The authors of the SmartBear report point out that this could reflect that people who test less often are less familiar and comfortable with their processes than their counterparts who test more often.
SmartBear also found that 35 percent of teams reported that their organizations conduct performance testing, up 5 percent from 2018. Moreover, 47 percent report performance testing with every deployment, up from 33 percent last year.
How is software testing performed?
SmartBear’s research reveals that manual testing is most prevalent among survey respondents, used by 75 percent. Other common testing practices include:
- Automated testing, 63 percent
- Regression testing, 58 percent
- End-to-end testing, 56 percent
- Unit testing, 46 percent
- Exploratory testing, 40 percent
- Continuous integration/continuous delivery, 35 percent
- Security testing, 22 percent
- Behavior-driven development, 20 percent
- Test-driven development, 20 percent
- Service virtualization, 7 percent
- Other, 3 percent
The research shows that as company size increases, so does the use of automated software testing, with 56 percent of the largest firms (10,000+) automating vs. 37 percent of organizations of between 1 and 25 people. Using manual or automated methods isn’t all or nothing — the research shows that about one-third of all organizations utilize both manual and automated testing methods.
A Look Ahead
In addition to how software development teams conduct software testing now, SmartBear asked survey respondents about their software testing plans for the future. QA engineers expressed an interest in greater automation in the future; manual testers, however, had the lowest interest.
Respondents also anticipate that web applications and API/web services will represent most testing. They also feel desktop application testing will decrease by half over the next two years, and hybrid/progressive web application and Chromium embedded framework testing will increase. Software professionals expect artificial intelligence (AI) and machine learning will play a bigger role in software testing in the coming years.
For more information from the research that can provide you with benchmarks for your own software testing standards and processes, download SmartBear’s Testing Community Survey. 
