The health of your software development life cycle (SDLC) is an important indicator of your organizationsā quality assurance, cost effectiveness, customer satisfaction, and compliance. While the executive order (EO) on improving the nation's cybersecurity issued in May 2021 only required software Bill of Materials (SBOM)s for federal agencies, it has resulted in widespread promotion and adoption of SBOMs as an industry best practice. The EO has also driven many organizations to re-evaluate their software development processes and implement measures to improve the security and resilience of their software supply chains. It also underscores that organizations need to pay closer attention to their development practices and health, as SBOM creation can expose ādirty laundryā to the world. This is why itās time to stop using pen tests to simply find problems in your software, and start using them to find problems in your processes.
While software developers have long used third-party web app and API pen tests to find application security defects, pen tests are also a great way to gauge the health of an SDLC. Third-party automated pen tests, which are often mandated by regulatory agencies, require very little experience with tooling or training on the part of the customer, and are fairly thorough in identifying security flaws that need to be fixed. This is why theyāre popular. From both parties' points of view, it's a pretty fulfilling activity; the pen tester gets to hand over a number of alarming-sounding vulnerabilities, and the developers receive a number of defects to add to the backlog.
However, what third-party automated pen tests cannot replace is the human behind pen testing execution. While humans are slow and more expensive than automated defect discovery tooling, because they can mimic human hackers, they are better at evaluating an application's response to a pen test, and can potentially catch responses that automated software may miss.
One way to think about how you are using pen tests is to look at the cost of finding defects. For example, finding a cross-site scripting defect using a SAST tool is much more efficient than finding it using a pen test. Using pen tests for vulnerability detection means that each one costs a percentage of your limited testing time, a fraction of your checklist, and a portion of your report writeup effort. And when companies run pen tests on an annual basis, the cost per defect increases as vulnerabilities get patched, and the critical and high vulnerabilities dwindle to nothing. So if pen testing shows diminishing returns as a defect discovery tool, how can you use the expensive but more accurate human element of pen tests to take full advantage of their value?
Where pen testing really begins to pay off is when organizations leave routine defect discovery to automated tools and shift the human effort toward AppSec program assurance. Just as there are multiple points in a piece of software that could validate, detect, or encode a cross-site scripting attack, there are multiple points in an SDLC that can anticipate, prevent, detect, or mitigate various classes of vulnerabilities. By shifting pen testing efforts to finding those inflection points in your SDLC where vulnerabilities can be prevented, youāre using the human and financial costs of pen testing more efficiently than if youāre just focused on defect discovery. Using pen testing this way can help you detect the processes in your SDLC that allow vulnerabilities to creep in, so if you begin fixing those processes, youāll also minimize future vulnerabilities.
Rather than relying on pen tests to detect security flaws that must be patched individually, pen testing should be used to perform a blameless postmortem, and analyze whether improvements are needed to ensure that potential failures are recognized at specific points in the SDLC. There are many defect identification systems that can find single types of issues, but they are usually not great at finding edge cases. For that, you need human beings, and this is where the pen test can really work for your organization.
You can use pen test results in various ways to secure your SDLC as well as your code.
Once these vulnerability issues are covered, you can begin to reposition pen testing in the risk management portfolio. When delivering penetration test results, there should be tooling and knowledge in place to facilitate a discussion with teams to not only talk about how to fix it in the code, but with the AppSec program owners about which of their capabilities may not be operating at the level they think they are. When incorporating pen test results into your SDLC, the results can help prioritize recommendations, but pen testing canāt provide direct evidence for training needs, compliance enforcement, vulnerability management needs, security champions, or other key capabilities that donāt directly prevent or detect pen test security defects.
It's time to rethink how pen tests are used in risk management. There are cheaper ways to find defects and faster ways to reduce risk in an application, but the pen testās human-driven nature means that it will thoroughly exercise whichever capabilities are not being capitalized on.