You are here

Open source security: 4 metrics that matter to app sec teams

Robert Lemos, Freelance writer

Open-source software components are incorporated into almost every major development effort, but the security of those components continue to be a problem. The annual Open Source Security and Risk Analysis (OSSRA) report published by Synopsys found that more than 96% of codebases scanned in 2018 had open-source components. In 2019 that jumped to more than 99%.

Meanwhile, the report found the average age of open-source vulnerabilities detected by software scans is not decreasing, suggesting that companies are not getting better at cleaning their code of known issues. In fact, 40% of codebases are using a component with a vulnerability more than a decade old.

Most developers don't look at the security track record of open-source projects before adopting a framework or including a component in their development efforts. Yet secure code is critical in business today.

To better manage the security impact of such projects, application security professionals and security champions within development groups need to pay attention to fundamental metrics. Here are the four that matter most.

[ Get up to speed fast on the state of app sec and risk with TechBeacon's new guide. Plus: Get the 2019 Application Security Risk Report. ]

1. Indicators of good project management

Major projects with many users that issue frequent commits tend to have fewer vulnerabilities than smaller projects on GitHub with only a handful of contributors, according to the WhiteSource data.

For that reason, some readily available measures of the popularity of an open-source component can help companies prioritize their security scrutiny, said Rami Sass, co-founder and CEO of open-source software scanning firm WhiteSource Software.

The size of the user base, the number of commits per month, the number of GitHub stars the project has collected and over what period of time, and the number of developers are all important metrics. These stats are easily collected from GitHub or other project pages that can determine popularity of a project and efficiency of its management, he said.

"There is a correlation between the overall maintainability of a project and the level of security protection or the amount of risk it represents. The more eyes you have on it—the more active users and active contributors that you have—the more likely they are to find and fix vulnerabilities relatively quickly."
Rami Sass

2. Mean time to update the project

Projects that are updated more frequently are more likely to issue security patches in a timely manner, making the components more secure. A project's mean time to update (MTTU) is harder to quantify, but can give developers and application security professionals a good idea of how responsive a project is to security issues, said Matt Howard, senior vice president at Sonatype.

Well-managed projects supported by a nonprofit foundation are typically six times more popular, have an MTTU that's 18 times faster, and have one-third more committers than their peers, according to the State of the Software Supply Chain report from open-source component management firm Sonatype. 

"If an open-source project is really good at staying fresh with respect to its dependencies, then the quality and the hygiene of the project are going to be higher."
Matt Howard

[ Take a deep-dive with our Application Security Trends and Tools Guide, which includes our 2019 App Sec Buyer's Guide. ]

3. Mean time to remediate in your organization

Another metric that is usually difficult to generate is the mean time to remediate a vulnerability. However, for open-source software projects, Sonatype and other groups have found that this metric is closely linked to the MTTU, which can be used as a proxy.

However, the speed at which an organization applies a security patch within its application is a metric that you should track.

All signs are that companies, and many open-source projects, have a long way to go before being efficient and vigilant at patching security vulnerabilities in their own software. A survey of some 36,000 Java-based open-source projects found that the mean time to remediate (MTTR) was 326 days.

While that sounds bad, most development teams at companies fare even worse. The average age of vulnerabilities discovered during software audits in 2018 by Synopsys' Black Duck division was 6.6 years, according to the OSSRA report, a bit worse than in the previous year. The oldest vulnerability discovered was 28 years old.

The OSSRA report noted:

"The open source community does an exemplary job of issuing patches, often at a much faster pace than their proprietary counterparts. But whether companies are using proprietary or open source software, an alarming number of them don't apply patches, opening themselves to risk."

4. Measuring secure coding ability

Even if vulnerabilities are caught early in the development process, having some measure of how likely developers are to check in vulnerable code is instructive, said Caroline Wong, chief security strategist at, a penetration-testing service provider. This is a measure of secure coding knowledge and the capability to catch such errors in a company's development process.

Other metrics that can indicate that a company has a mature software development cycle include:

  • How quickly developers can confirm a vulnerability report and issue a patch in their own software
  • How quickly developers update their software to include the latest dependencies

"Many eyeballs are not the same as skilled, objective-driven eyeballs which have been tasked with applying a structured and comprehensive manual pen-testing methodology."
Caroline Wong

Paying attention means secure code

Developers do not always have the luxury of considering security first, said Sass. 

"When developers look for open-source packages, the main motivation is that they need some functionality—they have a job to do and they are looking for open source to be productive and do it better. So I really don't think that you can factor security considerations into the selection process, unless some project is grossly vulnerable."
—Rami Sass

However, companies that focus on security and pay attention to the security of the open-source software that they incorporate have a better chance of producing secure code. While the task is daunting, there is hope. In 2018, 60% of the codebases audited by Synopsys had a vulnerability. This was high, but lower than in the previous year, when 78% had a vulnerability.

Wong said it came down to assuming all third-party is a risk.

"Development teams should consider open software to be just as vulnerable as any off-the-shelf, third-party source code that they integrate into their project. It’s simply not accurate to believe that because many so different developers may contribute to open-source projects the software is secure."
—Caroline Wong
