Forrester questions Linux security
Matthew Broersma, Techworld.com
05/04/2004 16:20:23
A new study from Forrester Research has concluded that the Linux operating system is not necessarily more secure than Windows. The report finds that on average, Linux distributors took longer than Microsoft to patch security holes, although Microsoft flaws tended to be more severe.
But leading Linux vendor Red Hat said that while Forrester's underlying figures were sound, its conclusions didn't give an accurate idea of relative security, as they failed to distinguish between patch times for critical updates and routine, obscure problems.
The report arrives in the midst of a fierce debate around the relative merits of Linux and Windows, and follows a number of reports perceived to have been slanted in Microsoft's favor. Last October, Forrester forbade its customers to publicize studies they had commissioned; it made the move partly because of criticism of a report from Forrester subsidiary Giga Research that found some companies saved money by developing with Windows rather than Linux. Forrester said it stood by the integrity of the study, but had erred in allowing Microsoft to use it in anti-Linux advertising.
Forrester's report may lend credibility to Microsoft's ongoing efforts to play down security concerns about its software. A new tactic in that battle has been to compare how long it takes for various operating system vendors to patch flaws -- the "days of risk" for each operating system. Microsoft's argument is simple, said Bradley Tipp, Microsoft's National Systems Engineer for the U.K., last autumn: "Open source systems are likely to be at risk for more days than Windows systems."
Indeed, Forrester found that, between June 1, 2002 and May 31, 2003, Microsoft had the lowest average "all days of risk", the time between the public disclosure of a patch and the time that patch is released by the operating system maintainer, compared with the Red Hat, Debian, MandrakeSoft and SUSE Linux distributions.
Microsoft took on average 25 days to release a patch; Red Hat and Debian 57, SUSE 74 and MandrakeSoft 82, Forrester said. "Microsoft's average of 25 days between disclosure and release of a fix was the lowest of all the platform maintainers we evaluated," wrote analyst Laura Koetzle in the report. "Microsoft also addressed all of the 128 publicly disclosed security flaws in Windows during our 12-month evaluation period."
Koetzle noted, however, that 67 percent of Windows flaws had been rated "critical", under the U.S.' National Institutes for Standards and Technology's ICAT project standard for high-severity vulnerabilities, compared with 63 percent for SUSE, 60 percent for MandrakeSoft, 57 percent for Debian and 56 percent for Red Hat.
Since Linux distributions are compilations of large numbers of independent components, the study also examined lag-times between the release of a patch for a Linux component and the release of the same fix by the operating system vendor, what Forrester called "distribution days of risk". Debian scored best in this metric, with 32 days, followed by Red Hat with 47 days, SUSE with 54 days and MandrakeSoft with 56 days.
Red Hat said the figures Forrester relied on for Linux distributions were above reproach, as various Linux distributors worked with the analyst firm on weeding out errors. But the conclusions drawn from those figures are nearly useless, the Linux company said. "A simple average doesn't give you a good picture at all," said Red Hat security response team lead Mark Cox. "It wastes the work put into the raw data."
The figures Forrester uses for "all days of risk" are arrived at by averaging the number of days needed to fix a flaw, without distinguishing between critical flaws and harmless ones. Thus, if a vendor took six months to patch a low-risk bug, it would make them appear to have a slow security response time overall, even if all critical bugs had been fixed instantly.
Using Microsoft's own definition of a critical flaw as a bug which could allow a worm to propagate without user interaction, only 13 Red Hat vulnerabilities were critical during the one-year time period, and they took an average of just over a day to fix, Cox said. "If you add denial of service attacks and privilege escalations, there were 47 issues in total, which took seven days on average to fix," he added.
"We fix issues that are critical to users first," he said. "When a remote exploit comes out, we drop everything to make sure it comes out quickly. That's more important than a bug in some obscure package no one uses. The report really doesn't take that into account. It's a shame because the raw data is there."
Cox also took issue with the perception that there is necessarily a lag between a module patch and a distribution patch - Forrester's "distribution days of risk". If a bug is critical, it will be released by the Linux vendor immediately, he said; if module maintainers haven't yet released a patch, Red Hat and other distributors do it themselves.
Cox said Red Hat is taking measures to deal with the lag time between the release of a patch and users' implementation of it, including making each Red Hat machine slightly different and a kernel program called exec-shield. Red Hat and other distributors are also participating in the Security Enhanced Linux project.
Microsoft is in the midst of a highly-publicized security push, which has involved an in-depth code review and a switch to a monthly patch release schedule, designed to ease enterprise patch installation.