In my last article I spoke at some length about not just why a Security Policy is important, what its content should be, but also how it should be written. There is no default setting for Security Policy. Remember, what works for one organisation probably won’t work for another.
There are numerous websites on the Internet that will offer Security Policy templates for sale, but you must always be mindful that even using one of these templates may require a considerable rewriting of the Policy to ensure its relevant to your organisation. And you must never forget, it has to be written so that it is accessible to your target audience and in a way that is likely to engage them.
The cost of producing and implementing a Security Policy varies. Why shouldn’t it? After all, if the Policy content itself differs from organisation to organisation, it follows that the expense incurred in introducing a Policy will also vary. It goes without saying that an organisation with 20 employees and 20 workstations will probably spend considerably less on their Policy creation and implementation than a multi-national organisation. But remember, it is all relative. $20,000 spent by a small company on developing and implementing a Policy may be dwarfed by the $1million spent by a multi-national, but each could be a sizeable chunk of the respective organisation’s annual security budget.
Whatever the cost to the organisation, the Chief Executive or Managing Director will want to know that the funds and resources they assigned to the development and production of the Policy is justified. How are you going to persuade them that it was money well spent? How do we know if it is working? How can we confirm the organisation’s workforce is aware of its contents? Most importantly, how can we verify if the policy is being followed?
So what can give us that level of assurance that our Security Policy is suitable and relevant? How do we know what parts of our program are working and what aren’t? The answer to these questions is security metrics.
Type security metrics into a search engine, and the results identify somewhere in the region of 18 million hits. Security metrics are widely accepted as an excellent, and to many the best, way of gathering your policy’s performance data and turning it into a tangible set of measurements. How you come to gather these measurements is the subject of much debate and the intricacies of using the different methods of gathering measurements cannot be given the time or space it deserves in this article. However, I will discuss what I consider the fundamentals of any security metrics program.
Many subscribe to the idea that good metrics need to be SMART. By SMART we mean they should be Specific, Measurable, Attainable, Repeatable and
Time-dependent. There are variations on this, with Brotby and Hinson’s PRAGMATIC (Predictive, Relevant, Actionable, Genuine, Meaningful, Accurate, Timely, Independent, Cheap) being one of the most popular. Which you subscribe to depends entirely on whoever is implementing the metrics program. Ultimately, the metrics must:
• Support the business’s goals – It is essential that the connection to those goals be unambiguous. A raft of numbers and percentages may not mean a great deal to your managers, how they are impacting the business will.
• Be manageable – The example I’d use here is that reporting on the number of vulnerabilities in your environment may be important, but the time taken to patch those vulnerabilities is more important. The vulnerabilities are not in your control, the time taken to patch them is.
• Be quantitative – There is a time and place for traffic light or qualitative measures (e.g. LOW, MEDIUM or HIGH), however, they are not useful when reporting metrics. For metrics to be truly useful, they need to be quantitative (i.e. based on real numbers and/or percentages). One of the most important functions of a metric is to allow the organisation to develop a baseline, against which all future metrics results can be compared, to ascertain its performance overtime.
• Be easy to collect and analyse – Both the SMART and PRAGMATIC approach highlight the importance of timely metrics. If your metrics are gathered monthly, but some take 2 or 3 weeks to gather data, they are not going to be a true refection of a specific point in time. Further, they may not enable the business to make decisions in a timely manner.
• Be measured consistently – Having a process for gathering metrics is essential. Ideally, the collection of metric data should be automated to ensure this is achieved. If they cannot be automated, it is essential there is a formula that can be consistently followed by those who are responsible for manually gathering the data.
So what is a security metric? How do we define it? Interestingly, one of the search results directed me to that very question that had been asked on LinkedIn. This particular question produced a multitude of answers from numerous contributors and everyone one had a slightly different take on it. To my mind, a metric is a means of using measurements via a formula to produce a result, although there appears to be no definitive meaning. The analogy I find useful is how we determine a person’s Body Mass Index (BMI). The formula is:
Weight (kg) ÷ (Height (m) x Height (m)) = BMI
e.g. 60 kg ÷ (1.7 x 1.7) = 20.8
Weight and height (i.e. the measurements) can in isolation mean very little. They may tell us someone is tall or heavy, but it is only by using the formula, or the metric, that we can find the BMI result and assess whether they are overweight.
If we transpose this to an IT environment, examples of measurements would be the number of end-user devices, and the number of end-user devices with anti-virus (AV) software. We would then gain an average using a simple formula:
200 devices (with AV installed) ÷ 800 total devices x 100 =
25% of end-user devices have AV installed.
This high-level metric tells us that just one quarter of our end user devices have AV installed. We can then begin to get more granular with our metrics and identify the different types on end user devices, for example:
100 workstations (with AV installed) ÷ 500 workstations (total) x 100 =
20% of our workstations have AV installed.
100 laptops (with AV installed) ÷ 300 laptops (total) x 100 =
33.3% of our laptops have AV installed.
We could develop these metrics even further. For example, whilst we have already identified how many of our end-used devices have AV installed, how many, and more importantly what percentage, have up to date AV signatures? Additionally, we could get even more detailed, by identifying how many, and again what percentage, of end-user devices in different business units, such as HR, Finance and Operations, have AV installed, and up to date AV signatures. This becomes particularly useful in identifying possible weaknesses in our security program. For example, if our metric indicates that HR have significantly fewer workstations with current AV signatures than other departments, we can start to investigate what the reason(s) for this could be.
So how do you decide what metrics to produce? We were recently engaged to produce a series of security metrics for a customer who had developed and implemented an Information Security Policy, aligned with the ISO 27001. The customer was keen to measure the effectiveness of their policy and identify which security metrics they could feasibly implement.
During the course of the engagement, it became obvious that to implement all possible metrics would be cost-prohibitive for the organisation. Rationally, they decided on a ‘Top-5’ approach, whereby they identified those 5 areas where they considered the development of security metrics a priority. This is a sensible tactic, as I would argue it is much more practical to have fewer well-designed and implemented metrics than an abundance of meaningless or inefficient ones.
The ability to report against meaningful metrics is important to the success of any security program, as they provide management with assurance that the organisation’s information security risks are being effectively managed, together with evidence to help justify the continued funding of the program. Additionally, security metrics are an excellent tool for us as security professionals to identify how effective a security programs is, identify where it may not be performing, and allow us to address any failings. After all, if we have invested so much time, energy and more importantly resources into developing and implementing our programs, shouldn’t we make sure they work?