At Axenic, we have two ISO 27001 Lead auditors and perform a significant number of certification reviews for NZ government agencies. One of the common challenges of auditing is selecting which controls (both procedural and technical) to assess when a client has a limited time-frame or budget.
Most auditors would agree that it is not possible to examine 100% of the audit evidence within a typical given time-frame or budget. Therefore we need some way to complete an audit, form an opinion and draw valid conclusions, without examining all of the evidence. This is where effective sampling can aid auditors to achieve their audit objectives, while examining a representative sample of the evidence.
Audit sampling is defined as ‘the application of audit procedures to less than 100% of an identified population’. This enables the auditor to evaluate evidence about some characteristic of the items selected, without examining all of the evidence provided. An auditor can draw valid conclusions by using sampling. However, there are different types of sampling methods that an auditor can apply to gather sufficient evidence to address the audit objectives and the size of identified risks. Sampling methods can be statistical or non-statistical:
• Statistical Sampling – the selection of the sample involves the use of mathematical techniques, from which conclusions regarding the entire population can be drawn and derived. To be clear, this method allows for audit results based on a sample to be extrapolated to the entire population. The methods commonly used are “random sampling” and “systematic sampling”.
• Non-Statistical Sampling – the selection of the sample population is not based on any statistics or mathematical calculations. The results should not be extrapolated over the entire population as the sample is unlikely to be representative of the entire population. This form of sampling does not use a structured technique to identify the sample. It is often biased and based on certain question, the auditor’s previous experience or personal judgement. The most commonly used non-statistical methods are “haphazard sampling” and “judgmental sampling”.
Whichever sampling method is used, the auditor needs to design and select the audit sample, perform an audit procedure on the selected items and evaluate the results to obtain sufficient, reliable, useful and relevant audit evidence. Here are some things that need to be considered when designing a sampling audit:
• Sampling risk – This is the risk that an auditor’s conclusions may be different if the entire population were subjected to the same audit procedure rather than the sample population that has been used.
• Audit Scope, Objective and Procedure – For audit sampling to be effective the auditor needs to define the scope, identify the objective of the audit and consider which audit procedure is most likely to achieve the objective before assessing if sampling is the best tool for this task.
• Population, Sampling Unit and Sample Size – As defined above, the population is the entire set of data from which the auditor wants to select a sample. A sample is comprised of sampling units, that is the individual item to which the audit procedures need to be applied. The size of the sample depends on the acceptable level of sampling risk and the amount of errors expected to be present in the population.
• Selection Methods – As well as ensuring that the entire population is identified, the auditor also need to control the selection of the sample to maintain audit integrity. Selection can be based on statistical or non-statistical sampling methods.
The outcome of the decisions of each of these consideration should be documented in an audit sampling plan, which will be incorporated into your final audit report. Based on the audit sampling plan the sample can be selected, the identified tests can be carried out and the full audit procedure can be completed. This will be followed by evaluation of the results, the write up of the audit findings, recommendations and actions in a full audit report.
In conclusion there are many sampling methods that can be used during an audit or Certification and Accreditation review. If sampling is used and the objective of the audit requires that results and recommendations are applicable to the entire population, it is necessary to use statistical sampling. Selecting samples through a statistical selection method will ensure a good representation of the population and therefore allow the test and audit results (performed on the sample) to be extrapolated to the whole population. Ultimately, this gives greater confidence to stakeholders that the findings of the audit are representative across the entire control set and that the related assertions about the fitness or completeness of controls are accurate.