Audit Office criticises PSNI decision to abandon key performance figures
The PSNI and Policing Board have been criticised by Northern Ireland’s Auditor General for abandoning the use of key performance targets to identify improvement levels within policing during the last 12 months.
Auditor General Kieran Donnelly said the PSNI decision to replace percentage performance targets with a narrative-based assessment represented a “major shift in approach” and made it difficult for auditors to properly evaluate what improvements had actually been achieved within the police service.
Each financial year the Northern Ireland Policing Board (NIPB) is required to publish a Policing Plan setting out how continuous improvement targets for the PSNI have been established and achieved.
However for the first time ever the Auditor General has qualified his opinion on the Police Plan for 2012/13.
Mr Donnelly has expressed concern that 40 out of 44 performance targets contained in the board’s Policing Plan all lacked sufficient clarity to convince him that the level of required improvements had been achieved by the PSNI within the agreed time frame.
As a result, Mr Donnelly said that he did not consider the standards which the PSNI used to measure improvement targets as reasonable.
The Auditor General said that while the PSNI had achieved the majority of performance targets, a significant proportion had not been met.
Despite this the Policing Board had failed to provide any explanation for the failure to meet the required targets.
The Audit Office examined the Policing Plan for 2012/13 and the performance of NIPB and PSNI in 2011/12.
As previously stated above the Policing Plan had included 44 performance standards.
Only four of those performance standards (relating to road safety), were evaluated by using percentage achievable terms.
The remaining 40 performance standards failed to identify any percentage evaluation upon which targets were to be improved; instead the new system refers simply to a nondescript increase or decrease in targets.
Mr Donnelly said the decision to replace percentage based targets with narrative-based performance assessments, represented a major shift in the PSNI’s approach as to how improvements in policing should be properly evaluated.
In 2011/12 the PSNI set itself a range of percentage targets to increase detection rates for a range of crimes.
The use of those percentage targets meant that police chiefs were able to identify key facts such as a 10.8% rise in detection of armed robbieres on business properties but a 12% drop in the detection of sectarian crime.
However the decision to abandon the use of percentage indicators in 2012/13 meant the PSNI no longer commits itself to any specific percentage targets in relation to improvement arrangements in policing.
The PSNI said the decision to drop the use of percentage indicators had been taken because of continued difficulties with a ‘target driven approach’ to planning and that setting ‘realistic but challenging targets’ had been difficult to achieve.
Mr Donnelly acknowledged that setting numeric targets involved a degree of uncertainty but said that he could not understand how this could be used by the PSNI as “justification for not setting any target levels at all”.
“While I note PSNI’s comments on target setting, I have concerns about their approach,” he said.
“What precisely the ‘highest reduction/increase possible’ will mean in practice and the basis, on which performance will be judged acceptable or otherwise, is not clear.”
Auditors found that some PSNI targets focused only on measuring activity, rather than examining the final outcomes of that activity.
In one example, auditors found that the PSNI had measured only the submission of progress reports relating to the implementation of a ‘Policing with the Community’ strategy, rather than assessing the actual impact of the strategy itself.
In another instance, auditors found that target performance data used by the PSNI to increase the detection rate for serious crimes failed to include any reference to the actual number of crimes and detections involved.
Commenting on the PSNI’s failure to identify the most reliable data to properly measure specific performance targets, Mr Donnelly said: “This is significant, since variations in the numbers of crimes committed can affect the performance measurement. For example, in a given situation, achievement of (or failure to meet) a detection target may be effected merely through a change in the number of crimes committed, rather than through any increase or decrease in the actual number of detections.
“In my view, the performance assessment should include both the percentage achievement and the actual numbers involved.”
Auditors further found that PSNI performance figures had failed to make reference to potential variations in local areas or districts.
In one instance the PSNI used figures to record the percentage of residents who believed that the level of anti-social behaviour in their local area was abnormally high.
However auditors found that the figures used by the PSNI focused only on the overall Northern Ireland figure and not individual districts, potentially masking important local variations where poor performance may have been an issue.
While Mr Donnelly said his report was not a criticism of PSNI performance, he said it did question how objectives and targets have been set and the extent to which a meaningful assessment of the PSNI’s ongoing performance can be best analysed.
“In the absence of target levels, I am concerned that the effectiveness of such scrutiny, and thereby accountability, will be diminished,” he said.
When auditors reviewed the 2012/13 Policing Plan they found that, with just one exception, there were no performance measures for crime detection/outcomes.
“This is surprising, given the references within the (policing) plan to helping victims of crime,” Mr Donnelly noted.
Auditors found that other proposals for police improvements within the plan were expressed in “quite vague terms”, including one proposal for the PSNI to develop its ability to communicate through use of the internet.
Mr Donnelly said it was unclear what was to be measured and how it would be achieved.
Only four out of 44 performance standards (relating to road safety measures) were found to meet the required level and timeframe of improvement targets set out in the Policing Plan.
However auditors found that the target dates for even those four road safety-related targets will not be achieved until 2020, some seven years away.
“In the context of a Policing Plan for 2012-13/2012-15, this has limited meaning,” Mr Donnelly said.
Of the remaining 40 performance measures, none have a deadline for completion/achievement.
NEED FOR ‘MEANINGFUL ENGAGMENT’ BETWEEN NIPB AND PSNIThe audit report found there is a growing need for a more “meaningful engagement” between the NIPB, Department of Justice (DoJ) and the PSNI.
In January 2012 NIPB members were invited to sit on the PSNI’s ‘Continuous Improvement Programme’ boards.
However auditors found that to date the level of attendance has been ‘mixed’.
Auditors found that the PSNI has failed to supply NIPB members with key documents need to properly assess the true value of its continuous improvement programmes.
“In my view, without these, the extent to which the Board can meaningfully review PSNI’s performance on continuous improvement projects is limited,” Mr Donnelly concluded.
In 2004 a Continuous Improvement Strategic Working Group (CISWG) was set up involving officials from the PSNI, NIPB, DoJ, Criminal Justice Inspectorate (CJI), Her Majesty’s Inspectorate of Constabulary (HMIC) and the Northern Ireland Audit Office (NIAO) itself.
The working group, chaired by NIPB, was designed to improve co-ordination in the external inspection activity involving PSNI and avoid unnecessary duplication.
However the working group has only met three times in five years.
“I note that in recent years the level of activity of the group has been quite irregular; for example, it did not meet during 2008-09 and 2009-10 and only once in 2010-11.
“Activity revived the following year with two meetings, in October 2011 and March 2012, but since then the work of the group has again fallen into abeyance.
“In my view, the group has significant potential, but it needs to meet on a regular basis – two to three times each year – if it is to be fully effective.”
Highlighting ongoing difficulties in the working relationships between the Policing Board and PSNI, Mr Donnelly said: “I recognise that there is an inherent tension within the relationship, given the need to strike a balance between holding the PSNI properly to account while, at the same time, providing the necessary support to help it achieve its goals.”
However the auditor said it was important that NIPB and the PSNI made “every effort” to further enhance the quality of their working relationship.
“In my view, it is vital that the senior officers within the Department (of Justice), the Policing Board and the PSNI continue to take a close personal interest in monitoring progress in this regard.”
A Policing Board spokeswoman said it accepted the recommendations contained within the report.
“The board is committed to ensuring that the best possible policing service is provided to the public and that improvements are delivered which both benefit the community and the overall performance of the PSNI.
“Board members have already held discussions with the Chief Constable and his senior team on the issues raised in the NIAO report and these will continue as plans are progressed for developing performance targets and measures for the 2014- 2017 period.”
Despite first being contacted on Monday, no one from the PSNI was available for comment.
On Tuesday, a full day after first being contacted by the Detail, the PSNI released a statement.
“Crime in Northern Ireland is at a ten year low and confidence in policing at 87% is the highest ever recorded level.
“PSNI will continue to work with the NI Policing Board to develop a robust policing plan and we remain totally committed to ensuring that we deliver a sustainable and professional service which our communities have confidence in.”
Read full NIAO report here
© The Detail 2013