Outcomes Based on Evidence

If the  outcome is known in advance don’t fool yourself into thinking you are doing research.  

When police agencies are considering the deployment of new technology, new tools or innovative service delivery approaches, it’s not good enough to cite their use in other jurisdictions as justification to replicate them.  Unfortunately, all too often that is what happens.  Mayors and police chiefs look at what other jurisdictions are doing and fall into the ‘cookie cutter trap’, an approach that assumes that if something works in another jurisdiction it will work here as well.  A review of reports from the jurisdictions that are doing what you might like to do or emulate is substituted for real research.  The conclusions and anecdotal accounts contained in those reports are treated as evidence.    

The fact is, what works in one jurisdiction may not work elsewhere.  Consider the  use of bait cars  to address auto theft.    Some cities that use bait cars claim they are the answer to curbing the auto theft problem.  This may be true if the cars being stolen in a particular jurisdiction are high-end vehicles destined for export.   When the strategy was tried in Winnipeg, it was largely based on recommendations from police agencies in other jurisdictions and the company selling the technology.  Following the expenditure of a significant amount of cash it was determined that based on the unique characteristics of the auto theft problem in Winnipeg, as well as the  climate, bait cars were largely ineffective in Winnipeg and the program was discontinued.    

Another example is the ‘lets recreate Edmonton in Winnipeg’ experiment of the mid-1990’s that saw the proliferation of Service Centers, Community Offices, and Foot Patrol Offices under the guise of community policing, Edmonton style.  At the height of the euphoria (some called it madness), the Winnipeg Police Service was attempting to staff as many as 22 public access locations at a cost of millions of dollars.  These included 6 District Stations and 16 satellite offices.  With the departure of the  Chief (who was from Edmonton) most of those offices were quietly closed down.  The Police Service now operates a total of 7 public access locations, 5 District Stations and 2 satellite offices.  This is another example of a failed and very expensive experiment that can be attributed to the cookie cutter trap. 

Examples abound of failed attempts to superimpose apparently successful approaches and technologies from one jurisdiction to another.  

As argued during the helicopter debate (which was more of an announcement than a debate), evidence based approaches are essential.  In the case of the helicopter, determining  its effectiveness and efficiency from an evidence based perspective involves at a minimum the following steps: 

  • Establish the number of hours the helicopter will be airborne
  • Establish the number of hours that the helicopter will be on stand-by
  • Determine the type of calls for service to which the helicopter will be deployed
  • Call up the historical call for service data
  • Superimpose the deployment template over the historical data
  • Determine the number of instances the helicopter would have been deployed
  • Establish a cost per deployment
  • Establish the benefit or added value that the presence of a helicopter would have provided in the instances where it would have been deployed. 

Take vehicle pursuits as an example.  Police collect data on vehicle pursuits which include the number of pursuits per year, broken down by month, week, day of the week and hour of day.  There is also data about how long pursuits last.  

It would not be difficult or time-consuming to take the historical pursuit data, superimpose the anticipated helicopter flight times and  specify with a reasonable degree of certainty  the number of pursuits that a  helicopter would have been available to deal with in 2008.   

Let’s say for the sake of argument that Winnipeg has 100 vehicular pursuits per year and that the average pursuit lasts about 4 minutes.  This means that unless the helicopter is airborne it will arrive too late to be of use.  Based on an optimistic projection of 1000 hours of flight time, that means the helicopter would be available to aid in perhaps a dozen vehicular pursuits annually.  That’s one a month.  And based on experience in dealing with auto thieves in Winnipeg, I’m not convinced that in those dozen cases the thieves will stop the stolen vehicle, get out and lie on the ground, and wait to be arrested as has been suggested,  just because a helicopter is hovering overhead.  

This means that every time we have a pursuit in the city that ends badly and someone gets up on their soap box and says ‘it would have been different if we had a helicopter’ you need to take a big gulp of reality and say ‘it might have been different perhaps 12 percent of the time’.  The other 88 percent of the time the helicopter would have been sitting on the ground. 

It is research and data that bring reality to the debate.  It is data that elevates it from a political discussion to an evidence based debate and a decision-making process based on evidence 

Anyone who suggests that the recent council decision on the helicopter issue was an example of evidence based decision-making either does not fully understand the concept or has allowed their personal bias on the issue to cloud their judgment.   Anecdotes and rhetoric are not evidence.  

Winnipeg is a city with limited resources.  Our tax dollars need to be spent wisely.  Look at the spending of tax dollars like an investment.  Do we want our tax dollars invested with a money manager who does not do his homework, who does not do research, who simply looks at what other money managers are doing and attempts to mirror their investments and then crosses his fingers and hopes for the best?  

I suppose if you are not a Winnipeg resident and your  municipal tax dollars are going to East St. Paul  this may be less of an issue for you.   For the citizens of Winnipeg, however, it is a real issue.

Advertisements