How Do I Make My Program More Evidence Based? You Don’t

One of the more interesting, and challenging, questions I’ve received lately has to do with the question of ‘how do you retrofit a 30-year-old program to make it more evidence based?’ The answer is simple: you don’t.

The program in this case is Neighbourhood Watch. Community watch programs were popular in the late 1980s, at a time when the criminologist David Garland (1996) observed that governments were keen to ‘responsibilize’ citizens into taking ownership of crime problems (as evidenced also by the proliferation of tips programs, community policing initiatives and so on). What we now know after some 30-odd years is that the evidence base for community watch programs is pretty weak. I was able to locate one Campbell Collaboration meta-analysis from 2008 that found a 16-26% reduction in crime across 12 studies (Bennett et al. 2008). That’s telling. In 2008, Neighbourhood Watch programs had been in existence for some 20 years and the authors could only find 11 sufficiently strong enough studies upon which to base a meta-analysis. None of the studies, I note, were randomized controlled trials. In researcher geek language, they were a level 3 (out of 5) on the Maryland Scientific Scale (translation: okay, but not the best). Further, we now know they have massive attrition rates. This fact wasn’t surprising to an old time Vancouver Police Sergeant I interviewed back in 2000. Speaking on his work training community groups to enact Watches he observed that new people would come out and be full of enthusiasm, but that enthusiasm would wane when they realized that crime prevention work is “eternal.” It’s also important to acknowledge another well documented fact, Neighbourhood Watch programs only work well – when they do – in more affluent neighbourhoods. As Steve Herbert (2006) has said of community policing more generally, such initiatives have an ‘unbearable lightness of being’ in poor neighbourhoods, meaning that, despite all the rhetoric and promise, impoverished, marginalized and/or transitional neighbourhoods typically lack the capacity to successfully maintain programs that put responsibility on local community members. Keep in mind: when you’re working 3 jobs to put food on the table, it’s a bit hard to also be trying to pull watch duty or run meetings as a Block Captain.

Now 30 years later, I’m being asked how to take this program that never had much of an evidence base and retrofit it to become ‘more evidence based’, and thus supported by police and communities. My short answer is: you can’t. The long answer is: but what you can do is take pieces of ideas, programs and practices that do have a strong evidence base, adopt and adapt them, test them and then come back and say, ‘under the umbrella of our Neighbourhood Watch program, we have Initiative x. The evidence base for Initiative X is (list 40 articles here) and our own research (a couple of rigorous studies looking at different aspects of the issue) shows that it reduced (burglaries, arsons, assaults) by y percentage. And, if you really want to be innovative, you could experiment (see last week’s blog) by modifying previous trials to test whether you can produce positive deterrent effects using some new elements. To illustrate: you could ask what works better for preventing near repeat burglaries: targeting homeowners who live near a burglarized home through warnings by text message, email, phone call or paper in the mailbox?

And that, my friends, is how you take an old dog and teach it a new trick.