A recent email inquiry got me thinking about the question of ethics and the problem of confirmation bias. What my correspondent wanted to know was how to respond
to concerns raised by a colleague over a proposed randomized controlled trial of a policing program that purports to help people "at risk."
Quick refresher (from the CAN-SEBP ‘dictionary’):
[A randomized controlled trial] is a form of experimental research in which two groups are randomly assigned to one of two groups (the control group and the test/experimental group) and given a treatment to see if there were any noticeable effects among those within the treatment group that are not likely due to error or random chance. Randomization is used to eliminate the potential for researcher bias.
At the heart of the randomized controlled trial (RCT) is the decision – made randomly – to deny some study participants a treatment, whether that be a possible medical break-through or access to a new and untried service or program. We do this because, simply put, it’s one of the best methods for showing that the treatment did or did not live up to its expectations.
Although more popularly used in policing research in the U.K. and the U.S., RCTs remain unicorns in Canada – that is, they exist largely in our imaginations. There are many reasons for the lack of RCTs here, but the one I’m interested in tackling is that outlined by my correspondent: the belief, as expressed by his colleague, that by denying someone a treatment you risk harming them.
In responding to this perceived ethical quandary, I noted that the colleagues’ views are based on a little something called ‘confirmation bias.’ Confirmation bias is the human tendency to believe that something we find desirable is true and then to interpret and/or reinterpret all incoming data and evidence so as to confirm one’s belief (anyone who has suffered from an unrequited crush will understand this process perfectly). In research, this form of bias not only leads individuals to believe their intervention is producing the desired outcomes, but to resist any challenges to that belief. In my correspondent’s case, his colleague’s protest that they could harm someone by denying them treatment is based on her belief, unfounded as far as the research evidence shows, but firmly held by her, that the intervention is ‘saving lives.’
In short, it’s not much of an ethical quandary. However, it does raise another issue: what about the ethics of subjecting people to untried interventions based on nothing more than good intentions and personal beliefs? This is, I would argue, a particularly important question to ask when there's a possibility of a backfire effect: that the treatment may not help, may worsen people's conditions and/or create unintended negative effects for others. Don’t we have an ethical duty to know these things?
