Building Fairness Feedback Loops into AI Systems

This is a simple idea, but not one I’ve heard discussed before. It’s worth raising because I see the move to more ethical AI needing to happen on different levels. So while many organisations focus on the macro level of system fairness, another level that needs to be addressed is the micro level.

We saw something similar happening in the associated field of price walking. The FCA looked at the fairness of insurance pricing through a macro lens, and saw only minor problems. Citizens Advice looked at the same situation through a micro lens, and saw a big problem. And they populated their micro lens with lots of individual case data fed to their data science team from their many advisers working across the country.

Positioning the Micro Lens

So how can something like this micro lens be created for the decisions being output by sophisticated AI systems? One way would be to look at the interface between the system and the human that's in the loop. In other words, the person communicating with the customer. And while firms like to talk about automating such communications, at the moment they’re mainly working with humans supported by AI systems.

An opportunity exists then for those interfaces to engage with the insurance person and gain their regular, micro level feedback around the decisions they’re being supported in making and communicating to the customer, in relation to, say, the handling of a claim.

This feedback sought could be very simple. Something along the lines perhaps of seeking a graded view on how fair they felt the claims decision they’ve just been discussing was. There are numerous possible iterations for how this could be worded, drawn from how people can reflect on the ethical side of the decisions they’re making.

The feedback being sought requires of course a judgement to be converted into a score, but the downside of this would seem to be well offset by the quick and spontaneous feedback received.

Too Subjective?

Some of you will of course point out that it is all very subjective and variable. After all, one person’s fair is another person’s unfair. Yet any quick fire survey is like this, but works, so long as you collect enough inputs. After all, the claims person was being asked not whether the decision was fair, but whether they felt the decision was fair. You’re asking for their subjective opinion.

Another possible issue could be that this is an opinion being sought not in relation to a corporate definition of fair, but in relation to a personal definition of fair. What if the two are different?

There will be differences of course, in both directions, but overall they are likely to have more in common than not. And to be honest, you cannot ask a claims handler to only judge a situation through a corporate lens and not though their personal lens. The latter kicks in more quickly and is given more weight, for it is more familiar and put to use more often than its corporate equivalent. Those two things are immensely influential.  

Let’s quickly look at what the firm would do with this micro feedback. My relatively simple brain imagines some form of heat map across the decision framework that the AI system would be working within. It would show where problems are thought to be occurring and highlights issues that need to be reviewed.

Has this sort of system feedback arrangement been put to use before? Perhaps, but I’ve not heard of it. Please enlighten me if you have!

What I hear a lot about are systems nudging humans to make more optimal decisions. It doesn’t seem like rocket science to incorporate a feedback loop, whereby the humans are nudging the systems to make fairer decisions.