How ranking may be polarising society

Facebook News Feeds can be infuriating at times, just as Google search results, and Amazon recommendations for purchase. Each uses machine learning, a form of artificial intelligence (AI), to make choices for you, and those choices often do not come up to expectations.

Google uses what was the PageRank algorithm, now buried in amongst a host of other smartness, and almost inscrutable. Facebook’s EdgeRank has gone the same way: what used to be an algorithm based on three factors (user affinity, weighting of content, time decay) is now claimed to take into account more than 100,000 factors in all. Amazon’s recommendations are also fairly completely opaque.

All three – like other machine learning systems – attempt to come up with content and products which you will like, based in part on what you have just been liking. In the case of Facebook, this means that you will be shown articles which the system thinks that you will like. The psychological effect, which is the complement of the AI wizardry, is reinforcement learning.

One example of this is the recent UK referendum, and opinions shared by friends. Of my friends on Facebook, the great majority clearly intended to vote one way; only two made it clear that they held the opposite opinion. Because I tended to like those articles posted by those friends of similar view, my News Feed filled up with those articles. I seldom saw the articles posted by my two friends who were of the opposite persuasion. So I read a lot more content which strengthened my resolve to vote the way that I did.

Back in the real world, had I been in a workplace with many others, or in an extended family, my contacts would have tended to be more representative of the population as a whole. In other words, as the numbers rose and they were selected from a more random cross-section of the public as a whole, the proportion of those agreeing with me would have tended towards the 52% – 48% split recorded in the referendum. I would therefore have been more exposed to views differing from mine. This would have been modified by grouping, of course, but there would not have been an AI engine carefully manipulating my exposure to keep me clicking on Like.

Amazon recommendations are an excellent example of this self-reinforcing selection, and how it can go wrong. A couple of months ago, I had to buy an audio CD for a relative, of music which I would never wish to hear. Ever since then, that single purchase of a very popular CD has poisoned Amazon’s recommendations for me, and it persists in inviting me to buy even more CDs which I detest with a vengeance. I have tried telling the AI engine that I am not interested in those related products, but all it does is show me more of the same. It can take a long time for that sort of poisoned choice to work its way out of the system.

The two problems at the heart of these phenomena are that machine learning can only use historical data, and that the goal is to get me to like or purchase.

Amazon illustrated well the shortcomings of historical data, as it does not know my purchasing intentions. Each year, I buy presents for my grandson’s birthday. As he gets older, my choices change. What at first were Duplo sets became Lego sets, then littleBits kits, and so on. Amazon’s algorithm does not spot this pattern, because it is quite subtle, so for the weeks following each birthday, I have to ignore recommendations based on the last birthday’s presents, but next year Amazon fails to anticipate the event.

This approach works quite well for products which tend to have broad appeal, like popular fiction books. People who enjoy detective stories are usually very happy to be offered the more popular of the many thousands of detective stories available. It breaks down with more specialist non-fiction: a while back, I was buying a lot of books about Masaccio and his time, now I am concentrating more on William Blake, but Amazon keeps trying to get me to buy books about the Italian Renaissance more generally.

I dread to think how Facebook News Feeds would have coped (or not) with someone who changed their voting intention shortly before the referendum. Presumably they would have done their darndest to persuade them to recant.

The goal set in an AI engine is even more critical. If someone starts with a mild preference for something, serving them ever-increasing content about that will either turn them off altogether (as with Amazon preferences), or reinforce. In the latter case, the AI engine is setting up a vicious circle, a positive-feedback loop, which in control terms is an unstable system which will eventually go wrong. Normal interactions in a more open and mixed social circle tend to result in negative-feedback loops, where the group buffers out any tendency to extremes.

In today’s machine learning ranking systems, there seems little to buffer the extreme, and every encouragement to reinforce the dangerous.