Is that so? How to check the veracity of ‘facts’

The Internet is replete with information and opinions, many of them plain wrong. This is most true of the subtle advertising of opinions which takes place on social media such as Facebook, where all sorts of preposterous claims seem to be readily accepted, as if logging on switches off powers of critical thought.

This article is a brief examination of how you can determine whether information of this type is likely to be true and accurate. If you are happy to make such judgements by emotion alone, I have nothing to offer you. But if you want to assess what you should treat as gossip and rumour, I hope these notes will provide a catalyst to your thought and critical skills.

As a rolling example, I will try to get to the truth surrounding the extraordinary claim on Fox News (January 2015) by Steve Emerson that Birmingham was totally Muslim – a claim which Emerson later withdrew, and for which he apologised. However the claim was echoed by Nigel Farage of UKIP, so bears closer examination.

Central to the evaluation of any proffered fact or opinion are its source, rationale, and corroboration.

All stated facts which are important to any conclusion or opinion should include citation of their source. If no citation is given then the information can be no more than rumour: anyone can claim that 90% of those living in Birmingham are Muslims, but if they want any credibility they must state where that figure comes from. This is the basis for Wikipedia’s approach to its content, for example.

In the case of Emerson and Farage, neither provides any indication of the source of their claims, which could be checked. However good starting points might be the ethnic breakdown of the population of Birmingham and its distribution, the city’s demography, and criminal court cases (as they claimed that Sharia law, not statute law, ‘ruled the streets’).

In many cases, when you check back through the source chain to identify the origin of facts or information, you will also need to assess the reliability of the original source. Sometimes this can be a fascinating investigative task on its own, as it is surprising how many ‘think tanks’ and research groups turn out to be organs of, or strongly related to, organisations with vested interests such as political parties and campaign groups.

Oddly, the story of Muslim ghettos in Birmingham has cropped up on the Internet from time to time. For instance, the ‘Gatestone Institute International Policy Council’ ran a spookily similar account on 22 August 2011, which was then republished in the New Media Journal on 29 January 2013. The same author, Soeren Kern, revisited the question ‘in depth’ on 3 February 2015, following the Fox News stories. This includes alleged local percentages of Muslim residents taken from 2011 ‘census data’, but lacks any further citation of sources. Other figures given appear to be predictions from quite old (2007) studies. A very thorough and well-referenced Wikipedia article on the demography of Birmingham provides properly referenced figures from the 2001 and 2011 official censuses, with maps, covering both ethnicities and religions.

Even official information should be subject to the same rigorous checks, as most governments are only too happy to manipulate information to their own advantage. A healthy dose of scepticism is very important, and this is where I find Wikipedia to be weakest: many of the sources which it cites are not the primary sources, and many are highly suspect. Sometimes it would be preferable to rely on the opinion of an expert without any further source chain, than a source chain which turns out to be flimsy or bogus.

The least trustworthy sources of information are generally the popular press (and press releases to it), surveys of opinions, and anything financial or economic. If you can drill down no further than a popular press item, it is best to relegate that information as mere rumour. Surveying opinions is extremely difficult, and many such surveys are no better than the tabloid press; be particularly wary of any survey which has been conducted online, such as website polls and anything using SurveyMonkey. Whilst it is possible to carry out good work using those tools, most are flawed through and through.

Financial and economic figures are particularly difficult to use, because there are so many who make careers out of massaging them to please their paymasters. Companies employ accounts to minimise their tax burdens, and arrange their accounts to benefit their business aims. Although such practices should be uncovered by auditors, as they too are paid by the company you should not expect them to be particularly objective or critical either. Official financial figures are notoriously controversial, and again there are many very bright people whose sole mission is to ensure that what you are presented with meets the satisfaction of whoever pays their salary.

One good way to obtain and present a fact with a degree of uncertainty is to obtain several estimates and give their range. So for example if you wanted to know the average annual wage of an adult in the UK, you might obtain different estimates from several sources, and give the range. The best estimator of the real value could be the average, or preferably the middle value (median) of the range.

Census data are normally of quite high quality, but have known limitations. The figures detailed on Wikipedia, sourced from the 2011 census, give 21.85% of the total population in Birmingham as Muslim, and 46.07% as being Christian. In actual numbers, these are 234,411 and 494,358 respectively. However when reported by Mark Howarth for the Daily Mail on 14 September 2014, the totals of adults and children were 245,244 and 507,302 respectively, even though Howarth claimed to be using the same 2011 census data.

Religion and similar self-reported data are inherently wobbly too. If you totalled Christian church attendances on any given Sunday in the Birmingham area, you would be very unlikely to arrive at even half the 494,358 who reported on their census forms that they are Christian. However obtaining more specific information about numbers of practising Muslims is much harder, and such surveys are likely to be of lower reliability than the census data.

Even when a source appears reliable, a single source can be wrong. To increase confidence you need to obtain corroborating sources which can be seen to be independent of one another, particularly if they appear in different media, e.g. website and printed book. Because information flows so freely online, you should be wary of using more than one online source; it is very likely that a second online source has ultimately cribbed its figures from the same primary source, so there would be no safety in numbers. Tracing the source chain can help clarify that.

In an ideal world, then, all important facts would be traced back to reliable original sources, and would be corroborated by independent sources.

Then you have to use those facts to build the opinion or conclusion in a rational way. A common error here is in the expression of change: just as a shop offering items at ‘50% off’ may not actually mean at half price, expressing percentage change is often done incorrectly. If this week I sell 100 copies of my book, and next week 120, then the percentage rise in sales is 20% – that is the change (120 – 100) divided by the first value (100), all multiplied by 100 to convert into percentage. If next week’s sales are 80, then the percentage fall in sales is 20%.

Although rates of change are interesting, you should not allow them to detract from the overall magnitude of the underlying figures. Selling an extra 20 books should not be too difficult; selling 20% more when the first week sales are a million, is likely to be much harder.

Those playing fast and furious with figures are often skilled at selecting comparisons carefully. Large corporations like Apple usually compare equivalent quarterly figures for successive years, because of the seasonality in sales, but these can be misleading if they ship new products at different times in those years. A suspect comparison could be obvious, such as a beachwear firm comparing sales in the first calendar quarter with those in the second or third, or more subtle, when they have changed the basis of calculation from one quarter to the next, perhaps. Governments are particularly adept at changing the basis for key figures, and claiming that it is an improvement.

Arguments and discussion are also prone to basic errors of logic and rhetoric. If I visit Birmingham and (assuming that I can enter one of the mythical ‘no-go’ areas!) the first 100 people that I see on the streets are overtly Muslim, I cannot conclude that all inhabitants of Birmingham are Muslim. There are some basic logical tests which you can apply, and many good books on the science of logic and argumentation which would be worth reading. But one essential test must be whether each step in the argument, and the overall case, makes common sense. Test these against other information about the subject, and the opinions of those experienced in the field.

Map of 2011 census distributions of self-declared Christians in Birmingham. By SkateTier under CC BY-SA 4.0, via Wikimedia Commons. Does this show Christian 'ghettos', or local communities?
Map of 2011 census distributions of self-declared Christians in Birmingham. By SkateTier under CC BY-SA 4.0, via Wikimedia Commons. Does this show Christian ‘ghettos’, or local communities?

One very obvious observation about the distribution maps for religion in Birmingham displayed on Wikipedia is that it is not just Muslims who appear concentrated in certain areas of the city. Jewish, Hindu, Sikh, Buddhist, and even Christian populations are very unevenly spread. Should any of the more densely aggregated religious groups be claimed to form a ghetto, or are they simply communities? Look at similar maps for cities like Plymouth or Salisbury, and could you not there ask very different questions, perhaps? It is also interesting to note that the Gatestone articles quote a group alleged to be trying to impose militant Sharia law, whose members have been subjected to ASBO procedures; this evidence surely flies in the face of the claims.

Remember that each step in an argument must remain proportionate: a common fallacy when considering medical and similar matters is that something should apply to everyone. When discussing the merits of immunisation, for example, a commentator may establish that as many as 10% of those immunised have little or no immunity as a result. Ask yourself what that figure might be if there was no immunisation at all (much closer to 100%), and accept that humans are not Cartesian systems but have great individual variation.

One particularly astute UK radio programme which dares to question many of the figures that are bandied about is More or Less, and its presenter Tim Hartford has his own excellent website on which he debunks modern statistical myths.

This sounds a great deal to have to do every time that you spot a posting that rankles or begs questions. In most cases, the effort involved is tiny, as the case being put falls apart the moment that you start checking sources. You can then have the pleasure of laying this flaw bare, and seeing spectacular displays of pique as others realise their stupidity. Whatever you do, please think carefully before clicking on Like.