We examine the claim that an independent Scotland would have to ‘join the queue’ to get into the European Union.
For more see: https://theferret.scot/independent-scotland-queue-eu-membership/
Has the SNP overseen the biggest in-work poverty increase since devolution?
Great to see this project kicking off!
Have you thought about how you’d represent “technically true, but lacks context” on your truth scale? “Misrepresentative” or “Twisting the truth” might be briefer ways of putting it. This is one aspect I worry about on other fact checking websites, which in focusing narrowly on truth/falsity ignore the larger argument.
A good current example would be the life expectancy story that Wings Over Scotland highlighted over the weekend: that Scottish life expectancy has “stalled” for the first time in 150 years. While technically true, this lacks a UK context wherein UK life expectancy is actually dropping, so the narrow focus of the story allows the author to present as a negative something that could be argued as a success in fighting against a greater tide. A related approach is where a particular negative statistic is seized upon from a bouquet of positives — quite common among health and economics journalism, where bundles of figures tend to all come out at once…
We put together our scale and fact-checking methodology after research and contact with other fact- checking groups, and with reference to the International Fact Checkers code of principles.
Our scale goes has six points, from True to FFS! which are explained in our Ferret Fact Service methodology page, which will be linked to each Fact Check.
Your point about how to evaluate claims which are lacking in context or omit certain facts is an important one which we thought a lot about when developing the project.
We rate our fact checks on a six-point scale:
True – The claim is accurate, and has not left out mitigating factors or important context
Mostly True – The claim is still true, but requires further information or clarification to create a full picture
Half True – The claim is somewhat or partially accurate, but leaves out crucial information or is selectively taken out of context
Mostly False – The claim may contain a kernel of truth but leaves out facts which lead one to a different impression.
False – The claim is incorrect, not accurate.
FFS! (For Facts’ Sake) – The claim is baseless, ridiculous and/or logically impossible!
This scale allows to take into account that many claims may not be entirely true or false, but can still be misleading.
Selective use of statistics is a very important area to fact check, and will definitely feature in future checks as the FFS develops.
Alastair from Ferret Fact Service
Thanks for the explanation Alastair! As you work on the system, I guess I might be inclined to add two items to your final rating graphic to reflect this excellent information:
- Make the whole rating image a clickable link to your methodology page, ideally to the paragraph that explains the exact rating you’ve given.
- Add mouseovers to each of the markings on your scale, so people can quickly see the brief explanations you’ve given above, just by hovering over them on the graphic.
Really cool there’s an International Fact Checkers’ Code of Principles
In a previous life I ran a fact-checking site, and a “factcheck watch,” where I followed the famous US-based fact checkers (Politifact, The Washington Post) and detailed how they distort the truth to fit a narrative. I am interested to see how The Ferret handles its fact checking, so please indulge this observation.
One of the ways the fact checkers distort their results is by reducing complex answers to a simple scale, which hides the complexity of the issue. Then, inevitably, they ask questions that suit the scale, and torture the analysis to fit the scale.
In this case, a decent investigation is ruined by a similar desire to reduce a complex issue to fit the scale. For most readers the real issue is not about the “queue” but about the process, and you do a good job of explaining that. It is clear that Scotland will be out of the EU and have many requirements to get back in. But by focussing on the question of the “queue” you can then say that the result is ‘false’, misleading readers who might not read the full analysis.
While it seems you are aware of the issue, you should be wary of The Fact Checkers Code of Principles. Those ‘principles’ have led to a situation where less than 30% of people in the US trust fact checkers. Those principles might make the fact-checking community happy, but they don’t give confidence to the reader. You’d be better to abandon the scale and simply give multiple points as answers, lest you become slaves to the scale.
I’d have expected at least an acknowledgement of the fact that no country has ever left the EU before and in the event of a successful independence vote, prior to March 2019, politicians may seek to negotiate a continuation of the current EU citizen rights held by Scottish residents. But caveated that the “holding tank” has been the subject of speculation and until the event becomes reality, it is not possible to offer a conclusion, either way…
A little late (must check my notifications), but yes, I strongly agree with this. Another case in point is the recent fact check on in-work poverty (https://theferret.scot/snp-overseen-biggest-work-poverty-increase-since-devolution/). I think it’s often legitimately argued that the unionist parties delight in complaining about SNP “failures” in policy areas over which the Scottish parliament has less than total control — indeed, frequently policy areas which are entirely reserved, for example energy or telecommunications.
I think a fact check on in-work poverty “overseen by the SNP” is ignoring important context if it doesn’t acknowledge that vast swathes of trade and industry and social security powers are reserved. I note that some of the comments on that fact check have picked up on this.
It’s still early days, but I think reconsideration of the simple linear scale may yet be in order.