Identifying checkable claims with machine learning

Mevan Babakar

19 Mar 2019, 4:30 p.m.
George Marshall Room (Chateau)

To assist in the process of fact-checking, independent charity Full Fact takes on the substantial task of automated claim detection, along with academic partners.

Before determining the veracity of a claim, any fact-checker must identify the set of sentences, often from within a longer text, that can feasibly be checked for veracity. Using machine learning and leveraging the expertise of professional fact-checkers, Full Fact have developed a tool which can separate sentences containing claims from those that do not. This accelerates the process of fact-checking, and has provided useful in the organisation's own research.

In addition, the organisation has developed an annotation schema and a benchmark for automated claim detection that is more consistent across time, topics and annotators than previous approaches. As well as the obvious benefits for fact-checkers, Full Fact foresee uses for journalists and researchers to arrive at better stories or better understanding of how stories may be misrepresented.

Mevan builds on previous TICTeC presentations of Full Fact's work in automated fact-checking with this showcase of an approach that achieves a relative improvement of more than 5% over existing the state-of-the-art methods.

Read more in Full Fact’s paper, Towards Automated Factchecking: Developing an Annotation Schema and Benchmark for Consistent Automated Claim Detection.


Identifying checkable claims with machine learning (Mevan Babakar, Full Fact, UK) from mysociety