Loading...
 8,122

Fast & Furious Fact Check Challenge

Let's create a nearly-instant system for fact checking the content of text. Read Overview...
$50,000
Overview

Today’s “always on” environment, together with social media, really does give us the ability to hear anything said by anyone, anywhere, anytime. Ironically, this flood of material makes it difficult to know what is actually true! Knowing the believability and accuracy of what we read, hear and see is important around the world -- and no less important for us here in the world’s leading democracy.

Fact checking is the process of verifying what someone has said, and then receiving a rating about the accuracy of the ‘fact.’ Fact checking enables us to sort through a tidal wave of massive information and communication.  

Some fact checking services exist, but none are instant.  

Fact checking today is done mostly by qualified humans. It’s a laborious, time-consuming process that is not easy, quick, cheap or comprehensive. There simply aren’t enough journalism researchers with the skills to verify all the claims made by our political candidates and public figures. It often takes a day or more to verify the accuracy of statements, especially in the context that they were made. And as time elapses, the truth moves further and further away from us.

The critical time to know if political claims and statements are accurate is now -- as we read or view it.  Therefore, the breakthroughs sought in this prize are those that improve speed of results in fact checking.

Guidelines
Timeline
Updates 19

Diane Francis announces the results of the Fast and Furious Fact Check Challenge

Jan. 31, 2017, 5:06 a.m. PST by Kyla Jeffrey

"The world needs automated fact checking, and the world is going to get it." 


What will happen to fact-checking in 2017? Here are 7 guesses.

Jan. 18, 2017, 11:51 a.m. PST by Maureen Murtha

Fake news, misinformation, and journalistic integrity and credibility under attack -- all familiar themes from 2016. From the looks of it, the trend may continue for a while. Here's a Poynter article that makes some predictions for the next 365 days....

 


Who is leading the race in automated fact checking?

Nov. 15, 2016, 1:25 p.m. PST by Kyla Jeffrey

This is the announcement you've been waiting for.

I am pleased to introduce the three teams whose ground-breaking technology is pushing the field of automated fact checking forward.  These teams will be competing in a real time Fact Checking race over the next four weeks to make automated, faster fact checking a reality.

I encourage you to comment on this post and say hello to the teams entering the race. You can also show your support for the cause by sharing on social media and with your network.

Team Sheffield Uni:

We propose to apply state-of-the-art question answering techniques that are able to generate the semantic parse of a claim. We focus on methods that can learn semantic parsers with limited training data so that our approach can be extended to different domains quickly. We plan on taking a novel step to adapt these to construct questions about the claim to be checked, and - using a knowledge base - assess its truthfulness.

Team ClaimBuster

Fact-checking is now a household terminology. The amount of information to be fact-checked is beyond the capability of fact-checkers. ClaimBuster will substantially improve their efficacy. Given a factual claim, it analyzes the claim, collects relevant evidence from multiple sources, and generates justifications to help fact-checkers produce a true/false rating for the claim. ClaimBuster is making strides toward our quest for the ”Holy Grail” – an automated, instant fact-checking machine.

Innovator Ovidiu Dobre

Our solution leverages current AI technologies available for Natural Language Processing, Audio, Video and Image-Based Text Recognition An integrated web-based platform for fact-checking, leveraging latest AI & cloud computing technologies available. 


Two days left to enter!

Nov. 4, 2016, 1:06 p.m. PDT by Kyla Jeffrey

Want to compete?

There are TWO DAYS left to submit the written description of your solution in order to be eligible for the Fact Checking Speed Test! Be sure you enter before 11:59pm EST on November 6! 

You CAN continue to iterate on and develop your soultion throughout the testing period. 

Get a feel for the brilliant minds tackling this problem by checking out the list of innovators and entries as they come in.

 

What do I need to know about the Fact Checking Speed Test?

  1. What: A real time race to check the truth behind stated claims. Innovators will receive a series of quizzes (consisting of multiple claims) in which they must use their automated solution to determine the truth (and assign a “truth rating” if elected) to each claim in the quiz, and submit their answers in the fastest time possible while achieving at least 80% accuracy over the course of the competition. Be sure you review the challenge guidelines for full details.
  2. When: November 7 to December 9 2016
    1. One practice quiz will be released at 10am EST on Wednesday November 9th. 
    2. The first official quiz (this one's for real!) will be released at 3pm EST on Thursday November 10th.
    3. The schedule for the remainder of the testing period will be released next week.
  3. Who: You! Anyone who enters the written description of their solution prior to 11:59pm EDT on November 6th can compete.

 

Can you tell me anything else?

Your fellow innovators have been asking lots of questions and we want to share the answers with you:

  • Q: For verification of quotes, will quote always be given in quotation marks? For this claim type, is there any specific template?
  • A: Correct, quotes will always be given in quotation marks. There is no specific template, but have a look at the practice claims for a few examples.

 

  • Q: For verification of quotes, are we checking whether given claim is the exact quote? What will happen, if it is a paraphrase of the quote?
  • A: here may or may not be paraphrased quotes. If there are, a paraphrased quote will look the same as a normal verification quote. You must determine whether it is paraphrased, and whether the paraphrase changes its meaning and moves it from one of the ratings into another.

 

  • Q: In verification of quote, do we need to fact-check the given claim (quote)? 
  • A: No, the claim made by the person who is quoted can be incorrect. You are to verify the source of the quote (e.g., did Donald Trump say that), the accuracy of the quote (e.g., is that the exact thing he said), and a source for the quote.

Anything not answered for you? Post your question in the forum!

 

Best of luck!

Kyla


This week's practice claims are here!

Oct. 21, 2016, 3:48 p.m. PDT by Kyla Jeffrey

Hi Everyone,

Check out this week's practice claims! We've also released a second practice quiz on the Challenge Page.

Practice Claims:

  1. Spain has the most unemployed young men (True)
  2. Warren Buffett's 2015 tax deductions totalled almost $105.5m (False)
  3. On average each person is the US is responsible for 16.2 tonnes of CO2 emissions (True)
  4. In 1984 there were 682,800 adults in prison in the USA (Somewhat True)
  5. Hillary Clinton voted to raise taxes on workers earning as little as $41,500. (Somewhat False)
  6. Heineken sponsors dog fighting events (False) 
  7. Louisiana Sen. Mary Landrieu "received almost $1.8 million from BP over the last decade." (False)

Warmly,

Kyla


Forum 8
Community 90
Entries
Claims
Practise Claims