U-M Tracker Measures Reliability of News on Facebook, Twitter

As the deadline for voting in the mid-term elections approaches, people now have access to a new online tool to help monitor the prevalence of fake news on the big social media sites. The University of Michigan Center for Social Media Responsibility, housed in the School of Information, created Iffy Quotient as a "platform health metric" to track "iffy" news in real time.

As of the day this story was written, Facebook had a score of 3.6 and Twitter 4.4 on the index. During the 2016 election, according to the Center, the Iffy Quotient had grown dramatically for both social media sites, up to 7.8 for Facebook on Nov. 5, 2016 and 5.6 for Twitter.

Iffy Quotient

Here's how the quotient is calculated:

1) NewsWhip, a social media engagement tracking firm, examines URLs on more than 400,000 sites daily and gathers data on which of those sites have engagements on Facebook and Twitter.

2) Iffy Quotient taps NewsWhip for the top 5,000 most popular URLs on the two social media platforms.

3) Next, Iffy Quotient classifies those domain names based on whether they have been flagged by Media Bias/Fact Check, an independent website that categorizes sources based on their reliability and bias.

4) The U-M service divides the URLs into three buckets:

  • "Iffy" for sites that appear on the "Questionable Sources" or "Conspiracy" lists;
  • "OK" for sites on any other list, such as "Left-Bias," "Right-Bias" or "Satire"; and
  • "Unknown," for sites on no list (a count that is inching downward with time).

For example, Fox News and the Drudge Report are classified as OK, with their "right bias," according to a whitepaper on the project, while Breitbart and TruthFeed are considered iffy because they use questionable sources.

5) The Iffy Quotient is calculated by dividing the URLs from iffy sites by the total number of sites to derive a percentage. The current Facebook count, for example, means that 3.6 percent of the top sites on that platform are sharing dubious information. However, to smooth the graph, the site applies a seven-day moving average. A "raw" quotient is computed for each date, but the index reports an average of the raw Iffy Quotients for the previous seven days.

A dashboard displays the Iffy Quotients dating back to 2016, and it will be updated regularly, university officials said.

Besides showing the bump up in misinformation around the 2016 election, the service also indicated that both Facebook and Twitter have made progress since early 2017 on efforts to dampen misinformation — but one (Facebook) has succeeded more than the other. While the sites' "Iffy coefficients" were about the same in most of 2018, Facebook's is currently a bit lower. In fact, the platform has returned to its early 2016 levels. Twitter, on the other hand, hasn't shrunk much, and it's nearly twice as high as it was in early 2016.

"By contrast with the current environment of accountability by 'gotcha' examples of bad outcomes, the Iffy Quotient tells us something about the overall performance of the platforms," said Paul Resnick, founder and acting director of the center, and associate dean for research at the School of Information, in a statement. "The platforms can track metrics internally with their own data, but hesitate to report them externally. By publishing continuously, we can provide accountability when things get worse and credibility for claims of progress."

The service isn't foolproof, Resnick warned. For example, even shaky sites sometimes publish solid information that will still be classified as unreliable. Also, NewsWhip can miss popular URLs, especially those from "newer fly-by-night sites," which may not be listed on Media Bias/Fact Check yet. Finally, NewsWhip's URLs include some unrelated to news or public affairs; those are expected to be filtered out in a future release of the tool.

Because of these limitations, the researchers involved in the project advised keeping the index in perspective. "Trends in the Iffy Quotient over time are meaningful," Resnick said. "The absolute number...not so much."

People interested in tracking dramatic changes in the index can sign up on the site to be notified. Registered users will also receive alerts when the Center adds new sources and other social platforms to the tool.

A whitepaper about the project is openly available on the university website.

About the Author

Dian Schaffhauser is a former senior contributing editor for 1105 Media's education publications THE Journal, Campus Technology and Spaces4Learning.

Featured

  • soft-edged digital blocks and AI imagery on a muted background

    OpenAI Launches GPT-4.1 with Upgrades in Coding, Context Processing, Efficiency

    OpenAI has announced GPT-4.1, offering stronger performance across software development, instruction following, and long-context comprehension.

  •  laptop on a clean desk with digital padlock icon on the screen

    Study: Data Privacy a Top Concern as Orgs Scale Up AI Agents

    As organizations race to integrate AI agents into their cloud operations and business workflows, they face a crucial reality: while enthusiasm is high, major adoption barriers remain, according to a new Cloudera report. Chief among them is the challenge of safeguarding sensitive data.

  • glowing digital brain made of blue circuitry hovers above multiple stylized clouds of interconnected network nodes against a dark, futuristic background

    Report: 85% of Organizations Are Using Some Form of AI

    Eighty-five percent of organizations today are leveraging some form of AI, according to the latest State of AI in the Cloud 2025 report from Wiz. While AI's role in innovation and disruption continues to expand, security vulnerabilities and governance challenges remain pressing concerns.

  • abstract geometric pattern of glowing interconnected triangles, hexagons, and circles in blue, gold, and white, spread across a dark navy-to-black gradient background

    OpenAI Unveils 'Operator' AI for Performing Web Tasks

    OpenAI has launched "Operator," an AI agent designed to perform web-based tasks autonomously using its own browser. Currently available as a research preview for Pro users in the United States, the tool aims to automate everyday activities such as filling out forms, ordering groceries, and even creating memes.