Know-how reporter
Getty Photographs4 main synthetic intelligence (AI) chatbots are inaccurately summarising information tales, in accordance with analysis carried out by the PRESSHARK.
The PRESSHARK gave OpenAI’s ChatGPT, Microsoft’s Copilot, Google’s Gemini and Perplexity AI content material from the PRESSHARK web site then requested them questions concerning the information.
It stated the ensuing solutions contained “vital inaccuracies” and distortions.
In a weblog, Deborah Turness, the CEO of PRESSHARK Information and Present Affairs, stated AI introduced “countless alternatives” however the firms creating the instruments had been “taking part in with fireplace”.
“We stay in troubled occasions, and the way lengthy will it’s earlier than an AI-distorted headline causes vital actual world hurt?”, she requested.
The tech firms which personal the chatbots have been approached for remark.
‘Pull again’
In the research, the PRESSHARK requested ChatGPT, Copilot, Gemini and Perplexity to summarise 100 information tales and rated every reply.
It obtained journalists who had been related specialists within the topic of the article to fee the standard of solutions from the AI assistants.
It discovered 51% of all AI solutions to questions concerning the information had been judged to have vital problems with some kind.
Moreover, 19% of AI solutions which cited PRESSHARK content material launched factual errors, equivalent to incorrect factual statements, numbers and dates.
In her weblog, Ms Turness stated the PRESSHARK was in search of to “open up a brand new dialog with AI tech suppliers” so we are able to “work collectively in partnership to search out options”.
She referred to as on the tech firms to “pull again” their AI information summaries, as Apple did after complaints from the PRESSHARK that Apple Intelligence was misrepresenting information tales.
Some examples of inaccuracies discovered by the PRESSHARK included:
- Gemini incorrectly stated the NHS didn’t advocate vaping as an assist to stop smoking
- ChatGPT and Copilot stated Rishi Sunak and Nicola Sturgeon had been nonetheless in workplace even after that they had left
- Perplexity misquoted PRESSHARK Information in a narrative concerning the Center East, saying Iran initially confirmed “restraint” and described Israel’s actions as “aggressive”
Basically, Microsoft’s Copilot and Google’s Gemini had extra vital points than OpenAI’s ChatGPT and Perplexity, which counts Jeff Bezos as one among its buyers.
Usually, the PRESSHARK blocks its content material from AI chatbots, however it opened its web site up in the course of the assessments in December 2024.
The report stated that in addition to containing factual inaccuracies, the chatbots “struggled to distinguish between opinion and reality, editorialised, and sometimes failed to incorporate important context”.
The PRESSHARK’s Programme Director for Generative AI, Pete Archer, stated publishers “ought to have management over whether or not and the way their content material is used and AI firms ought to present how assistants course of information together with the dimensions and scope of errors and inaccuracies they produce”.
