CJR study shows AI search services misinform users and ignore publisher exclusion requests.
See full article...
See full article...
AI search engines give incorrect answers at an alarming 60% rate, study says
- Thread starter JournalBot
- Start date
On that particular point:
It is beyond dispute to assert that Grok, doesn't. It follows the Muskian convention of explicitly NOT doing what his products' names describe, with perhaps the exception of Boring, though in that case it is more of an accurate adjective of its results than a verb of action and accomplishment.
Upvote
8
(8
/
0)
Of course in this context, "updated" translates as "Congressional revision or repeal of existing law or defunding of its enforcement in response to perfectly legal corporate bribery."
Upvote
3
(3
/
0)
Speaking of artificial stupidity, never has it been clearer that every human endeavor is ultimately a metaphor for humanity itself. We can create nothing except in our own image.
Publishers, editors, peer-review, academic competition, and professional criticism were once a significant part of humanity's collective intelligence. The internet having eliminated all these and more, it's now anyone's guess which will kill us first: willful destruction of our own habitats or digitally enforced idiocy.
Publishers, editors, peer-review, academic competition, and professional criticism were once a significant part of humanity's collective intelligence. The internet having eliminated all these and more, it's now anyone's guess which will kill us first: willful destruction of our own habitats or digitally enforced idiocy.
Upvote
6
(6
/
0)
It's still really, really uncertain, but if you're really into reading the tea leaves there are some possible signs:
1) Microsoft starting to pull back on data center build-out, suggesting that it thinks either A) its current compute capacity is sufficient for current and future demand, or that it's overbuilt. Even in the best-case scenario of "Deepseek made MS realize they can run LLMs much more efficiently," this still means they're jettisoning OpenAI, who has shown zero interest in adopting Deepseek's approach. This is more likely considering that...
2) OpenAI has turned to Softbank for continuing investment, and Softbank is borrowing like a gambling addict trying to claw back from the penny slots to the high-roller table in order to fund its investment in OpenAI (borrowing to invest in OpenAI itself, and borrowing to invest in data centers for OpenAI to use).
3) Oh, and this is ignoring nVidia, who made a fortune selling shovels for the gold rush and may soon find itself getting rugpulled.
4) There's also a company called CoreWeave whose entire purpose is renting out GPU data centers for AI, and whose biggest customer is Microsoft. They were the second biggest owners of data centers, and were already unprofitable, and then Microsoft canceled a bunch of contacts (see 1).
Granted, it could all go the other way and shake out in the end, who knows? It's also unlikely to "pop" in the sense of Google, Microsoft, nVidia, Meta, etc. going out of business, or LLMs going away entirely, since they do have some use cases. But I simply don't see OpenAI, Anthropic, Perplexity, or other companies whose entire business is AI surviving long-term.
Upvote
17
(17
/
0)
The Sheep Look Up
Ars Centurion
And let's not forget that Masayoshi Son has backed some of the stupidest sucker bets to come down the VC pipe. Something something Wirecard...
Upvote
12
(12
/
0)
As a former developer and high-level manager at a development firm I’m so sick and tired and utterly disgusted by the attitudes of these companies. I fought like hell to keep people (most especially the CEO) from going down the path that these morally corrupt shitstains continue to follow.
It’s depressing that so many people are lazy and completely lacking common sense and critical thinking skills to not see the totality of the sham that all of these companies and their boosters are foisting on the public.
It’s depressing that so many people are lazy and completely lacking common sense and critical thinking skills to not see the totality of the sham that all of these companies and their boosters are foisting on the public.
Upvote
13
(13
/
0)
Burying the bar there. We should be aiming for something where if it doesn't "know", it says that it doesn't "know", instead of making stuff up out of whole cloth. Not accepting something just because it lies less than a known liar who lies as naturally as he breathes (if not more so).
Upvote
6
(6
/
0)
Significantly. Google’s AI search can’t even give you a list of winged dinosaurs or tell you which reptiles are/were warm-blooded. Even worse. It’s confident in its wrongness.
Upvote
5
(5
/
0)
I may be having a Friday morning moment, but do the graphs not show the opposite?
Edit: I'm having a Friday morning moment. I read that as "correct information" rather that "incorrect information".
Upvote
1
(1
/
0)
I think you might have "whoooshed" yourself mate. What is your use case where search is 75% wrong? Are you searching for a wendigo or bigfoot on google?
Upvote
8
(9
/
-1)
If these models would just have the ability to decline to answer or come with a confidence score instead of confabulating, they would be a lot less useless. Do the underlying technology even allow that as a possibility?
Upvote
4
(4
/
0)
Fake citations is still the easiest way to catch and prove students are writing research papers in my class with AI. Like most AI crap, it passes a quick skim inspection, but is still way easier to prove to student disciplinary committees than other "maybe" AI generated text. That said, I worry that these "look there are citations" AI output will fool more people into believing fake science info because very few people go to check original citations.
Upvote
10
(10
/
0)
This is excellent and useful information if you are a food photographer. That 1/8th cup of glue makes your pizza photo look so much better.
Upvote
4
(4
/
0)
When you consider that a pure guess should be wrong only 50% of the time.
Upvote
-1
(3
/
-4)
Often on LLM articles, I see people saying what amounts to "I can't see an immediate use for me, therefore they are useless to everyone". Your post seems to be more "I don't use LLMs this way so no one else does." When LLMs are being promoted as research tools, being able to identify sources and provide links is important.
Last edited:
Upvote
11
(11
/
0)
I think this is unfair. I am plenty sceptical of the results and I've found that pretty much 100% of the time when I have deduced that the response to a prompt is incorrect, even when I did not already know the answer going in, I have been right about that. And even on occasions where I couldn't immediately tell the answer was wrong, as I always follow the "citations" to verify the information, a good chunk of the time the LLM still got it wrong at least in some way (such as, for example, giving battery replacement instructions for a different model than the one requested because the manufacturer provides both models' instructions on a single page and the chatbot has incorrectly parsed it).
But I don't really blame anyone else, especially less technical users, for trusting the tool they are provided. Should they? Of course not, but there's no earthly reason why everyone should be aware of the details of how these things work. Everyone has different priorities and interests. I do my best to impress upon those around me who are less informed and engaged with these things what the limitations and risks are, but I do not shame them for not already knowing. Looking at it from their point of view, they have been presented with a service that purports to provide information using a natural language interface.
And it is, let's be honest, superficially very impressive to observe what they do. People generally are not used to computers being able to do things like this in this specific way. It's easy to see how that would engender trust and appreciation from users, especially those less technically inclined who do not have the frame of reference to understand what's going on under the hood. And, again, there is no reason why they should either - it's fine and even good to be interested in how tech works, but it should not be obligatory in order to use it for its most basic purposes. We don't expect every driver to be able to to explain in detail how an internal combustion engine works or be capable of constructing precise suspension geometry.
The shame should be directed instead at the purveyors of this technology who are overpromising and underdelivering, setting their users up to fail by misleading them into using and trusting their services for financial gain. Howard is blaming some of the victims. The blame, and the shame, rightly belongs to OpenAI, Perplexity, Anthropic, Microsoft, Google et al, along with their sycophants and boosters - particularly in the media.
Upvote
8
(8
/
0)
lol. There is an interesting point though. First, training data. Much of online commentary is authoritative sounding bullshit, good faith people who don't know tend not to try and answer questions. Plus, no one publishes text books filled with "I don't know" as the answer to problems or publishes articles without a view point or narrative. Then there are the commercial drivers, you don't want your chatbot to appear less capable than the competition by treating 'IDK' as a good output. Finally the tech itself seems to be exploiting the relationships between words and phrases, not the things or concepts (such as correct and incorrect answers) those words and phrases evoke in a human mind.
I'm sure there is room for LLMs to improve outputs, but any reporting on realistic avenues for experimentation seems to be drowned out by marketing guff. "It's a magic mind in a box, just make the box bigger!".
Upvote
6
(6
/
0)
I'm cynical enough not to think this is the tech's problem. I've met too many people who just don't care if information is true or not. Is it a simple answer? Does it keep teacher / boss off my back? Does it confirm my biases? Does it give me a feeling of security? Is it something I can leverage for advantage? etc.
Upvote
5
(5
/
0)
The search engine the modern web deserves, really. We thought we hit enshittification bedrock but the hole can still get deeper.
Upvote
1
(1
/
0)
In an article partly about absent links to sources, I feel compelled to append Ed Zitron's article to your post.
Upvote
6
(6
/
0)
I know I'm setting myself up for some flaming here... but I think this headline is a bit misleading.
Yes the study highlights a critical flaw in AI search engines' ability to provide accurate source attribution. This has implications for news integrity and publisher control.
But I think that, for most people, the headline suggests that AI search engines site incorrect or fabricated sources. When it's more talking about that, given a text, they can't identify the correct source.
In terms of the research itself, it's a bit disappointing that they didn't also test the underlying search engines. This would have been an interesting comparison. My guess is that the first source of listed when you copy a piece of article text into a search engine is probably not the correct one given how may times things are copied. Most likely the AI layer on top of those searches has not been set up to specifically try to find the ultimate source. But that's a less sexy headline.
As with most scenarios it's probably good to consider an AI to be a well read, articulate temp worker that will sometimes make things up because they must write at a fixed word per minute rate.
Yes the study highlights a critical flaw in AI search engines' ability to provide accurate source attribution. This has implications for news integrity and publisher control.
But I think that, for most people, the headline suggests that AI search engines site incorrect or fabricated sources. When it's more talking about that, given a text, they can't identify the correct source.
In terms of the research itself, it's a bit disappointing that they didn't also test the underlying search engines. This would have been an interesting comparison. My guess is that the first source of listed when you copy a piece of article text into a search engine is probably not the correct one given how may times things are copied. Most likely the AI layer on top of those searches has not been set up to specifically try to find the ultimate source. But that's a less sexy headline.
As with most scenarios it's probably good to consider an AI to be a well read, articulate temp worker that will sometimes make things up because they must write at a fixed word per minute rate.
Upvote
-6
(0
/
-6)
The spreaders of fake science have been around since long before LLMs, in my experience their links will lead somewhere but it will be to a very low quality source or they will misrepresent the content of a trustworthy source. It's the ease of astroturfing enabled by LLMs that worries me. Drown the truth in shit.
Upvote
3
(3
/
0)
I sort of agree with you, in that the first paragraph of this article is a bit misleading, 'queries about news sources' could have been better worded, but the headline I'm seeing puts it better, 'AI search engines cite incorrect sources at an alarming 60% rate, study says'. I don't see the inaccuracy there that you do. In regard to your point above, they did. They only used extracts that returned the original source in the top three results of traditional search.
Last edited:
Upvote
0
(0
/
0)
If feels a bit like the mid-late 90s again, but this time SNL is funny, I'm no longer a teenage virgin, and a presidential blowjob seems almost quaint.
Upvote
3
(3
/
0)
And even if it were true… then why release something that performs so horribly?
I know it's a fictionalized version of the events, but more tech bros should consider that memorable exchange in Blackberry (2023):
Mike Lazaridis: I will build a prototype, but I'll do it perfectly or I don't do it.
Jim Balsillie: Mike, are you familiar with the saying "Perfect is the enemy of good"?
Mike Lazaridis: Well, "good enough" is the enemy of humanity.
Upvote
2
(2
/
0)
Does this mean I have to stop my 400 amp home electrical upgrade with the 22AWG hobby wire?
AI said 22AWG is fine for 500-some amps, it'll be so much cheaper than the BS huge wires the electricians said they wanted to use!
AI said 22AWG is fine for 500-some amps, it'll be so much cheaper than the BS huge wires the electricians said they wanted to use!
Upvote
-2
(0
/
-2)
On top of that, it doesn't take into account enshittification. Clearly, products can peak and get worse. Another good example - U.S. made cars in the late 1970s and early 1980s. Badge engineered shitboxes.
Upvote
3
(3
/
0)
Then don't label it as a search specific tool as though the accuracy problem were anywhere near solved. You're not putting this out there only for savvy users, but for the general public so take a modicum of responsibility.
Anyway, what's the difference between my asking for some clear facts about something that's easily discernable through traditional search and asking for any other accurate information? Apparently if it can handle the task it does so a low percentage of the time, and when it can't do it it spits out confabulatory garbage indiscernible from an accurate response unless I do the legwork myself. So why in the world would I use such a tool in the context of search? These so-called "search" flavors of models should simply spit out, oh I dunno, a list of 10 links for me to peruse on my own I guess.
Upvote
5
(5
/
0)
Should I start at the Stone Age or the Iron Age with the list? Clearly, you're not that in touch with how technology has evolved over the millenia.
Upvote
0
(1
/
-1)
Can we go back to just the links please? With AI at the helm of Internet searches and able to infer context with our inquires and knowing AI can be biased by its maintainers, this quote from Noam Chomsky comes to mind:
The smart way to keep people passive and obedient is to strictly limit the spectrum of acceptable opinion, but allow very lively debate within that spectrum - even encourage the more critical and dissident views. That gives people the sense that there's free thinking going on, while all the time the presuppositions of the system are being reinforced by the limits put on the range of the debate.
The smart way to keep people passive and obedient is to strictly limit the spectrum of acceptable opinion, but allow very lively debate within that spectrum - even encourage the more critical and dissident views. That gives people the sense that there's free thinking going on, while all the time the presuppositions of the system are being reinforced by the limits put on the range of the debate.
Upvote
3
(3
/
0)
The thing is, LLM's don't know what they "know" and don't know, in terms of knowledge, because they're not designed that way, and that's not how they were trained. The only thing they "know" are statistical relationships between tokens, where the tokens are words or phrases, and the output is just usually picking the next statistically likely token, giving the input and/or previous output, with a bit of randomness thrown in to occasionally pick a less likely token, so the output isn't fully deterministic. The only reason it provides anything factual at all is that it was trained on a bunch of text that was largely factual, so the relationships between tokens that describe a fact are particularly strong. But lies can have strong statistical relationships between tokens, and since it's always going to pick something, even some weak statistical relationships could get picked.
The best it could do is provide the statistical probabilities of the next token it picked, but that's no inherently an indication of factualness or "knowledge".
Upvote
13
(13
/
0)
Worse than that. Artificial General Intelligence systems gobble up Internet content irrespective of copyright.
Considering the rampant amount of disinformation prevalent on the Internet, I frankly wouldn’t trust anything AI responded with. In fact, the big tech companies in not policing disinformation, have sown the seeds of uselessness in this potential tool.
Garbage in, garbage out.
Upvote
5
(5
/
0)