ChatGPT Search, an AI-powered search engine that went live this month, might be tricked into producing completely misleading summaries, based on a British newspaper The Guardian found it.
ChatGPT's search function is meant to hurry up browsing, for instance by summarizing product reviews on an internet site. But The Guardian found that it could trick ChatGPT into ignoring negative reviews and creating “completely positive” summaries by inserting hidden text Websites it created. ChatGPT Search is also made to spit out malicious code using this method.
Such hidden text attacks are a known risk to large language models, but this appears to be the primary time they’ve been demonstrated in a live AI-powered search product. According to The Guardian, Google, the search leader, has more experience coping with similar issues.
OpenAI didn’t comment on this specific incident when contacted by TechCrunch, but said it uses various methods to dam malicious web sites and is continually improving.
