🐈 Cats are confused by laser pointers and LLMs are confused by cats 🐈
We had to share this paper where researchers tested how LLMs responded to irrelevant facts embedded in prompts, for example inserting random cat trivia into math problems. They found that LLMs were significantly more likely to get the answers wrong when these facts were included, even though the facts had nothing to do with the question.
This article is a great demonstration of the issues still present in LLMs, that they can be easily distracted and misled (whether that’s the intention of the user or not).
And this is why we don’t allow free prompting in Keebler’s product. Instead, we control our model workflows so at each step, we can tune its performance and make sure no sneaky cat facts trip it up.