I Gave an On-Device LLM a Search Tool. It Ignored It and Made Up Data Instead

The feature is one sentence. Type "what do I send someone who hasn't paid?" into your own snippet library and get back a sentence naming a real snippet, with the list below filtered to the snippets it looked at.

The obvious way to build that on iOS 27 is now roughly four lines, which is exactly the problem. Apple ships SpotlightSearchTool, you hand it to a LanguageModelSession, and the model can search your app's index and answer from what it finds. No embeddings, no vector store, no server. It compiles, it runs on device, and it produces beautiful answers immediately.

I shipped that version to my iPad on 13 August and asked it about twenty questions. Every answer was good. I wrote two conclusions into my planning document on the strength of them.

Both conclusions were false, and so were most of the answers.

The wrong output

Here is what...

Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE