How to handle invalid tool calls?

[mlynch]

I'm using tool/function calling to modify a data structure (using gpt-4o). My tool has a number of parameters, such as label and description of a new item to add to the structure. Often, the tool will be called with missing fields as required by the zod schema, causing a runtime error. I am unsure what to do in this situation. I would think the model just wouldn't call my tool if it wasn't able to provide the required parameters, yet it does. I'm really at a loss about how to handle this scenario or even how error handling works in this case.

Any advice?

[lgrammel]

@mlynch I've written up a short guide on how to improve tool calling with the AI SDK: https://sdk.vercel.ai/docs/ai-sdk-core/tools-and-tool-calling#prompt-engineering-with-tools

Automatic retries when the parsing fails are planned (no timeline yet)

[mlynch]

Thanks @lgrammel. I wrote a custom implementation that calls chatgpt directly using gpt-4o and the same prompt, and tool calls are working successfully. There seems to be a bug with the AI SDK with tool calling params not being correctly built, which causes validation to fail and it seems impossible to catch this condition as it's happening async somewhere deep in the call stack and not awaited.

[lgrammel]

interesting - we use zod-to-json-schema under the hood. would you mind filing a bug (ideally with a reproduction if you can share and the json schema that works for you)? I'd like to look into this in more detail.

[lgrammel]

also, in a few minutes I'm going to ship the option of having custom fetch functions, so you could wrap fetch at a low level and inspect what we send in

[lgrammel]

@mlynch I've found one issue that may have contributed to tool instability (bc the temperature was not automatically set to 0): #2012 - would you be open to trying out your examples again one this lands (prob in 3.2.0)?

[amirm363]

It happened to me when I set the whole z.object to optional, if you do so, try removing it (or if you have a nested optional on the parameters) and check if it works.

[defless]

@lgrammel I'm asking this here as it's the most related topic I've found.

But do you know how I can handle these zod validation errors in tools calls with streamUi?
I've found something with createStreamableUI but nothing with streamUI in the doc and examples.

[oeken]

sub to the thread, i am having the same issue, not being able to gracefully handle argument validation errors.

[testinguser22]

+1

[carolineloppi]

@lgrammel I'm also struggling with handling Zod validation errors for tool parameters in streamText. As previously mentioned, when tool schema validation fails, the error crashes the application instead of being handled gracefully. While experimental_repairToolCall exists, it's designed for retrying/fixing tool calls rather than returning user-friendly error messages directly. Using toDataStreamResponse isn't suitable since I need pipeDataStreamToResponse for stream manipulation, and wrapping streamText in a try-catch seems too broad since many unrelated errors would be handled the same way. I'd love to see a built-in way to intercept tool validation errors specifically and convert them into user-friendly messages (similar to how toDataStreamResponse handles errors) while maintaining the streaming flow. Sorry for the giant message, just struggling with this a few days now.

[saifvazir]

+1

[oeken]

a quick solution here is using json schema instead of zod schema, since i made that change i dont get hard fails