log executeCode return values - #300
Conversation
github-actions
Bot
added
kernel
Preview:
|
bf9da6a to
60cdb65
Compare
| } | ||
| await agent(self, env, this.ctx); | ||
| let result = await agent(self, env, this.ctx); | ||
| if (result !== undefined) console.log(result); |
There was a problem hiding this comment.
Should we prefix this with some text explaining it is the return value of the function, to disambiguate from any logs appearing before that?
| * Return or print the schedule ID so it appears in the `executeCode` output. Empty output is not | ||
| * evidence that registration failed, and the disabled hook appearing in Connections confirms | ||
| * success. Do not retry solely because no ID was printed. |
There was a problem hiding this comment.
This text is weird. Why are we explaining the UI to the agent?
Did we actually have problems with agents getting confused and thinking that schedules had failed to be registered? If so, is the solution perhaps to simply emphasize that if the call doesn't throw, then the schedule has been registered?
Why does an agent care about the schedule ID at all? What can it do with this? Why are we telling it to log or return it?
There was a problem hiding this comment.
The schedule id can be used by the agent to confirm the schedule exists via list()
Non-SOTA models would often get confused because they would call executeCode but to them nothing happened - it would just get a void output.
The would interpret this as a fail and attempt to re-register (often many times)
This is probably less important now that it won't be a void return, but returning the schedule id may still be useful so it knows how to check if it is still enabled.
There was a problem hiding this comment.
When you experienced this, were those non-SOTA models specifically trying to return the schedule ID? If so, maybe simply the fact that we're logging now will mean that they no longer get confused, without the need for this text?
There was a problem hiding this comment.
Another thing we could do is: If the executeCode logs nothing, we could write "(function succeeded with no output)"
![devin-ai-integration[bot]](https://cdn.statically.io/img/avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
- help kimi and other non-sota models understand that no output doesn't mean something when wrong, just that the function ran without logging or returning anything
3148589 to
ebc13c4
Compare
|
|
- report errors that serialize to "" as uncaught exceptions instead of reporting success
|
No additional actionable findings beyond the existing return-value serialization issue. |
3 participants
Kimi likes to return values when using the
executeCodetool instead of logging them. Return values are currently discarded, which causes confusion. We can automatically log them instead.This is probably a better alternative to #299
Fixes #209