Commit graph

292 commits

Author SHA1 Message Date
Simon Willison
c41c122239 Use tools in templates with llm chat, closes #1239 2025-08-11 22:11:17 -07:00
Simon Willison
2f206d0e26 Fix for duplicated prompts in llm chat with templates, closes #1240
Also includes a bug fix for system fragments, see https://github.com/simonw/llm/issues/1240#issuecomment-3177684518
2025-08-11 21:52:54 -07:00
Simon Willison
c6e158071a Ran black 2025-08-11 21:47:11 -07:00
Simon Willison
e6ac18fbcb Fix for confusing error, closes #1238 2025-08-11 16:52:16 -07:00
Simon Willison
9f1417f6e8 Fix for enum options and --save, refs #1237 2025-08-11 16:16:27 -07:00
Simon Willison
e4c1a46d90 Fix test failure caused by version bump, refs #1218 2025-08-11 14:27:16 -07:00
James Sanford
2a54939951
Fix streaming tool calls with tests for many variants. (#1218)
* Recorded instance of streaming tool response variant "a".

This is the typical response, where "arguments":"" arrives
first in the stream, followed by "arguments":"{}"

The response data is a real capture from the OpenRouter API,
however some request and header data may be from other test fixtures.

* Recorded instance of streaming tool response variant "b".

This is a streaming response where the first arguments
you get is a fully formed "arguments":"{}"

The response data is a real capture from the OpenRouter API,
however some request and header data may be from other test fixtures.

* Test cases for streaming tool responses.

Note that the replays are marked as "read-only", as they are variants
seen in the wild where the streaming tool call argument fragments
arrive in a specific order.

* Fix streaming tool response variant "b", where "arguments":"{}" is what arrives first.

The previous code erroneously caused the first "arguments" to be duplicated,
by using "+=" even when being initially set.

This went unnoticed as many models stream "arguments":"" first.

When a more fully formed "arguments" fragment arrived first, it was causing
"Error: Extra data: line 1 column 3 (char 2)"

* Recorded instance of streaming tool response variant "c".

This was failing with "Error: unsupported operand type(s) for +=: 'NoneType' and 'str'"

The response data is a real capture from the OpenRouter API,
however some request and header data may be from other test fixtures.

* Test case for streaming tool response variant "c".

* Fix streaming tool response variant "c".

However, I'm not sure why arguments was initially not present or seen as None.
2025-08-11 14:17:52 -07:00
Simon Willison
5204a11f33 Allow -o option when calling tools, closes #1233 2025-08-11 13:44:26 -07:00
Simon Willison
08094082f2
Toolbox.add_tool(), prepare() and prepare_async() methods
Closes #1111
2025-08-11 13:19:31 -07:00
Simon Willison
0863ed460e llm logs -l/--latest -q option, closes #1177 2025-06-17 23:23:45 -07:00
Simon Willison
544ce17c1d Tests to confirm responses FTS triggers
Refs https://github.com/simonw/llm/issues/1177#issuecomment-2982832935
2025-06-17 23:18:22 -07:00
Simon Willison
3a96d52895 Better handling of before_call cancellation, closes #1148 2025-06-01 18:36:55 -07:00
Simon Willison
d96ae4ed8d Fix --async logging to database, closes #1150 2025-06-01 17:38:26 -07:00
Simon Willison
30e0c4abe8
ToolResult.exception for tool errors, now logged to DB
Closes #1104
2025-06-01 17:01:40 -07:00
Simon Willison
ed64fc3362
chain_limit/before_call/after_call for conversations
* chain_limit/before_call/after_call for conversations, closes #1088
* Docs for before_call/after_call including for model.conversation
2025-06-01 12:00:29 -07:00
Simon Willison
b5d1c5ee90
Tools can now return attachments
Closes #1014

- llm.ToolOutput(output='...', attachments=[...]) for tools to return attachments
- New table: `tool_results_attachments`
- Table is populated when tools return attachments
- llm --tools-debug shows attachments returned by tools
- llm logs shows attachments returned by tools
2025-06-01 10:08:36 -07:00
Simon Willison
330a568683 Raise error if a non-Toolbox subclass is passed to register(), closes #1114 2025-05-30 21:37:41 -07:00
Simon Willison
a3a2996fed
Tools in templates (#1138)
Closes #1009
2025-05-30 17:44:52 -07:00
Simon Willison
b858b0083e set_resolved_model() for async models, closes #1117 2025-05-28 07:39:57 -07:00
Simon Willison
301db6d76c responses.resolved_model column and response.set_resolved_model(model_id) method, closes #1117 2025-05-28 07:17:03 -07:00
Simon Willison
6bab712cdd Fix for module 'builtins' has no attribute 'instance_id, closes #1107 2025-05-27 13:13:28 -07:00
Simon Willison
dbb02d65b9 Fix for llm_time plugin test 2025-05-27 10:26:08 -07:00
Simon Willison
9e25055765 Turn incorrect tool names into errors, closes #1104 2025-05-27 09:29:05 -07:00
Simon Willison
2bc6d7679c New default tool, llm_time, closes #1103 2025-05-26 23:00:18 -07:00
Simon Willison
d1c5c688e2 Error on llm -c with a Toolbox, closes #1101
Refs #1092
2025-05-26 21:51:16 -07:00
Simon Willison
e4ecb86421
Log tool_instances to database (#1098)
* Log tool_instances to database, closes #1089
* Tested for both sync and async models
2025-05-26 21:01:55 -07:00
Simon Willison
c9e8593095 Assert specific order again thanks to monotonic ULIDs, refs #1099 2025-05-26 20:03:22 -07:00
Simon Willison
1dc7a1d1f9 Monotonic ULIDs, refs #1099 2025-05-26 19:49:42 -07:00
Simon Willison
57459c0a6e Fix the tests I broke in #1096 2025-05-26 19:02:14 -07:00
Simon Willison
9bbb37fae0 New default llm_version tool, closes #1096
Refs https://github.com/simonw/llm/issues/1095#issuecomment-2910574597
2025-05-26 13:30:47 -07:00
Simon Willison
e23e13e6c7 Test for async toolbox, docs for toolboxes in general
Closes #1090, refs #997
2025-05-26 10:23:03 -07:00
Simon Willison
00f44a848a Test that async def methods work on Toolbox 2025-05-26 10:08:31 -07:00
Simon Willison
7eb8acb767 llm.get_key() is now a documented utility, closes #1094
Refs #1093, https://github.com/simonw/llm-tools-datasette/issues/2
2025-05-26 09:39:40 -07:00
Simon Willison
bb336d33a0
Toolbox class for class-based tool collections (#1086)
* Toolbox class for class-based tool collections

Refs #1059, #1058, #1057
2025-05-25 22:42:52 -07:00
Simon Willison
abc4f473f4 --td now pretty-prints JSON if possible, closes #1083 2025-05-25 12:59:48 -07:00
Simon Willison
7837335eea Remove colon in Token usage header for consistency
More consistent with other headings in logs now.
2025-05-24 16:55:05 -07:00
André Arko
fc44fd6a0d
llm chat now respects default model options (#985) 2025-05-24 14:24:53 -07:00
Simon Willison
713ee9b032 Ignore order in test, refs #1073, #1079 2025-05-24 13:18:09 -07:00
Simon Willison
3d25708acd Instantiate from spec utility function, refs #1059
Code by o3, docstring by Claude Sonnet 4
2025-05-24 08:21:33 -07:00
Simon Willison
deafb7e76a Removed unused fixture 2025-05-24 00:03:27 -07:00
Simon Willison
b6f4838b17 Test for fragments in chat, refs #1044, #1048 2025-05-23 23:48:48 -07:00
Dan Turkel
88d3e11c65
Support fragments in chat through -f, --sf, and !fragments (#1048)
* add vscode and uv artifacts to gitignore
* !fragment feature
* Update fragment command to accept multiple fragments
* Add support for multiple fragments in !fragment command, factor out fragment processing
* Add support for fragment/system fragment arguments in chat command
* update docs

---------

Co-authored-by: Simon Willison <swillison@gmail.com>
2025-05-23 23:31:16 -07:00
Simon Willison
4281fd5101 Fixed test for #1073 2025-05-23 23:26:33 -07:00
Dan Turkel
1d5d73481a
Filter IDs with --prefix in llm similar (#1052) 2025-05-23 22:54:55 -07:00
Simon Willison
70092f2e78 llm logs --tools flag, closes #1072 2025-05-23 21:58:24 -07:00
Simon Willison
8a1fc5a90e llm logs --tool/-T option, closes #1013 2025-05-23 21:46:10 -07:00
Simon Willison
36477cf9e5 llm chat -c and llm -c carry forward tools, closes #1020 2025-05-23 21:10:51 -07:00
Simon Willison
3cbc37a470 Log tool plugin to database
Refs https://github.com/simonw/llm/issues/1020#issuecomment-2905968166
2025-05-23 15:59:43 -07:00
Simon Willison
d2886d4692 Record which plugin a tool came from, including in DB - refs #1020 2025-05-23 15:44:12 -07:00
Simon Willison
f6fefb3816 llm schemas list --json/--nl options, closes #1070 2025-05-23 15:17:32 -07:00