Simon Willison
e4ecb86421
Log tool_instances to database ( #1098 )
...
* Log tool_instances to database, closes #1089
* Tested for both sync and async models
2025-05-26 21:01:55 -07:00
Simon Willison
70092f2e78
llm logs --tools flag, closes #1072
2025-05-23 21:58:24 -07:00
Simon Willison
8a1fc5a90e
llm logs --tool/-T option, closes #1013
2025-05-23 21:46:10 -07:00
Simon Willison
d2886d4692
Record which plugin a tool came from, including in DB - refs #1020
2025-05-23 15:44:12 -07:00
Simon Willison
e18eb3a595
Include tools in documented schema
2025-05-22 07:43:12 -07:00
Sergio Garcia
98249710f9
Fix typo ( #1002 )
2025-05-12 19:55:32 -07:00
Simon Willison
0fbbe6a054
llm logs backup command, closes #879
2025-04-07 21:45:52 -07:00
Simon Willison
3de33be74f
Support multiple fragments with same content, closes #888
2025-04-06 16:26:57 -07:00
Simon Willison
ac49075129
llm logs -e/--expand option, closes #881
2025-04-06 00:25:28 -07:00
Simon Willison
f740a5cbbd
Fragments ( #859 )
...
* WIP fragments: schema plus reading but not yet writing, refs #617
* Unique index on fragments.alias, refs #617
* Fragments are now persisted, added basic CLI commands
* Fragment aliases work now, refs #617
* Improved help for -f/--fragment
* Support fragment hash as well
* Documentation for fragments
* Better non-JSON display of llm fragments list
* llm fragments -q search option
* _truncate_string is now truncate_string
* Use condense_json to avoid duplicate data in JSON in DB, refs #617
* Follow up to 3 redirects for fragments
* Python API docs for fragments= and system_fragments=
* Fragment aliases cannot contain a : - this is to ensure we can add custom fragment loaders later on, refs https://github.com/simonw/llm/pull/859#issuecomment-2761534692
* Use template fragments when running prompts
* llm fragments show command plus llm fragments group tests
* Tests for fragments family of commands
* Test for --save with fragments
* Add fragments tables to docs/logging.md
* Slightly better llm fragments --help
* Handle fragments in past conversations correctly
* Hint at llm prompt --help in llm --help, closes #868
* llm logs -f filter plus show fragments in llm logs --json
* Include prompt and system fragments in llm logs -s
* llm logs markdown fragment output and tests, refs #617
2025-04-05 17:22:37 -07:00
Simon Willison
3a60290c82
llm logs --id-gt and --id-gte options, closes #801
2025-02-28 00:15:59 -08:00
Simon Willison
74baf33a56
Moved some docs into schemas.md, refs #788
2025-02-27 11:20:50 -08:00
Simon Willison
f35ac31c21
llm logs --schema, --data, --data-array and --data-key options ( #785 )
...
* llm logs --schema option, refs #782
* --data and --data-array and --data-key options, refs #782
* Tests for llm logs --schema options, refs #785
* Also implemented --schema ID lookup, refs #780
* Using --data-key implies --data
* Docs for llm logs --schema and --data etc
2025-02-26 21:51:08 -08:00
Simon Willison
62c90dd472
llm prompt --schema X option and model.prompt(..., schema=) parameter ( #777 )
...
Refs #776
* Implemented new llm prompt --schema and model.prompt(schema=)
* Log schema to responses.schema_id and schemas table
* Include schema in llm logs Markdown output
* Test for schema=pydantic_model
* Initial --schema CLI documentation
* Python docs for schema=
* Advanced plugin docs on schemas
2025-02-26 16:58:28 -08:00
Simon Willison
c053e214ec
include token usage information, not get - refs #756
2025-02-16 15:18:04 -08:00
Simon Willison
53d6ecdd59
Documentation for logs --short, refs #756
2025-02-16 15:16:51 -08:00
Simon Willison
41d64a8f12
llm logs --prompts option ( #737 )
...
Closes #736
2025-02-02 12:03:01 -08:00
Simon Willison
656d8fa3c4
--xl/--extract-last flag for prompt and log list commands ( #718 )
...
Closes #717
2025-01-24 10:52:46 -08:00
Csaba Henk
88a8cfd9e4
llm logs -x/--extract option ( #693 )
...
* llm logs -x/--extract option
* Update docs/help.md for llm logs -x
* Added test for llm logs -x/--extract, refs #693
* llm logs -xr behaves same as llm logs -x
* -x/--extract in llm logging docs
---------
Co-authored-by: Simon Willison <swillison@gmail.com>
2025-01-10 15:53:04 -08:00
Simon Willison
cfb10f4afd
Log input tokens, output tokens and token details ( #642 )
...
* Store input_tokens, output_tokens, token_details on Response, closes #610
* llm prompt -u/--usage option
* llm logs -u/--usage option
* Docs on tracking token usage in plugins
* OpenAI default plugin logs usage
2024-11-19 20:21:59 -08:00
Simon Willison
febbc04fb6
Run cog -r in PRs, use that to update logging.md with new tables ( #616 )
...
* Create cog.yml
* Document attachments and prompt_attachments table schemas
Closes #615
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-11-06 06:56:19 -08:00
Simon Willison
fb63c92cd2
llm logs -r/--response option, closes #431
2024-03-04 13:29:07 -08:00
Simon Willison
0699bb5e27
Docs for llm chat, refs #231
2023-09-04 23:36:25 -07:00
Simon Willison
a4f55e9987
--cid/--conversation option, docs for search - refs #160
2023-08-17 14:32:43 -07:00
Simon Willison
9a177abfd5
Release 0.6
...
Refs #106 , #107 , #108 , #109
2023-07-18 14:36:37 -07:00
Simon Willison
178af27d95
llm logs list -q/--query option, closes #109
2023-07-15 16:20:28 -07:00
Simon Willison
75e40e7077
Enable FTS indexes on responses prompt and response columns, refs #109
2023-07-15 16:03:35 -07:00
Simon Willison
ed8cd776a4
llm logs list -m model_id/alias option, closes #108
2023-07-15 15:55:13 -07:00
Simon Willison
fa67b3fdaf
--log option, closes #68
2023-07-11 20:18:16 -07:00
Simon Willison
05b4bcf57c
Docs for llm logs on/off/status, closes #98
2023-07-11 20:09:07 -07:00
Simon Willison
56016aef59
Include conversations table in docs schema, refs #91
2023-07-11 07:21:22 -07:00
Simon Willison
2d3ebe7fe1
llm logs now uses new DB schema, refs #91
2023-07-11 07:21:22 -07:00
Simon Willison
307af2474c
Fix column order in logs
2023-07-03 07:29:58 -07:00
Simon Willison
345ad0d2dc
Implemented new logs database schema
2023-07-03 07:27:47 -07:00
Simon Willison
9a180e65a8
llm models default command, plus refactored env variables
...
Closes #76
Closes #31
2023-07-01 14:01:29 -07:00
Simon Willison
b27c2cdac9
Rename log.db to logs.db, closes #41
2023-06-17 09:29:46 +01:00
Simon Willison
cd1e0f85e4
Fix schema display, refs #34
2023-06-16 08:57:53 +01:00
Simon Willison
43f2cbd2e3
Schema in logging.md docs, closes #35
2023-06-16 08:39:01 +01:00
Simon Willison
af8d596e63
Moved logs.db to user_data_dir, closes #27
2023-06-15 20:19:02 +01:00
Simon Willison
b9865a5576
Multi-page docs using Markdown and Sphinx, refs #21
...
Also documents new keys.json mechanism, closes #13
2023-06-15 17:51:12 +01:00
