llm/docs/tools.md

(tools)=

Tools

Many Large Language Models have been trained to execute tools as part of responding to a prompt. LLM supports tool usage with both the command-line interface and the Python API.

How tools work

A tool is effectively a function that the model can request to be executed. Here's how that works:

1. The initial prompt to the model includes a list of available tools, containing their names, descriptions and parameters.
2. The model can choose to call one (or sometimes more than one) of those tools, returning a request for the tool to execute.
3. The code that calls the model - in this case LLM itself - then executes the specified tool with the provided arguments.
4. LLM prompts the model a second time, this time including the output of the tool execution.
5. The model can then use that output to generate its next response.

Trying out tools

LLM comes with a default tool installed, called llm_version. You can try that out like this:

llm -T llm_version "What version of LLM is this?" --td

The output should look like this:

Tool call: llm_version({})
  0.26a0

The installed version of the LLM is 0.26a0.

Further tools can be installed using plugins, or you can use the llm --functions option to pass tools implemented as PYthon functions directly, as {ref}described here <usage-tools>.

LLM's implementation of tools

In LLM every tool is a defined as a Python function. The function can take any number of arguments and can return a string or an object that can be converted to a string.

Tool functions should include a docstring that describes what the function does. This docstring will become the description that is passed to the model.

The Python API can accept functions directly. The command-line interface has two ways for tools to be defined: via plugins that implement the {ref}register_tools() plugin hook <plugin-hooks-register-tools>, or directly on the command-line using the --functions argument to specify a block of Python code defining one or more functions - or a path to a Python file containing the same.

You can use tools {ref}with the LLM command-line tool <usage-tools> or {ref}with the Python API <python-api-tools>.

Tips for implementing tools

Consult the {ref}register_tools() plugin hook <plugin-hooks-register-tools> documentation for examples of how to implement tools in plugins.

If your plugin needs access to API secrets I recommend storing those using llm keys set api-name and then reading them using the {ref}plugin-utilities-get-key utility function. This avoids secrets being logged to the database as part of tool calls.