llm/docs/index.md

LLM

A CLI utility and Python library for interacting with Large Language Models, both via remote APIs and models that can be installed and run on your own machine.

{ref}Run prompts from the command-line <usage-executing-prompts>, {ref}store the results in SQLite <logging>, {ref}generate embeddings <embeddings> and more.

Here's a YouTube video demo and accompanying detailed notes.

Background on this project:

• llm, ttok and strip-tags—CLI tools for working with ChatGPT and other LLMs
• The LLM CLI tool now supports self-hosted language models via plugins
• LLM now provides tools for working with embeddings
• Build an image search engine with llm-clip, chat with models with llm chat
• You can now run prompts against images, audio and video in your terminal using LLM
• Structured data extraction from unstructured content using LLM schemas
• Long context support in LLM 0.24 using fragments and template plugins

For more check out the llm tag on my blog.

Quick start

First, install LLM using pip or Homebrew or pipx:

pip install llm

Or with Homebrew (see {ref}warning note <homebrew-warning>):

brew install llm

Or with pipx:

pipx install llm

Or with uv

uv tool install llm

If you have an OpenAI API key key you can run this:

# Paste your OpenAI API key into this
llm keys set openai

# Run a prompt (with the default gpt-4o-mini model)
llm "Ten fun names for a pet pelican"

# Extract text from an image
llm "extract text" -a scanned-document.jpg

# Use a system prompt against a file
cat myfile.py | llm -s "Explain this code"

Or you can {ref}install a plugin <installing-plugins> and use models that can run on your local device:

# Install the plugin
llm install llm-gpt4all

# Download and run a prompt against the Orca Mini 7B model
llm -m orca-mini-3b-gguf2-q4_0 'What is the capital of France?'

To start {ref}an interactive chat <usage-chat> with a model, use llm chat:

llm chat -m gpt-4o

Chatting with gpt-4o
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
Type '!edit' to open your default editor and modify the prompt.
Type '!fragment <my_fragment> [<another_fragment> ...]' to insert one or more fragments
> Tell me a joke about a pelican
Why don't pelicans like to tip waiters?

Because they always have a big bill!
>

Contents

---
maxdepth: 3
---
setup
usage
openai-models
other-models
tools
schemas
templates
fragments
aliases
embeddings/index
plugins/index
python-api
logging
related-tools
help
contributing
changelog