llm/docs/index.md

LLM

A CLI utility and Python library for interacting with Large Language Models, both via remote APIs and models that can be installed and run on your own machine.

{ref}Run prompts from the command-line <usage-executing-prompts>, {ref}store the results in SQLite <logging>, {ref}generate embeddings <embeddings> and more.

Here's a YouTube video demo and accompanying detailed notes.

Background on this project:

• llm, ttok and strip-tags—CLI tools for working with ChatGPT and other LLMs
• The LLM CLI tool now supports self-hosted language models via plugins
• LLM now provides tools for working with embeddings
• Build an image search engine with llm-clip, chat with models with llm chat
• You can now run prompts against images, audio and video in your terminal using LLM
• Structured data extraction from unstructured content using LLM schemas

For more check out the llm tag on my blog.

Quick start

First, install LLM using pip or Homebrew or pipx:

pip install llm

Or with Homebrew (see {ref}warning note <homebrew-warning>):

brew install llm

Or with pipx:

pipx install llm

Or with uv

uv tool install llm

If you have an OpenAI API key key you can run this:

# Paste your OpenAI API key into this
llm keys set openai

# Run a prompt (with the default gpt-4o-mini model)
llm "Ten fun names for a pet pelican"

# Extract text from an image
llm "extract text" -a scanned-document.jpg

# Use a system prompt against a file
cat myfile.py | llm -s "Explain this code"

Or you can {ref}install a plugin <installing-plugins> and use models that can run on your local device:

# Install the plugin
llm install llm-gpt4all

# Download and run a prompt against the Orca Mini 7B model
llm -m orca-mini-3b-gguf2-q4_0 'What is the capital of France?'

To start {ref}an interactive chat <usage-chat> with a model, use llm chat:

llm chat -m gpt-4o

Chatting with gpt-4o
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
> Tell me a joke about a pelican
Why don't pelicans like to tip waiters?

Because they always have a big bill!
>

Contents

---
maxdepth: 3
---
setup
usage
openai-models
other-models
embeddings/index
plugins/index
aliases
python-api
schemas
templates
logging
related-tools
help
contributing
changelog