llm/docs/index.md

LLM

A CLI utility and Python library for interacting with Large Language Models, both via remote APIs and models that can be installed and run on your own machine.

{ref}Run prompts from the command-line <usage-executing-prompts>, {ref}store the results in SQLite <logging>, {ref}generate embeddings <embeddings> and more.

Background on this project:

• llm, ttok and strip-tags—CLI tools for working with ChatGPT and other LLMs
• The LLM CLI tool now supports self-hosted language models via plugins
• Accessing Llama 2 from the command-line with the llm-replicate plugin
• Run Llama 2 on your own Mac using LLM and Homebrew
• Catching up on the weird world of LLMs
• LLM now provides tools for working with embeddings
• Build an image search engine with llm-clip, chat with models with llm chat
• Many options for running Mistral models in your terminal using LLM

For more check out the llm tag on my blog.

Quick start

First, install LLM using pip or Homebrew or pipx:

pip install llm

Or with Homebrew (see {ref}warning note <homebrew-warning>):

brew install llm

Or with pipx:

pipx install llm

If you have an OpenAI API key key you can run this:

# Paste your OpenAI API key into this
llm keys set openai

# Run a prompt
llm "Ten fun names for a pet pelican"

# Run a system prompt against a file
cat myfile.py | llm -s "Explain this code"

Or you can {ref}install a plugin <installing-plugins> and use models that can run on your local device:

# Install the plugin
llm install llm-gpt4all

# Download and run a prompt against the Orca Mini 7B model
llm -m orca-mini-3b-gguf2-q4_0 'What is the capital of France?'

To start {ref}an interactive chat <usage-chat> with a model, use llm chat:

llm chat -m chatgpt

Chatting with gpt-3.5-turbo
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
> Tell me a joke about a pelican
Why don't pelicans like to tip waiters?

Because they always have a big bill!
>

Contents

---
maxdepth: 3
---
setup
usage
openai-models
other-models
embeddings/index
plugins/index
aliases
python-api
templates
logging
related-tools
help
contributing
changelog