llm/docs/index.md

LLM

A CLI utility and Python library for interacting with Large Language Models, both via remote APIs and models that can be installed and run on your own machine.

{ref}Run prompts from the command-line <usage-executing-prompts>, {ref}store the results in SQLite <logging>, {ref}generate embeddings <embeddings> and more.

Background on this project:

• llm, ttok and strip-tags—CLI tools for working with ChatGPT and other LLMs
• The LLM CLI tool now supports self-hosted language models via plugins
• Accessing Llama 2 from the command-line with the llm-replicate plugin
• Run Llama 2 on your own Mac using LLM and Homebrew
• Catching up on the weird world of LLMs
• LLM now provides tools for working with embeddings
• Build an image search engine with llm-clip, chat with models with llm chat
• Many options for running Mistral models in your terminal using LLM

For more check out the llm tag on my blog.

Quick start

First, install LLM using pip:

pip install llm

Or with pipx (recommended, as then it won't clash with any other installed packages):

pipx install llm

If you have an OpenAI API key key you can run this:

# Paste your OpenAI API key into this
llm keys set openai

# Run a prompt
llm "Ten fun names for a pet pelican"

# Run a system prompt against a file
cat myfile.py | llm -s "Explain this code"

Or you can {ref}install a plugin <installing-plugins> and use models that can run on your local device:

# Install the plugin
llm install llm-gpt4all

# Download and run a prompt against the Orca Mini 7B model
llm -m orca-mini-3b-gguf2-q4_0 'What is the capital of France?'

To start {ref}an interactive chat <usage-chat> with a model, use llm chat:

llm chat -m chatgpt

Chatting with gpt-3.5-turbo
Type 'exit' or 'quit' to exit
Type '!multi' to enter multiple lines, then '!end' to finish
> Tell me a joke about a pelican
Why don't pelicans like to tip waiters?

Because they always have a big bill!
>

Contents

---
maxdepth: 3
---
setup
usage
other-models
embeddings/index
plugins/index
aliases
python-api
templates
logging
related-tools
help
contributing
changelog