llm/docs/plugins/directory.md

(plugin-directory)=

Plugin directory

The following plugins are available for LLM. Here's {ref}how to install them <installing-plugins>.

Local models

These plugins all help you run LLMs directly on your own computer:

• llm-mlc can run local models released by the MLC project, including models that can take advantage of the GPU on Apple Silicon M1/M2 devices.
• llm-llama-cpp uses llama.cpp to run models published in the GGML format.
• llm-gpt4all adds support for various models released by the GPT4All project that are optimized to run locally on your own machine. These models include versions of Vicuna, Orca, Falcon and MPT - here's a full list of models.
• llm-mpt30b adds support for the MPT-30B local model.

Remote APIs

These plugins can be used to interact with remotely hosted models via their API:

• llm-palm adds support for Google's PaLM 2 model.
• llm-replicate adds support for remote models hosted on Replicate, including Llama 2 from Meta AI.
• llm-claude by Tom Viner adds support for Claude and Claude Instant by Anthropic.
• llm-openrouter provides access to models hosted on OpenRouter.
• llm-anyscale-endpoints supports models hosted on the Anyscale Endpoints platform, including Llama 2 70B.

If an API model host provides an OpenAI-compatible API you can also configure LLM to talk to it without needing an extra plugin.

Just for fun

• llm-markov adds a simple model that generates output using a Markov chain. This example is used in the tutorial Writing a plugin to support a new model.