Extension IDE extension Limited info

TabbyML/tabby

Open-source, self-hosted AI coding assistant and on-premises alternative to GitHub Copilot, with an OpenAPI interface and support for consumer-grade GPUs.

  • 33.9k GitHub stars
  • Rust
View on GitHub (opens in a new tab) Setup guide
TabbyML/tabby preview image

What it is

Tabby is a self-hosted AI coding assistant, offered as an open-source, on-premises alternative to GitHub Copilot. It is self-contained, with no need for a DBMS or cloud service. It exposes an OpenAPI interface that is easy to integrate with existing infrastructure such as Cloud IDEs, and it supports consumer-grade GPUs.

Who it's for

  • Teams that want an on-premises, open-source alternative to GitHub Copilot
  • Developers who want to run a coding assistant on consumer-grade GPUs
  • Organizations that want to integrate a coding assistant with existing infrastructure (e.g. Cloud IDE) via an OpenAPI interface
  • Contributors who want to build Tabby from source (Rust)

Requirements

Requirements

  • Docker (for the quick-start server command)
  • NVIDIA GPU access (the example command uses --gpus all and --device cuda)
  • For building from source: Rust environment, protobuf (brew install protobuf on macOS; protobuf-compiler and libopenblas-dev on Ubuntu/Debian), and on Ubuntu make, sqlite3 and graphviz

Setup

  1. Run Tabby server with Docker

    The easiest way to start a Tabby server is with this Docker command. For additional options (e.g. inference type, parallelism), see the documentation.

    bash
    docker run -it \
  --gpus all -p 8080:8080 -v $HOME/.tabby:/data \
  tabbyml/tabby \
  serve --model StarCoder-1B --device cuda --chat-model Qwen2-1.5B-Instruct

  2. Get the source code (for contributing)

    Clone the repository with submodules. If already cloned, run git submodule update --recursive --init to fetch all submodules.

    bash
    git clone --recurse-submodules https://github.com/TabbyML/tabby
cd tabby

  3. Install build dependencies

    Set up the Rust environment first, then install the required dependencies. Useful tools can be installed on Ubuntu as well.

    bash
    # For MacOS
brew install protobuf

# For Ubuntu / Debian
apt install protobuf-compiler libopenblas-dev

# For Ubuntu
apt install make sqlite3 graphviz

  4. Build

    Build Tabby by running cargo build.

Examples

Start a Tabby server with a completion and chat model

bash
bash
docker run -it \
  --gpus all -p 8080:8080 -v $HOME/.tabby:/data \
  tabbyml/tabby \
  serve --model StarCoder-1B --device cuda --chat-model Qwen2-1.5B-Instruct

What it does: Runs the Tabby server on port 8080 with all GPUs, persisting data in $HOME/.tabby, using StarCoder-1B for completion on CUDA and Qwen2-1.5B-Instruct for chat.

Clone the repository with submodules

bash
bash
git clone --recurse-submodules https://github.com/TabbyML/tabby
cd tabby

What it does: Fetches the Tabby source including submodules, as a first step to contributing.

Pros & cons

Pros

  • Pro:Self-contained: no DBMS or cloud service required
  • Pro:OpenAPI interface eases integration with existing infrastructure such as Cloud IDEs
  • Pro:Supports consumer-grade GPUs
  • Pro:Open-source and on-premises alternative to GitHub Copilot, with IDE/editor extensions documented

Cons

  • Con:The README's quick-start command assumes a GPU (--gpus all, --device cuda); other setups require consulting the documentation
  • Con:The README gives only a minimal quick start; further options such as inference type and parallelism are deferred to external docs

Images

TabbyML/tabby preview image
TabbyML/tabby preview image · TabbyML/tabby on GitHub
TabbyML/tabby preview image