diff --git a/website/docs/client.md b/website/docs/getting-started.md similarity index 50% rename from website/docs/client.md rename to website/docs/getting-started.md index a2646ab..dfa1af3 100644 --- a/website/docs/client.md +++ b/website/docs/getting-started.md @@ -1,11 +1,6 @@ -# Client +# Getting Started -We offer multiple methods to connect to Tabby Server, including editor extensions and OpenAPI. - -## Editor Extensions +Install Tabby in your code editors, if your favorite editor is not supported yet, consider filing an [Feature Request](https://github.com/TabbyML/tabby/issues/new?assignees=&labels=enhancement&projects=&template=feature_request.md&title=). * [VSCode Extension](https://github.com/TabbyML/tabby/tree/main/clients/vscode) – Install from the [marketplace](https://marketplace.visualstudio.com/items?itemName=TabbyML.vscode-tabby), or [open-vsx.org](https://open-vsx.org/extension/TabbyML/vscode-tabby) * [VIM Extension](https://github.com/TabbyML/tabby/tree/main/clients/vim) - -## API -Tabby has opened a FastAPI server at [localhost:8080](https://localhost:8080), which includes an OpenAPI documentation of the HTTP API. The same API documentation is also hosted at [here](/api). diff --git a/website/docs/self-hosting/01-docker.md b/website/docs/self-hosting/01-docker.md new file mode 100644 index 0000000..e84e958 --- /dev/null +++ b/website/docs/self-hosting/01-docker.md @@ -0,0 +1,59 @@ +# Docker + +There is a supplied docker image to make deploying a server as a container easier. + + +## CPU + +**Command line** +```bash +docker run \ + -p 8080:8080 -v $HOME/.tabby:/data \ + tabbyml/tabby serve --model TabbyML/SantaCoder-1B +``` + + +**Docker Compose** +```yaml +version: '3.5' + +services: + tabby: + restart: always + image: tabbyml/tabby + command: serve --model TabbyML/SantaCoder-1B + volumes: + - "$HOME/.tabby:/data" + ports: + - 8080:8080 +``` + +## CUDA (requires NVIDIA Container Toolkit) + +**Command line** +```bash +docker run \ + --gpus all -p 8080:8080 -v $HOME/.tabby:/data \ + tabbyml/tabby \ + serve --model TabbyML/SantaCoder-1B --device cuda +``` + +**Docker Compose** +```yaml +version: '3.5' +services: + tabby: + restart: always + image: tabbyml/tabby + command: serve --model TabbyML/SantaCoder-1B --device cuda + volumes: + - "$HOME/.tabby:/data" + ports: + - 8080:8080 + resources: + reservations: + devices: + - driver: nvidia + count: 1 + capabilities: [gpu] +``` diff --git a/website/docs/self-hosting/02-apple.md b/website/docs/self-hosting/02-apple.md new file mode 100644 index 0000000..d29672b --- /dev/null +++ b/website/docs/self-hosting/02-apple.md @@ -0,0 +1,9 @@ +# Mac M1/M2 (Preview) + +Thanks to Apple's Accelerate and CoreML frameworks, we can now run Tabby on edge devices with reasonable inference speed. Follow the steps below to set it up: + +1. Download the `tabby` binary from the Release page [here](https://github.com/TabbyML/tabby/releases/tag/latest). + +2. Run `tabby --help` to verify successful installation. + +3. Start the server with `tabby serve model --models TabbyML/T5P-220M`. diff --git a/website/docs/self-hosting/self-hosting.md b/website/docs/self-hosting/self-hosting.md new file mode 100644 index 0000000..02a1b2f --- /dev/null +++ b/website/docs/self-hosting/self-hosting.md @@ -0,0 +1,17 @@ +# Self Hosting + +## Configuration + +Server config can be found at `~/.tabby/config.toml` + +it looks something like this + +```toml +[[repositories]] +git_url = "https://github.com/TabbyML/tabby.git" +``` + +| Parameter | Description | +| ------------------------- | ----------------------------------------------------------------------------------- | +| `repository` | List of source code repository to integrate with the instance. | +| `repository.git_url` | URL to git repository, where tabby extract snippets for prompting and fine tuning. | diff --git a/website/docs/server.md b/website/docs/server.md deleted file mode 100644 index 3d6dc7a..0000000 --- a/website/docs/server.md +++ /dev/null @@ -1,48 +0,0 @@ -# Server - -## Docker - -We recommend adding the following aliases to your `.bashrc` or `.zshrc` file: - -```shell -# Save aliases to bashrc / zshrc -alias tabby="docker run -u $(id -u) -p 8080:8080 -v $HOME/.tabby:/data tabbyml/tabby" - -# Alias for GPU (requires NVIDIA Container Toolkit) -alias tabby-gpu="docker run --gpus all -u $(id -u) -p 8080:8080 -v $HOME/.tabby:/data tabbyml/tabby" -``` - -After adding these aliases, you can use the `tabby` command as usual. Here are some examples of its usage: - -```shell -# Create work directory for tabby -mkdir -p $HOME/.tabby - -# Usage -tabby --help - -# Serve the model -tabby serve --model TabbyML/J-350M - -# Serve with cuda -tabby-gpu serve --model TabbyML/J-350M --device cuda -``` - -## Configuration - -Tabby maintains its runtime data in `~/.tabby`, with `~/.tabby/config.toml` serving as its configuration file. - -You can modify the path to the data directory by setting the `TABBY_ROOT` variable. For example: - -``` -export TABBY_ROOT=/data/tabby -``` - -## Example `~/.tabby/config.toml` - -```toml -[[repositories]] -git_url = "https://github.com/TabbyML/tabby.git" -``` - -`repositories` is a list of git repositories integrated into Tabby for prompting/fine-tuning.