root
/
tabby
mirror of https://gitee.com/zhang_1334717033/tabby.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
tabby/website/docs/models/index.md

38 lines
3.3 KiB
Markdown
Raw Normal View History

docs: add models directory (#437)
2023-09-12 15:41:00 +00:00
---
sidebar_position: 4
---
# 🧑‍🔬 Models Directory
docs: add modelscope registry information to model directory
2023-10-04 05:23:47 +00:00
## Completion models (`--model`)
docs: add device recommendations based on model size
2023-09-14 06:07:21 +00:00
We recommend using
docs: add wizardcoder models (#467) * Update README.md test * Update on 10k stars. * Update README.md on 10k stars. * doc: add wizardcoder models
2023-09-21 12:54:20 +00:00
* **small models (less than 400M)** for **CPU devices**.
* For **1B to 7B models**, it's advisable to have at least **NVIDIA T4, 10 Series, or 20 Series GPUs**.
* For **7B to 13B models**, we recommend using **NVIDIA V100, A100, 30 Series, or 40 Series GPUs**.
docs: update documentation to prepare for 0.2 release (#502) * docs: fix installation emoji * docs: set StarCoder-1B to be default model for docker install * docs: add `--chat-model` in model directory
2023-10-03 20:11:07 +00:00
| Model ID | License | Infilling Support | Apple M1/M2 Supports |
| --------------------------------------------------------------------- | :-----------------------------------------------------------------------------------------: | :---------------: | :------------: |
| [TabbyML/CodeLlama-13B](https://huggingface.co/TabbyML/CodeLlama-13B) | [Llama2](https://github.com/facebookresearch/llama/blob/main/LICENSE) | ✅ | ✅ |
| [TabbyML/CodeLlama-7B](https://huggingface.co/TabbyML/CodeLlama-7B) | [Llama2](https://github.com/facebookresearch/llama/blob/main/LICENSE) | ✅ | ✅ |
| [TabbyML/StarCoder-7B](https://huggingface.co/TabbyML/StarCoder-7B) | [BigCode-OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) | ✅ | ✅ |
| [TabbyML/StarCoder-3B](https://huggingface.co/TabbyML/StarCoder-3B) | [BigCode-OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) | ✅ | ✅ |
| [TabbyML/StarCoder-1B](https://huggingface.co/TabbyML/StarCoder-1B) | [BigCode-OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) | ✅ | ✅ |
| [TabbyML/J-350M](https://huggingface.co/TabbyML/J-350M) | [BSD-3](https://opensource.org/license/bsd-3-clause/) | ❌ | ❌ |
docs: update apple GPU support for StarCoder series
2023-09-16 03:44:33 +00:00
docs: add modelscope registry information to model directory
2023-10-04 05:23:47 +00:00
## Chat models (`--chat-model`)
docs: add device recommendations based on model size
2023-09-14 06:07:21 +00:00
docs: update documentation to prepare for 0.2 release (#502) * docs: fix installation emoji * docs: set StarCoder-1B to be default model for docker install * docs: add `--chat-model` in model directory
2023-10-03 20:11:07 +00:00
To ensure optimal response quality, and given that latency requirements are not stringent in this scenario, we recommend using a model with at least 3B parameters.
docs: add models directory (#437)
2023-09-12 15:41:00 +00:00
docs: update documentation to prepare for 0.2 release (#502) * docs: fix installation emoji * docs: set StarCoder-1B to be default model for docker install * docs: add `--chat-model` in model directory
2023-10-03 20:11:07 +00:00
| Model ID | License |
| ------------------------------------------------------------------------- | :---------------------------------------------------------------------------------: |
| [TabbyML/Mistral-7B](https://huggingface.co/TabbyML/Mistral-7B) | [Apache 2.0](https://opensource.org/licenses/Apache-2.0) |
docs: remove WizardCoder-15B from model directory
2023-10-04 03:27:51 +00:00
| [TabbyML/WizardCoder-3B](https://huggingface.co/TabbyML/WizardCoder-3B) | [OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) |
docs: add modelscope registry information to model directory
2023-10-04 05:23:47 +00:00
## Alternative Registry
By default, Tabby utilizes the [Hugging Face organization](https://huggingface.co/TabbyML) as its model registry. Mainland Chinese users have encountered challenges accessing Hugging Face for various reasons. The Tabby team has established a mirrored at [modelscope]([https://www.modelscope.cn](https://www.modelscope.cn/organization/TabbyML)), which can be utilized using the following environment variable:
```bash
TABBY_REGISTRY=modelscope tabby serve --model TabbyML/StarCoder-1B
```