3.0 KiB
3.0 KiB
| sidebar_position |
|---|
| 4 |
🧑🔬 Models Directory
Completion models (For --model)
We recommend using
- small models (less than 400M) for CPU devices.
- For 1B to 7B models, it's advisable to have at least NVIDIA T4, 10 Series, or 20 Series GPUs.
- For 7B to 13B models, we recommend using NVIDIA V100, A100, 30 Series, or 40 Series GPUs.
| Model ID | License | Infilling Support | Apple M1/M2 Supports |
|---|---|---|---|
| TabbyML/CodeLlama-13B | Llama2 | ✅ | ✅ |
| TabbyML/CodeLlama-7B | Llama2 | ✅ | ✅ |
| TabbyML/StarCoder-7B | BigCode-OpenRAIL-M | ✅ | ✅ |
| TabbyML/StarCoder-3B | BigCode-OpenRAIL-M | ✅ | ✅ |
| TabbyML/StarCoder-1B | BigCode-OpenRAIL-M | ✅ | ✅ |
| TabbyML/J-350M | BSD-3 | ❌ | ❌ |
Chat models (For --chat-model)
To ensure optimal response quality, and given that latency requirements are not stringent in this scenario, we recommend using a model with at least 3B parameters.
| Model ID | License |
|---|---|
| TabbyML/WizardCoder-15B | OpenRAIL-M |
| TabbyML/Mistral-7B | Apache 2.0 |
| TabbyML/WizardCoder-3B | OpenRAIL-M |