root
/
tabby
mirror of https://gitee.com/zhang_1334717033/tabby.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

docs: Update MODEL_SPEC.md (#504) 

Fix #503
wsxiaoys-patch-1
Meng Zhang 2023-10-04 23:33:51 +08:00 committed by GitHub
parent f44e87ff0e
commit 93b6766fca
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
MODEL_SPEC.md
View File

@ -45,6 +45,6 @@ ct2-transformers-converter --model ./ --output_dir ctranslate2 --quantization=fl
*Note that the model itself must be compatible with ctranslate2.* *Note that the model itself must be compatible with ctranslate2.*
### ggml/ ### ggml/
This directory contains binary files used by the [llama.cpp](https://github.com/ggerganov/llama.cpp) inference engine. Tabby utilizes ctranslate2 for inference on the `metal` device. This directory contains binary files used by the [llama.cpp](https://github.com/ggerganov/llama.cpp) inference engine. Tabby utilizes ggml for inference on the `metal` device.
Currently, only `q8_0.gguf` in this directory is in use. You can refer to the instructions in llama.cpp to learn how to acquire it. Currently, only `q8_0.gguf` in this directory is in use. You can refer to the instructions in llama.cpp to learn how to acquire it.