From 93b6766fcaf8c874323c998694b1a90874ef3be6 Mon Sep 17 00:00:00 2001 From: Meng Zhang Date: Wed, 4 Oct 2023 23:33:51 +0800 Subject: [PATCH] docs: Update MODEL_SPEC.md (#504) Fix #503 --- MODEL_SPEC.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/MODEL_SPEC.md b/MODEL_SPEC.md index 442b17b..32cf371 100644 --- a/MODEL_SPEC.md +++ b/MODEL_SPEC.md @@ -45,6 +45,6 @@ ct2-transformers-converter --model ./ --output_dir ctranslate2 --quantization=fl *Note that the model itself must be compatible with ctranslate2.* ### ggml/ -This directory contains binary files used by the [llama.cpp](https://github.com/ggerganov/llama.cpp) inference engine. Tabby utilizes ctranslate2 for inference on the `metal` device. +This directory contains binary files used by the [llama.cpp](https://github.com/ggerganov/llama.cpp) inference engine. Tabby utilizes ggml for inference on the `metal` device. Currently, only `q8_0.gguf` in this directory is in use. You can refer to the instructions in llama.cpp to learn how to acquire it.