Commit Graph

13 Commits (296342efd84e862d4c3de7a6e2b359288d46b7fb)

Author SHA1 Message Date
Meng Zhang 296342efd8
refactor: use llama.cpp tokenizer (#683)
* refactor: switch to llama.cpp tokenizer to simplify implementation

* refactor: remove tokenizer dependency from tabby

* refactor: renaming decoding to stop condition

* refactor: remove tokenizer dependency

* refactor: remove submodule

* chore: update formatting

* move tokenization to c++
2023-10-31 22:16:09 +00:00
Meng Zhang f37840566b
feat: upgrade llama.cpp (#645)
* feat: upgrade llama.cpp

* update download files

* update changelog

* Update CHANGELOG.md

* Update CHANGELOG.md
2023-10-27 12:18:46 -07:00
Meng Zhang e780031ed6
feat: add ggml fp16 / q8_0 files (#407)
* feat: add ggml fp16 / q8_0 files

* add q8_0.gguf to optional download files

* add download options to split ctranslate2 files and ggml files
2023-09-06 17:12:29 +00:00
Meng Zhang 3573d4378e
feat: llama.cpp for metal support [TAB-146] (#391)
* feat: init commit adding llama-cpp-bindings

* add llama.cpp submodule

* add LlamaEngine to hold llama context / llama model

* add cxxbridge

* add basic greedy sampling

* move files

* make compile success

* connect TextGeneration with LlamaEngine

* experimental support llama.cpp

* add metal device

* add Accelerate

* fix namespace for llama-cpp-bindings

* fix lint

* move stepping logic to rust

* add stop words package

* use stop-words in ctranslate2-bindings

* use raw string for regex

* use Arc<Tokenizer> for sharing tokenizers

* refactor: remove useless stop_words_encoding_offset

* switch to tokenizers 0.13.4-rc.3

* fix lints in cpp

* simplify implementation of greedy decoding

* feat: split metal feature for llama backend

* add ci

* update ci

* build tabby bin in ci build
2023-09-03 09:59:07 +08:00
Meng Zhang d0f6ad2d2a
feat: add anonymous usage tracker (#342)
* feat: add anonymous usage tracker

* improve deps

* update

* update
2023-08-09 07:31:13 +00:00
Meng Zhang 9af6607f85
feat: add dataset building (#231)
* feat: build dataset in scheduler

* add file rotate
2023-06-12 15:19:43 +00:00
Meng Zhang 249d51d0f5
feat: add indexer [TAB-17] (#199)
* add basic indexer

* formatting
2023-06-05 22:18:10 +00:00
Meng Zhang 272dde9769
refactor: rust nightly format (#197)
* chore: turn on group format

* turn on nightly fmt
2023-06-05 14:17:07 -07:00
Meng Zhang da02d471a9
chore: add tabby-scheduler [TAB-17] (#192)
* add scheduler

* update fmt

* add integration tests for scheduler
2023-06-05 03:08:43 +00:00
Meng Zhang 2779da3cba
feat: supports FIM inference [TAB-46] (#183)
* Add prefix / suffix

* update

* feat: support segments in inference

* chore: add tabby.json in model repository to store prompt_template

* make prompt_template optional.

* download tabby.json in downloader
2023-06-02 16:47:48 -07:00
Meng Zhang 3cac2607e7
refactor: improve error handlings, fix clippy warnings (#181)
* refactor: minor improvements on error handling

* refactor: cleanup error handlings

* update

* update

* fix

* add clippy / test workflow

* fix clippy

* fix clippy

* update
2023-06-01 17:23:05 -07:00
Meng Zhang f3b37b253b
feat: add events logger (#170)
* add common events

* log events in serve

* formatting

* extract serde_conv

* update

* update

* fix logging

* update
2023-05-30 15:44:29 -07:00
Meng Zhang d8cee4adac
chore: extract tabby-common (#169)
* chore: extract tabby-common

* simplify
2023-05-29 23:39:02 -07:00