root
/
tabby
mirror of https://gitee.com/zhang_1334717033/tabby.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
800 Commits
39 Branches
72 Tags
22 MiB
Commit Graph

14 Commits (dcf82d45b58ca40ec6c357339457ea823a11fdb4)

Author SHA1 Message Date
Meng Zhang 99a7053b6f
refactor: extract language configuration into individual toml file (#564) 
* refactor: extract language configuration into individual toml file

* feat: add golang language configuration (#565)
 2023-10-16 00:24:44 +00:00
Meng Zhang f05dd3a2f6
refactor: cleanup chat api make it message oriented (#497) 
* refactor: refactor into /chat/completions api

* Revert "feat: support request level stop words (#492)"

This reverts commit 0d6840e372.

* feat: adjust interface

* switch interface in tabby-playground

* move to chat/prompt, add unit test

* update interface
 2023-10-02 15:39:15 +00:00
Meng Zhang dfdd0373a6 fix: when llama model loads failed, panic in rust stack 2023-10-01 22:25:25 -07:00
Meng Zhang 2171ba72ff
refactor: cleanup llama cpp implementations to fix warnings (#495) 2023-09-30 08:37:36 -07:00
Meng Zhang 0d6840e372
feat: support request level stop words (#492) 2023-09-29 18:21:57 +00:00
Meng Zhang 486e507079
fix: correct Decoding behavior in incremental manner (#491) 
* feat: implement IncrementalDecoding

* refactor: use IncrementalDecoding for ctranslate2

* refactor: rename StopWords to DecodingFactory

* refactor: move decoding logic to tabby-inference

* feat: optimize decoding range

* cleanup
 2023-09-29 13:06:47 +00:00
Meng Zhang 5d9ca6928c
feat: update llama.cpp (#488) 
* feat: update llama.cpp

* remove useless include
 2023-09-28 23:59:59 +00:00
Meng Zhang 44f013f26e
feat: add /generate and /generate_streaming (#482) 
* feat: add generate_stream interface

* extract engine::create_engine

* feat add generate::generate

* support streaming in llama.cpp

* support streaming in ctranslate2

* update

* fix formatting

* refactor: extract helpers functions
 2023-09-28 17:20:50 +00:00
Meng Zhang 97eeb6b926
feat: update llama.cpp to fetch latest starcoder support (#452) 
* feat: bump llama.cpp to HEAD

* fix: turn off add_bos by default
 2023-09-16 03:41:49 +00:00
Meng Zhang 30afa19bc0
feat: add LLAMA_CPP_LOG_LEVEL to control log level of llama.cpp (#436) 2023-09-12 14:41:39 +00:00
Meng Zhang ad3b974d5c
feat: implement input truncation for llama-cpp-bindings (#416) 
* feat: implement input truncation for llama-cpp-bindings

* set max input length to 1024

* fix: batching tokens with n_batches

* fix batching
 2023-09-09 00:20:51 +08:00
Meng Zhang e93a971d0e
feat: tune llama metal backend performance (#393) 
* feat: support eos based stop

* feat: print performance stats after each inference

* update llama.cpp

* update commits
 2023-09-05 10:14:29 +08:00
Meng Zhang b0074d7e30
feat: support cancellation in llama backend [TAB-146] (#392) 
* feat: support cancellation in llama backend

* fix lint
 2023-09-03 02:15:54 +00:00
Meng Zhang 3573d4378e
feat: llama.cpp for metal support [TAB-146] (#391) 
* feat: init commit adding llama-cpp-bindings

* add llama.cpp submodule

* add LlamaEngine to hold llama context / llama model

* add cxxbridge

* add basic greedy sampling

* move files

* make compile success

* connect TextGeneration with LlamaEngine

* experimental support llama.cpp

* add metal device

* add Accelerate

* fix namespace for llama-cpp-bindings

* fix lint

* move stepping logic to rust

* add stop words package

* use stop-words in ctranslate2-bindings

* use raw string for regex

* use Arc<Tokenizer> for sharing tokenizers

* refactor: remove useless stop_words_encoding_offset

* switch to tokenizers 0.13.4-rc.3

* fix lints in cpp

* simplify implementation of greedy decoding

* feat: split metal feature for llama backend

* add ci

* update ci

* build tabby bin in ci build
 2023-09-03 09:59:07 +08:00