732f022708
Release 0.5.4
...
http-api-bindings@0.5.4
llama-cpp-bindings@0.5.4
tabby@0.5.4
tabby-common@0.5.4
tabby-download@0.5.4
tabby-inference@0.5.4
tabby-scheduler@0.5.4
Generated by cargo-workspaces
2023-11-07 13:12:33 -08:00
f72b87bf79
fix: deadlock between background job and requests ( #720 )
...
* fix: deadlock between background job and requests
* refactor: extract LlamaService
2023-11-07 13:12:13 -08:00
8046a482ea
Release 0.5.3
...
http-api-bindings@0.5.3
llama-cpp-bindings@0.5.3
tabby@0.5.3
tabby-common@0.5.3
tabby-download@0.5.3
tabby-inference@0.5.3
tabby-scheduler@0.5.3
Generated by cargo-workspaces
2023-11-07 00:57:48 -08:00
0226a379de
Release 0.5.2
...
http-api-bindings@0.5.2
llama-cpp-bindings@0.5.2
tabby@0.5.2
tabby-common@0.5.2
tabby-download@0.5.2
tabby-inference@0.5.2
tabby-scheduler@0.5.2
Generated by cargo-workspaces
2023-11-07 00:30:51 -08:00
551d1dc0ba
Release 0.5.2-rc.0
...
http-api-bindings@0.5.2-rc.0
llama-cpp-bindings@0.5.2-rc.0
tabby@0.5.2-rc.0
tabby-common@0.5.2-rc.0
tabby-download@0.5.2-rc.0
tabby-inference@0.5.2-rc.0
tabby-scheduler@0.5.2-rc.0
Generated by cargo-workspaces
2023-11-06 23:03:14 -08:00
c28f5838ce
fix: support cpu only run in llama.cpp cuda build
2023-11-06 23:02:31 -08:00
e87e78b74c
fix: llama.cpp requires kv cache to be N_CTX * parallelism ( #714 )
2023-11-06 23:02:28 -08:00
7ca3221b52
fix: when there's an error happens in background inference loop, it should exit the process ( #713 )
2023-11-06 23:02:23 -08:00
03a9c7dac3
chore: add machete check to ensure no unused dependencies ( #701 )
...
* refactor: remove useless dependencies
* add machete
2023-11-06 23:02:12 -08:00
36d13d2837
fix(llama.cpp): wrongly index for n_seq in warmup
2023-11-04 20:12:13 -07:00
01ce18fe1a
fix: llama.cpp warmp logic
2023-11-04 20:12:09 -07:00
536c7e86a0
Release 0.5.0
...
http-api-bindings@0.5.0
llama-cpp-bindings@0.5.0
tabby@0.5.0
tabby-common@0.5.0
tabby-download@0.5.0
tabby-inference@0.5.0
tabby-scheduler@0.5.0
Generated by cargo-workspaces
2023-11-03 18:00:38 -07:00
281d189848
Release 0.5.0-rc.4
...
http-api-bindings@0.5.0-rc.4
llama-cpp-bindings@0.5.0-rc.4
tabby@0.5.0-rc.4
tabby-common@0.5.0-rc.4
tabby-download@0.5.0-rc.4
tabby-inference@0.5.0-rc.4
tabby-scheduler@0.5.0-rc.4
Generated by cargo-workspaces
2023-11-03 17:36:51 -07:00
03fe1e9f6b
Release 0.5.0-rc.3
...
http-api-bindings@0.5.0-rc.3
llama-cpp-bindings@0.5.0-rc.3
tabby@0.5.0-rc.3
tabby-common@0.5.0-rc.3
tabby-download@0.5.0-rc.3
tabby-inference@0.5.0-rc.3
tabby-scheduler@0.5.0-rc.3
Generated by cargo-workspaces
2023-11-03 13:53:31 -07:00
a5bb20becb
Release 0.5.0-rc.2
...
http-api-bindings@0.5.0-rc.2
llama-cpp-bindings@0.5.0-rc.2
tabby@0.5.0-rc.2
tabby-common@0.5.0-rc.2
tabby-download@0.5.0-rc.2
tabby-inference@0.5.0-rc.2
tabby-scheduler@0.5.0-rc.2
Generated by cargo-workspaces
2023-11-03 13:26:51 -07:00
f97cdf2ad9
Release 0.5.0-rc.1
...
http-api-bindings@0.5.0-rc.1
llama-cpp-bindings@0.5.0-rc.1
tabby@0.5.0-rc.1
tabby-common@0.5.0-rc.1
tabby-download@0.5.0-rc.1
tabby-inference@0.5.0-rc.1
tabby-scheduler@0.5.0-rc.1
Generated by cargo-workspaces
2023-11-03 13:23:25 -07:00
61605ca553
Release 0.5.0-rc.0
...
http-api-bindings@0.5.0-rc.0
llama-cpp-bindings@0.5.0-rc.0
tabby@0.5.0-rc.0
tabby-common@0.5.0-rc.0
tabby-download@0.5.0-rc.0
tabby-inference@0.5.0-rc.0
tabby-scheduler@0.5.0-rc.0
Generated by cargo-workspaces
2023-11-03 11:35:26 -07:00
acb3a33d78
fix: handle non utf-8 / utf-16 error
2023-11-02 16:29:30 -07:00
eb34850a5e
fix: output err if step failed
2023-11-02 16:15:11 -07:00
4c7eae584e
feat: add model warmup logic ( #693 )
2023-11-02 23:07:32 +00:00
296342efd8
refactor: use llama.cpp tokenizer ( #683 )
...
* refactor: switch to llama.cpp tokenizer to simplify implementation
* refactor: remove tokenizer dependency from tabby
* refactor: renaming decoding to stop condition
* refactor: remove tokenizer dependency
* refactor: remove submodule
* chore: update formatting
* move tokenization to c++
2023-10-31 22:16:09 +00:00
89a63dbf33
fix: when send failed, treat the request as stopped ( #673 )
2023-10-30 06:27:09 +00:00
7330d75de6
chore: clear cache when there's no active requests
2023-10-29 16:30:30 -07:00
7bd99d14c0
feat: support continuous batching in llama.cpp backend ( #659 )
...
* refactor: switch back to llama batch interface
* feat: support cont batching
2023-10-28 23:37:05 -07:00
444222683a
fix(llama.cpp): bump upstream fix for starcoder model on cuda
2023-10-28 02:03:34 -07:00
9309e0314f
fix: fix docker build
2023-10-27 21:25:45 -07:00
6dd12ce1ec
fix: adding cuda search path to docker build.
2023-10-27 19:40:35 -07:00
2d948639be
fix: docker build for llama cuda backend
2023-10-27 16:36:54 -07:00
23bd542cec
feat: switch cuda backend to llama.cpp ( #656 )
...
* feat: switch cuda backend to llama.cpp
* fix
* fix
2023-10-27 13:41:22 -07:00
f37840566b
feat: upgrade llama.cpp ( #645 )
...
* feat: upgrade llama.cpp
* update download files
* update changelog
* Update CHANGELOG.md
* Update CHANGELOG.md
2023-10-27 12:18:46 -07:00
1a4c2aa71f
feat: swtich cpu backend to llama.cpp ( #638 )
...
* feat: swtich Cpu backend to llama.cpp
* feat: switch cpu serving to ggml
* fix cargo.toml
* use optional dependency
* fix compliation
* update ci target
2023-10-25 15:40:11 -07:00
e171776774
Release 0.5.0-dev
...
ctranslate2-bindings@0.5.0-dev
http-api-bindings@0.5.0-dev
llama-cpp-bindings@0.5.0-dev
rust-cxx-cmake-bridge@0.5.0-dev
tabby@0.5.0-dev
tabby-common@0.5.0-dev
tabby-download@0.5.0-dev
tabby-inference@0.5.0-dev
tabby-scheduler@0.5.0-dev
Generated by cargo-workspaces
2023-10-24 13:05:33 -07:00
99a7053b6f
refactor: extract language configuration into individual toml file ( #564 )
...
* refactor: extract language configuration into individual toml file
* feat: add golang language configuration (#565 )
2023-10-16 00:24:44 +00:00
82e893d569
Release 0.4.0-dev
...
ctranslate2-bindings@0.4.0-dev
http-api-bindings@0.4.0-dev
llama-cpp-bindings@0.4.0-dev
rust-cxx-cmake-bridge@0.4.0-dev
tabby@0.4.0-dev
tabby-common@0.4.0-dev
tabby-download@0.4.0-dev
tabby-inference@0.4.0-dev
tabby-scheduler@0.4.0-dev
Generated by cargo-workspaces
2023-10-13 17:54:14 -07:00
4dbaf4f312
Release 0.3.0
...
ctranslate2-bindings@0.3.0
http-api-bindings@0.3.0
llama-cpp-bindings@0.3.0
rust-cxx-cmake-bridge@0.3.0
tabby@0.3.0
tabby-common@0.3.0
tabby-download@0.3.0
tabby-inference@0.3.0
tabby-scheduler@0.3.0
Generated by cargo-workspaces
2023-10-13 17:45:07 -07:00
eb463ba496
Release 0.3.0-rc.1
...
ctranslate2-bindings@0.3.0-rc.1
http-api-bindings@0.3.0-rc.1
llama-cpp-bindings@0.3.0-rc.1
rust-cxx-cmake-bridge@0.3.0-rc.1
tabby@0.3.0-rc.1
tabby-common@0.3.0-rc.1
tabby-download@0.3.0-rc.1
tabby-inference@0.3.0-rc.1
tabby-scheduler@0.3.0-rc.1
Generated by cargo-workspaces
2023-10-13 11:43:34 -07:00
182aceed41
Release 0.3.0-rc.0
...
ctranslate2-bindings@0.3.0-rc.0
http-api-bindings@0.3.0-rc.0
llama-cpp-bindings@0.3.0-rc.0
tabby@0.3.0-rc.0
tabby-common@0.3.0-rc.0
tabby-download@0.3.0-rc.0
tabby-inference@0.3.0-rc.0
tabby-scheduler@0.3.0-rc.0
Generated by cargo-workspaces
2023-10-13 11:24:36 -07:00
6dbb712918
Release 0.3.0-dev
...
ctranslate2-bindings@0.3.0-dev
http-api-bindings@0.3.0-dev
llama-cpp-bindings@0.3.0-dev
tabby@0.3.0-dev
tabby-common@0.3.0-dev
tabby-download@0.3.0-dev
tabby-inference@0.3.0-dev
tabby-scheduler@0.3.0-dev
Generated by cargo-workspaces
2023-10-09 19:39:27 -07:00
1731c3075e
chore: Update version to 0.2.0
2023-10-03 13:32:21 -07:00
692c2fe0fd
Release 0.2.0-rc.0
...
ctranslate2-bindings@0.2.0-rc.0
http-api-bindings@0.2.0-rc.0
llama-cpp-bindings@0.2.0-rc.0
tabby@0.2.0-rc.0
tabby-common@0.2.0-rc.0
tabby-download@0.2.0-rc.0
tabby-inference@0.2.0-rc.0
tabby-scheduler@0.2.0-rc.0
Generated by cargo-workspaces
2023-10-02 19:14:12 -07:00
f05dd3a2f6
refactor: cleanup chat api make it message oriented ( #497 )
...
* refactor: refactor into /chat/completions api
* Revert "feat: support request level stop words (#492 )"
This reverts commit 0d6840e372
2023-10-02 15:39:15 +00:00
dfdd0373a6
fix: when llama model loads failed, panic in rust stack
2023-10-01 22:25:25 -07:00
2171ba72ff
refactor: cleanup llama cpp implementations to fix warnings ( #495 )
2023-09-30 08:37:36 -07:00
0d6840e372
feat: support request level stop words ( #492 )
2023-09-29 18:21:57 +00:00
486e507079
fix: correct Decoding behavior in incremental manner ( #491 )
...
* feat: implement IncrementalDecoding
* refactor: use IncrementalDecoding for ctranslate2
* refactor: rename StopWords to DecodingFactory
* refactor: move decoding logic to tabby-inference
* feat: optimize decoding range
* cleanup
2023-09-29 13:06:47 +00:00
5d9ca6928c
feat: update llama.cpp ( #488 )
...
* feat: update llama.cpp
* remove useless include
2023-09-28 23:59:59 +00:00
56b7b850af
fix: Linkage issue on latest xcode commandline tools clang ( #486 )
2023-09-28 17:46:02 +00:00
44f013f26e
feat: add /generate and /generate_streaming ( #482 )
...
* feat: add generate_stream interface
* extract engine::create_engine
* feat add generate::generate
* support streaming in llama.cpp
* support streaming in ctranslate2
* update
* fix formatting
* refactor: extract helpers functions
2023-09-28 17:20:50 +00:00
97eeb6b926
feat: update llama.cpp to fetch latest starcoder support ( #452 )
...
* feat: bump llama.cpp to HEAD
* fix: turn off add_bos by default
2023-09-16 03:41:49 +00:00
30afa19bc0
feat: add LLAMA_CPP_LOG_LEVEL to control log level of llama.cpp ( #436 )
2023-09-12 14:41:39 +00:00
Meng Zhang