1ad871e1ff
feat: add debug request / response to visualize prompting with source code index ( #544 )
...
* feat: logs segments in completion log
* feat: tune prompt format and improve testing
* add debug options for easier of visualizing the prompt
* update
2023-10-12 19:27:52 -07:00
888fbca7e1
feat: support indexing typescript / javascript / tsx / jsx ( #539 )
2023-10-12 00:42:58 +00:00
92c1f5a8c0
feat: turn on index server if avaiable ( #536 )
2023-10-11 23:27:41 +00:00
1f9e248dfa
refactor: support multiple page in playground ( #537 )
2023-10-11 23:27:20 +00:00
99c39375fd
feat: set default empty screen questions for tabby-playground ( #535 )
2023-10-11 22:31:35 +00:00
75d2944fb6
feat: support loading the source code index whenever it's ready in file system ( #530 )
...
* feat: support loading index whenever it's ready
* fix test
2023-10-10 21:35:20 -07:00
6dbb712918
Release 0.3.0-dev
...
ctranslate2-bindings@0.3.0-dev
http-api-bindings@0.3.0-dev
llama-cpp-bindings@0.3.0-dev
tabby@0.3.0-dev
tabby-common@0.3.0-dev
tabby-download@0.3.0-dev
tabby-inference@0.3.0-dev
tabby-scheduler@0.3.0-dev
Generated by cargo-workspaces
2023-10-09 19:39:27 -07:00
d7a8f50de8
fix: use spawn_blocking for sub task without await
2023-10-09 19:38:28 -07:00
d21a4de79c
chore: set max timeout for /v1/completions handler ( #526 )
...
* chore: set max timeout for /v1/completions handler
* refactor: extract sub routers
* fix
2023-10-09 18:44:55 -07:00
24eadf0de8
refactor: make /v1/health accept GET requests ( #527 )
2023-10-09 18:34:56 +00:00
3eb5f4132c
chore: add 'v*' match pattern to restrict git describe only compare against a actual release
2023-10-09 11:13:37 -07:00
0f8ee7f589
refactor: move http engine creation to its sub crates ( #524 )
2023-10-09 17:37:04 +00:00
8c09f75360
refactor: extract language related data into languages.rs ( #518 )
...
* refactor: extract language related data into languages.rs
* fix
* cleanup index
* fix
* further sanitize
* add a score threshold
2023-10-07 01:40:21 +00:00
d85a7892d1
feat: connect prompt rewriting part ( #517 )
...
* feat: enable /v1beta/search if index is available
* make prompt rewriting work
* update
* fix test
* fix api doc
2023-10-07 00:29:24 +00:00
8497fb1372
feat: implement /v1beta/search interface ( #516 )
...
* feat: implement /v1beta/search interface
* update
* update
* improve debugger
2023-10-06 18:54:12 +00:00
fd2a1ab865
fix: switch back to regex based implementation for stop words ( #513 )
2023-10-06 09:04:37 +00:00
4c00ac06fb
fix(download): mark ggml model downloading should be optional, as ggml is only used for metal backend for now ( #512 )
2023-10-05 16:54:56 +00:00
9cd2accbaa
feat: adjust code indexing logic ( #510 )
2023-10-05 05:29:41 +00:00
1babc38902
feat: add datset viewer, remove treesitter languages that hasn't been verified ( #509 )
...
* refactor: remove not verified tree sitter queries
* feat(experimental): add dataset viewer
update
2023-10-04 21:49:42 -07:00
55f68d4224
test: unit test for indexing job ( #508 )
...
* test: unit test for indexing job
* update
* reduce test fixture length
2023-10-04 23:27:19 +00:00
f7ebce2514
refactor: deprecate --compute-type ( #505 )
2023-10-04 18:45:34 +00:00
8a03c9bf17
refactor: use / as server url
...
Swagger access to the page, regardless of host / port, will just work.
2023-10-03 18:33:56 -07:00
2d5b3e4ff5
chore: release v0.2.1
2023-10-03 17:13:39 -07:00
503c44e7c5
fix: playground environment misconfig
2023-10-03 17:10:02 -07:00
b3b498624c
feat: deprecate num_replicas_per_thread, generate default value for it
2023-10-03 17:02:37 -07:00
1afba47059
feat: allow set num_replicas_per_device for CUDA to increase throughput
2023-10-03 15:52:25 -07:00
ceaa7ab012
chore: update main branch to v0.3.0-dev
2023-10-03 13:38:27 -07:00
1731c3075e
chore: Update version to 0.2.0
2023-10-03 13:32:21 -07:00
0e5128e8fb
feat: add chat_template field in tabby.json
2023-10-03 11:46:05 -07:00
7fc76228f7
chore: add debug log for /chat interface
2023-10-03 11:38:58 -07:00
692c2fe0fd
Release 0.2.0-rc.0
...
ctranslate2-bindings@0.2.0-rc.0
http-api-bindings@0.2.0-rc.0
llama-cpp-bindings@0.2.0-rc.0
tabby@0.2.0-rc.0
tabby-common@0.2.0-rc.0
tabby-download@0.2.0-rc.0
tabby-inference@0.2.0-rc.0
tabby-scheduler@0.2.0-rc.0
Generated by cargo-workspaces
2023-10-02 19:14:12 -07:00
6306bb3f01
fix: if local file doens't exist, local_cache_key should be cleared ( #501 )
...
* fix: if local file doens't exist, local_cache_key should be cleared
* fix
2023-10-02 23:48:35 +00:00
ce20bd6154
refactor: use RegexSet for cleaer stop regex construction ( #499 )
...
* fix: add a regression test cased for stop words regex matching
* refactor: use RegexSet for cleaer stop regex construction
2023-10-02 23:21:51 +00:00
63612d5a67
fix(tabby-download): even when prefer_local_file is set to true, we should still check for remote (if network is avaialble), to see if a file should be upgraded ( #500 )
2023-10-02 23:09:57 +00:00
80a17aea37
feat: only show /v1/chat api if --chat-model is set ( #498 )
2023-10-02 17:17:27 +00:00
f05dd3a2f6
refactor: cleanup chat api make it message oriented ( #497 )
...
* refactor: refactor into /chat/completions api
* Revert "feat: support request level stop words (#492 )"
This reverts commit 0d6840e372
2023-10-02 15:39:15 +00:00
dfdd0373a6
fix: when llama model loads failed, panic in rust stack
2023-10-01 22:25:25 -07:00
2171ba72ff
refactor: cleanup llama cpp implementations to fix warnings ( #495 )
2023-09-30 08:37:36 -07:00
aea8c74bdc
feat: add OpenAPI link to playground
2023-09-29 18:20:38 -07:00
10bf2d6c0c
feat: add param --instruct-model, allowing specify different model for q&a use cases. ( #494 )
2023-09-29 23:44:53 +00:00
eb15933255
feat: add tabby playground for q&a use case ( #493 )
...
* init commit
* support chat
* add theme toggle
* limit message to 2 lines
* update
* update formatting
* update
* update
* update
* fix formatting
* update
2023-09-29 15:51:54 -07:00
0d6840e372
feat: support request level stop words ( #492 )
2023-09-29 18:21:57 +00:00
486e507079
fix: correct Decoding behavior in incremental manner ( #491 )
...
* feat: implement IncrementalDecoding
* refactor: use IncrementalDecoding for ctranslate2
* refactor: rename StopWords to DecodingFactory
* refactor: move decoding logic to tabby-inference
* feat: optimize decoding range
* cleanup
2023-09-29 13:06:47 +00:00
5d9ca6928c
feat: update llama.cpp ( #488 )
...
* feat: update llama.cpp
* remove useless include
2023-09-28 23:59:59 +00:00
a159c2358d
refactor: move generate / generate_stream to /v1beta ( #487 )
2023-09-28 23:58:17 +00:00
56b7b850af
fix: Linkage issue on latest xcode commandline tools clang ( #486 )
2023-09-28 17:46:02 +00:00
44f013f26e
feat: add /generate and /generate_streaming ( #482 )
...
* feat: add generate_stream interface
* extract engine::create_engine
* feat add generate::generate
* support streaming in llama.cpp
* support streaming in ctranslate2
* update
* fix formatting
* refactor: extract helpers functions
2023-09-28 17:20:50 +00:00
d42942c379
feat: support ModelScope for model registry downloading ( #477 )
...
* feat: update cache info file after each file got downloaded
* refactor: extract Downloader for model downloading logic
* refactor: extract HuggingFaceRegistry
* refactor: extract serde_json to workspace dependency
* feat: add ModelScopeRegistry
* refactor: extract registry to its sub dir.
* feat: add scripts to mirror hf model to modelscope
2023-09-26 11:52:11 -07:00
fb5a5971d3
feat: proxy server address mapping to the model server ( #461 )
...
* feat: proxy server address mapping to the model server
* fix: add swagger in Config
* refactor: add_proxy_server
* fix: missing semicolo
2023-09-21 07:06:51 +00:00
de3a2271d6
fix(tabby): fix swagger's local server use local port ( #458 )
...
* fixed: swagger's local server use local port
* fix: extract fn add_localhost_server
* fix: add_localhost_server return doc
2023-09-19 04:36:08 +00:00
c107c991ff
chore: bump tabby version to 0.1.1
2023-09-17 17:09:56 +08:00
97eeb6b926
feat: update llama.cpp to fetch latest starcoder support ( #452 )
...
* feat: bump llama.cpp to HEAD
* fix: turn off add_bos by default
2023-09-16 03:41:49 +00:00
86e48afbe0
feat: bump ctranslate2 to HEAD
2023-09-16 02:40:23 +08:00
30afa19bc0
feat: add LLAMA_CPP_LOG_LEVEL to control log level of llama.cpp ( #436 )
2023-09-12 14:41:39 +00:00
1e7ecce697
fix: usize overflow issue in ctranslate2 max_length truncation
2023-09-12 20:56:35 +08:00
1ccf9b2323
refactor: run make fix
2023-09-11 12:58:38 +08:00
09efa1b22b
docs: add client extensions link in swagger landing page
2023-09-11 12:55:42 +08:00
e3c4a77fff
feat: add support fastchat http bindings ( #421 )
...
* feat: add support fastchat http bindings
Signed-off-by: Lei Wen <wenlei03@qiyi.com>
Co-authored-by: Lei Wen <wenlei03@qiyi.com>
2023-09-10 22:17:58 +08:00
f0ed366420
feat: add support vertex-ai http bindings ( #419 )
...
* feat: add support vertex-ai http bindings
* support prefix / suffix
2023-09-09 11:22:58 +00:00
17397c8c8c
feat: add http api bindings ( #410 )
...
* feat: add http-api-bindings
* feat: add http-api-bindings
* hand max_input_length
* rename
* update
* update
* add examples/simple.rs
* update
* add default value for stop words
* update
* fix lint
* update
2023-09-09 03:59:42 +00:00
ad3b974d5c
feat: implement input truncation for llama-cpp-bindings ( #416 )
...
* feat: implement input truncation for llama-cpp-bindings
* set max input length to 1024
* fix: batching tokens with n_batches
* fix batching
2023-09-09 00:20:51 +08:00
87b6b34120
feat: implement input truncation with options.max_input_length ( #415 )
2023-09-08 10:01:03 +00:00
e780031ed6
feat: add ggml fp16 / q8_0 files ( #407 )
...
* feat: add ggml fp16 / q8_0 files
* add q8_0.gguf to optional download files
* add download options to split ctranslate2 files and ggml files
2023-09-06 17:12:29 +00:00
a207520571
feat: turn on metal device by default on macosx / aarch64 devices ( #398 )
2023-09-05 13:03:49 +08:00
d85cd81139
fix: ensure default suffix to be non-empty ( #400 )
2023-09-05 03:45:29 +00:00
e93a971d0e
feat: tune llama metal backend performance ( #393 )
...
* feat: support eos based stop
* feat: print performance stats after each inference
* update llama.cpp
* update commits
2023-09-05 10:14:29 +08:00
2472cf3b55
test: use function call style snippet for prompt builder unit test ( #395 )
...
* test: better tests for build_prefix()
* chore
* chore: resolve comments
2023-09-04 04:54:18 +00:00
74073aa77a
test: add build prefix test and debug chars counting [TAB-184] ( #394 )
...
* test: add count char test
* chore: fix lint
* chore
* chore
2023-09-03 20:57:26 +08:00
3c7c8d9293
feat: add cargo test to github actions and run only unit tests in ci [TAB-185] ( #390 )
...
* feat: add cargo test to github actions
* chore: fix lint
* chore: add openblas dependency
* chore: update build dependency
* chore: resolve comments
* chore: fix lint
* chore: fix lint
* chore: test installing dependencies
* chore: refactor integ test
* update ci
* cleanup
---------
Co-authored-by: Meng Zhang <meng@tabbyml.com>
2023-09-03 05:04:52 +00:00
c8339a2912
refactor: use TabbyML/llama.cpp submodule
2023-09-03 12:38:54 +08:00
3acd5d9bc4
refactor: remove llama.cpp subtree
2023-09-03 12:37:26 +08:00
92c8ae8ee7
feat: embed ggml-metal.metal
2023-09-03 10:41:03 +08:00
ed6c5b2e60
Merge commit 'aad80a58b07836bfbf6aedd50993bc54b4257388' as 'crates/llama-cpp-bindings/llama.cpp'
2023-09-03 10:07:10 +08:00
d4137463ef
remove llama.cpp submodule
2023-09-03 10:04:26 +08:00
e360b438b4
fix lint
2023-09-03 10:01:28 +08:00
3f7aa99b0d
feat: support cancellation in llama backend
2023-09-03 09:59:40 +08:00
3573d4378e
feat: llama.cpp for metal support [TAB-146] ( #391 )
...
* feat: init commit adding llama-cpp-bindings
* add llama.cpp submodule
* add LlamaEngine to hold llama context / llama model
* add cxxbridge
* add basic greedy sampling
* move files
* make compile success
* connect TextGeneration with LlamaEngine
* experimental support llama.cpp
* add metal device
* add Accelerate
* fix namespace for llama-cpp-bindings
* fix lint
* move stepping logic to rust
* add stop words package
* use stop-words in ctranslate2-bindings
* use raw string for regex
* use Arc<Tokenizer> for sharing tokenizers
* refactor: remove useless stop_words_encoding_offset
* switch to tokenizers 0.13.4-rc.3
* fix lints in cpp
* simplify implementation of greedy decoding
* feat: split metal feature for llama backend
* add ci
* update ci
* build tabby bin in ci build
2023-09-03 09:59:07 +08:00
5dff349801
add single line comment to languages so they can be used in prompting [TAB-181] ( #388 )
...
* chore: add comment signs to extended languages
* Update crates/tabby/src/serve/completions/prompt.rs
---------
Co-authored-by: Meng Zhang <meng@tabbyml.com>
2023-09-01 03:43:27 +00:00
63c00494f3
test: unit tests to prompt builder [TAB-180] ( #387 )
...
* test: unit tests to prompt builder
* chore: fix typo
* chore: fix lint
* chore: resolve comments
2023-09-01 09:20:20 +08:00
90aadad3ce
feat: map js,ts,jsx and tsx to js-ts as unified language [TAB-181] ( #386 )
...
* feat: reduce js, ts, jsx and tsx to js-ts
* chore: refactor and add language reducing to both indexing and dataset jobs
* chore: only reduce language in dataset job
* chore: only reduce language in index job
* chore: fix lint
* chore: resolve comments
2023-08-31 17:21:39 +00:00
e5598e63f2
feat: extend language [TAB-181] ( #385 )
...
* feat: extend indexer's language support
* feat: extend language support
* chore: add support for mjs and mts
* chore: fix lint
2023-08-31 07:36:57 +00:00
c44a9c7195
fix: correct git_describe in /health ( #383 )
...
* fix: add missing Version component in OpenAPI definition
* fix: allow tag / dirty in git describe
2023-08-31 01:06:36 +00:00
57baecb370
fix: switch default running backend to openblas on x86 linux ( #380 )
2023-08-30 14:19:35 +00:00
054aefaf15
chore: add linux static build ( #379 )
...
* chore: add linux static build
* add touch
* update build env
* add sudo
* fix: protobuf ubuntu target
2023-08-30 18:45:05 +08:00
3526ca3164
chore: build with ruy (cpu only) on static mode for linux. ( #378 )
...
* chore: build with ruy (cpu only) on static mode for linux.
* update cmake min version
2023-08-30 18:04:40 +08:00
fc9a623e72
feat: add logging on server starting ( #372 )
2023-08-28 06:12:00 +00:00
65836ee199
feat: add stop words encoding offset for ctranslate model config ( #371 )
...
* feat: add stop words encoding offset for ctranslate model config
* feat: set default suffix to \n
* add special treatment for bytefallback tokens
2023-08-28 14:07:01 +08:00
2a91a21787
feat: add gpu info to health state [TAB-162] ( #364 )
...
* feat: add gpu info to health response
* chore: error handling
* chore: refactor cpu manager code
* chore: typo
* chore: fix context mutability
* chore: fix context mutability
* feat: add link to NVML lib
* chore: refactor
* lint
* chore: resolve comments
* chore: fix typo
* chore: fix
* chore: resolve comments
* chore: fix
* chore: resolve comments
2023-08-21 18:06:38 +08:00
b1ad936033
feat: add version information in health state. ( #363 )
...
* feat: add git_hash in health state
* add more version information in health state
2023-08-20 15:21:12 +00:00
df45573501
feat: reduce ServeHealth event to every 300s to reduce event volume ( #362 )
2023-08-20 12:36:59 +00:00
2026b4dd0e
feat: add architecture/cpu info to health api response [TAB-162] ( #355 )
...
* feat: add architecture, cpu and gpu info to health command
* chore: fix
* chore: fix
* chore: fix
* chore: fix lint
* chore: fix lint
* chore: remove gpu
* chore: resolve comments
* chore: resolve comments
* Update health.rs
---------
Co-authored-by: Meng Zhang <meng@tabbyml.com>
2023-08-15 15:22:03 +00:00
dbc89831b1
feat: add serve health heartbeat ( #343 )
...
* add serve health tracking
* fix lint
* fix
2023-08-09 08:08:42 +00:00
d0f6ad2d2a
feat: add anonymous usage tracker ( #342 )
...
* feat: add anonymous usage tracker
* improve deps
* update
* update
2023-08-09 07:31:13 +00:00
220fcc0d65
fix: make `config.experimental` optional ( #339 )
...
* fix: make config.experimental` optional
* add unit test for empty toml config
2023-08-07 09:53:00 +00:00
4eaae27ed3
Update Cargo.toml ( #331 )
2023-08-03 19:55:00 +08:00
6a50902ca7
fix: support ctranslate2 rev7 vocab files (.json) ( #327 )
2023-08-02 13:36:31 +00:00
57c811b30f
fix: improve download logging ( #325 )
...
* Suggest use `-it` so docker run generate progress bar of downloading information properly
* add info! log for model download
2023-08-02 06:30:35 +00:00
b8308b7118
refactor: extract TextGeneration trait ( #324 )
...
* add tabby-inference
* extract TextGeneration trait
* format
* Rename TextInferenceEngine to CTranslate2Engine
2023-08-02 06:12:51 +00:00
83e1cf76d8
feat: Upgrade ctranslate2 to v3.17.1 ( #323 )
2023-08-02 05:46:08 +00:00
95bd53ac9c
feat: add select kind param. Supported editors could log line select … ( #299 )
...
* feat: add select kind param. Supported editors could log line select or block select
* fix lint
2023-07-16 16:02:40 +08:00
Meng Zhang