e93a971d0e
feat: tune llama metal backend performance ( #393 )
...
* feat: support eos based stop
* feat: print performance stats after each inference
* update llama.cpp
* update commits
2023-09-05 10:14:29 +08:00
2472cf3b55
test: use function call style snippet for prompt builder unit test ( #395 )
...
* test: better tests for build_prefix()
* chore
* chore: resolve comments
2023-09-04 04:54:18 +00:00
74073aa77a
test: add build prefix test and debug chars counting [TAB-184] ( #394 )
...
* test: add count char test
* chore: fix lint
* chore
* chore
2023-09-03 20:57:26 +08:00
3c7c8d9293
feat: add cargo test to github actions and run only unit tests in ci [TAB-185] ( #390 )
...
* feat: add cargo test to github actions
* chore: fix lint
* chore: add openblas dependency
* chore: update build dependency
* chore: resolve comments
* chore: fix lint
* chore: fix lint
* chore: test installing dependencies
* chore: refactor integ test
* update ci
* cleanup
---------
Co-authored-by: Meng Zhang <meng@tabbyml.com>
2023-09-03 05:04:52 +00:00
c8339a2912
refactor: use TabbyML/llama.cpp submodule
2023-09-03 12:38:54 +08:00
3acd5d9bc4
refactor: remove llama.cpp subtree
2023-09-03 12:37:26 +08:00
92c8ae8ee7
feat: embed ggml-metal.metal
2023-09-03 10:41:03 +08:00
ed6c5b2e60
Merge commit 'aad80a58b07836bfbf6aedd50993bc54b4257388' as 'crates/llama-cpp-bindings/llama.cpp'
2023-09-03 10:07:10 +08:00
d4137463ef
remove llama.cpp submodule
2023-09-03 10:04:26 +08:00
e360b438b4
fix lint
2023-09-03 10:01:28 +08:00
3f7aa99b0d
feat: support cancellation in llama backend
2023-09-03 09:59:40 +08:00
3573d4378e
feat: llama.cpp for metal support [TAB-146] ( #391 )
...
* feat: init commit adding llama-cpp-bindings
* add llama.cpp submodule
* add LlamaEngine to hold llama context / llama model
* add cxxbridge
* add basic greedy sampling
* move files
* make compile success
* connect TextGeneration with LlamaEngine
* experimental support llama.cpp
* add metal device
* add Accelerate
* fix namespace for llama-cpp-bindings
* fix lint
* move stepping logic to rust
* add stop words package
* use stop-words in ctranslate2-bindings
* use raw string for regex
* use Arc<Tokenizer> for sharing tokenizers
* refactor: remove useless stop_words_encoding_offset
* switch to tokenizers 0.13.4-rc.3
* fix lints in cpp
* simplify implementation of greedy decoding
* feat: split metal feature for llama backend
* add ci
* update ci
* build tabby bin in ci build
2023-09-03 09:59:07 +08:00
5dff349801
add single line comment to languages so they can be used in prompting [TAB-181] ( #388 )
...
* chore: add comment signs to extended languages
* Update crates/tabby/src/serve/completions/prompt.rs
---------
Co-authored-by: Meng Zhang <meng@tabbyml.com>
2023-09-01 03:43:27 +00:00
63c00494f3
test: unit tests to prompt builder [TAB-180] ( #387 )
...
* test: unit tests to prompt builder
* chore: fix typo
* chore: fix lint
* chore: resolve comments
2023-09-01 09:20:20 +08:00
90aadad3ce
feat: map js,ts,jsx and tsx to js-ts as unified language [TAB-181] ( #386 )
...
* feat: reduce js, ts, jsx and tsx to js-ts
* chore: refactor and add language reducing to both indexing and dataset jobs
* chore: only reduce language in dataset job
* chore: only reduce language in index job
* chore: fix lint
* chore: resolve comments
2023-08-31 17:21:39 +00:00
e5598e63f2
feat: extend language [TAB-181] ( #385 )
...
* feat: extend indexer's language support
* feat: extend language support
* chore: add support for mjs and mts
* chore: fix lint
2023-08-31 07:36:57 +00:00
c44a9c7195
fix: correct git_describe in /health ( #383 )
...
* fix: add missing Version component in OpenAPI definition
* fix: allow tag / dirty in git describe
2023-08-31 01:06:36 +00:00
57baecb370
fix: switch default running backend to openblas on x86 linux ( #380 )
2023-08-30 14:19:35 +00:00
054aefaf15
chore: add linux static build ( #379 )
...
* chore: add linux static build
* add touch
* update build env
* add sudo
* fix: protobuf ubuntu target
2023-08-30 18:45:05 +08:00
3526ca3164
chore: build with ruy (cpu only) on static mode for linux. ( #378 )
...
* chore: build with ruy (cpu only) on static mode for linux.
* update cmake min version
2023-08-30 18:04:40 +08:00
fc9a623e72
feat: add logging on server starting ( #372 )
2023-08-28 06:12:00 +00:00
65836ee199
feat: add stop words encoding offset for ctranslate model config ( #371 )
...
* feat: add stop words encoding offset for ctranslate model config
* feat: set default suffix to \n
* add special treatment for bytefallback tokens
2023-08-28 14:07:01 +08:00
2a91a21787
feat: add gpu info to health state [TAB-162] ( #364 )
...
* feat: add gpu info to health response
* chore: error handling
* chore: refactor cpu manager code
* chore: typo
* chore: fix context mutability
* chore: fix context mutability
* feat: add link to NVML lib
* chore: refactor
* lint
* chore: resolve comments
* chore: fix typo
* chore: fix
* chore: resolve comments
* chore: fix
* chore: resolve comments
2023-08-21 18:06:38 +08:00
b1ad936033
feat: add version information in health state. ( #363 )
...
* feat: add git_hash in health state
* add more version information in health state
2023-08-20 15:21:12 +00:00
df45573501
feat: reduce ServeHealth event to every 300s to reduce event volume ( #362 )
2023-08-20 12:36:59 +00:00
2026b4dd0e
feat: add architecture/cpu info to health api response [TAB-162] ( #355 )
...
* feat: add architecture, cpu and gpu info to health command
* chore: fix
* chore: fix
* chore: fix
* chore: fix lint
* chore: fix lint
* chore: remove gpu
* chore: resolve comments
* chore: resolve comments
* Update health.rs
---------
Co-authored-by: Meng Zhang <meng@tabbyml.com>
2023-08-15 15:22:03 +00:00
dbc89831b1
feat: add serve health heartbeat ( #343 )
...
* add serve health tracking
* fix lint
* fix
2023-08-09 08:08:42 +00:00
d0f6ad2d2a
feat: add anonymous usage tracker ( #342 )
...
* feat: add anonymous usage tracker
* improve deps
* update
* update
2023-08-09 07:31:13 +00:00
220fcc0d65
fix: make `config.experimental` optional ( #339 )
...
* fix: make config.experimental` optional
* add unit test for empty toml config
2023-08-07 09:53:00 +00:00
4eaae27ed3
Update Cargo.toml ( #331 )
2023-08-03 19:55:00 +08:00
6a50902ca7
fix: support ctranslate2 rev7 vocab files (.json) ( #327 )
2023-08-02 13:36:31 +00:00
57c811b30f
fix: improve download logging ( #325 )
...
* Suggest use `-it` so docker run generate progress bar of downloading information properly
* add info! log for model download
2023-08-02 06:30:35 +00:00
b8308b7118
refactor: extract TextGeneration trait ( #324 )
...
* add tabby-inference
* extract TextGeneration trait
* format
* Rename TextInferenceEngine to CTranslate2Engine
2023-08-02 06:12:51 +00:00
83e1cf76d8
feat: Upgrade ctranslate2 to v3.17.1 ( #323 )
2023-08-02 05:46:08 +00:00
95bd53ac9c
feat: add select kind param. Supported editors could log line select … ( #299 )
...
* feat: add select kind param. Supported editors could log line select or block select
* fix lint
2023-07-16 16:02:40 +08:00
be5fe0d737
feat: add rust prompt rewrite support ( #296 )
2023-07-13 09:31:44 +00:00
4388fd0050
feat: support prompt rewriting ( #295 )
...
* refactor: extract PromptBuilder
* feat: load tantivy index in prompt builder
* integrate with searcher
* add enable_prompt_rewrite to control rewrite behavior
* nit docs
* limit 1 snippet per identifier
* extract magic numbers
2023-07-13 09:05:41 +00:00
19586a4926
fix: fix bad indexing field of tag indexer ( #293 )
2023-07-13 07:19:46 +00:00
24d17dc1ed
feat: switching index to tag based ( #265 )
...
* add tag based index
* update for python
* fix lint
2023-07-13 06:25:39 +00:00
be90047477
fix: fix int8 compute type, fix auto compute type selection (include float32 into consideration for cuda compute capability <= 6.0) ( #291 )
2023-07-12 11:09:38 +08:00
e822d1857f
fix: add retry strategy in tabby download ( #285 )
2023-07-09 05:18:19 +00:00
9ca1f7e5f1
fix: add additional whitespace to match tokens that combining space and li… ( #270 )
...
* fix: add additional whitespace to match tokens that combining space and line break
* fix lint
2023-06-25 01:15:52 +00:00
631cff3aed
docs: update url of playground server
2023-06-23 18:55:23 -07:00
8879ed7655
feat: add experimental self host setup ( #261 )
2023-06-23 12:13:22 -07:00
59b03ae477
fix: use full clone in scheduler
2023-06-22 15:06:44 -07:00
fcbc5edc55
Revert "feat: add /experimental/search endpoint ( #258 )" ( #260 )
...
This reverts commit 04980160e5
2023-06-22 14:23:35 -07:00
04980160e5
feat: add /experimental/search endpoint ( #258 )
...
* feat: add /experimental/search endpoint
* fix format
2023-06-22 20:47:32 +00:00
8c1f6c2806
feat(scheduler): add java / python tags support ( #259 )
2023-06-22 20:25:56 +00:00
7ed5dd584d
feat: experiment ctags support in scheduler ( #207 )
...
* experiment ctags support
* add document.rs
* extract Document to common
* integrate tags into dataset builder
* skip if none
* do not add scheduler in client binary
* fix fmt
2023-06-21 19:48:13 -07:00
6eae16d475
fix: typo in openapi documentation
2023-06-16 20:22:38 -07:00
Meng Zhang