Meng Zhang 8ab35b2639 feat: add --parallelism to control throughput and vram usage (#727) ... * feat: add --parallelism to control throughput and vram usage * update default * Revert "update default" This reverts commit 349792c0d48d913dcd8be4ce1c9d7ce887918f29. * cargo fmt		2023-11-08 18:31:22 +00:00
..
include	feat: add --parallelism to control throughput and vram usage (#727)	2023-11-08 18:31:22 +00:00
llama.cpp@75fb6f2ba0	fix: support cpu only run in llama.cpp cuda build	2023-11-06 22:59:24 -08:00
src	feat: add --parallelism to control throughput and vram usage (#727)	2023-11-08 18:31:22 +00:00
Cargo.toml	fix: when there's an error happens in background inference loop, it should exit the process (#713)	2023-11-06 20:41:49 +00:00
build.rs	fix: fix docker build	2023-10-27 21:25:45 -07:00