Meng Zhang 296342efd8 refactor: use llama.cpp tokenizer (#683) ... * refactor: switch to llama.cpp tokenizer to simplify implementation * refactor: remove tokenizer dependency from tabby * refactor: renaming decoding to stop condition * refactor: remove tokenizer dependency * refactor: remove submodule * chore: update formatting * move tokenization to c++		2023-10-31 22:16:09 +00:00
..
engine.cc	refactor: use llama.cpp tokenizer (#683)	2023-10-31 22:16:09 +00:00
lib.rs	refactor: use llama.cpp tokenizer (#683)	2023-10-31 22:16:09 +00:00