fd1baff8d5
* refactor: pass step and string token to callback * add token to callback * add stop regexp * implement stop words logic * pass token_ids from inference * improve effiency of regexp match with reversed regex * fmt * add typescript and javascript stop words * add cache for stop words regexp |
||
|---|---|---|
| .. | ||
| CTranslate2@d4b6f3849a | ||
| cmake | ||
| include | ||
| src | ||
| .gitignore | ||
| CMakeLists.txt | ||
| Cargo.toml | ||
| build.rs | ||