* feat: add stop words encoding offset for ctranslate model config * feat: set default suffix to \n * add special treatment for bytefallback tokens |
||
|---|---|---|
| .. | ||
| CTranslate2@4978339f30 | ||
| cmake | ||
| include | ||
| src | ||
| .gitignore | ||
| CMakeLists.txt | ||
| Cargo.toml | ||
| build.rs | ||
* feat: add stop words encoding offset for ctranslate model config * feat: set default suffix to \n * add special treatment for bytefallback tokens |
||
|---|---|---|
| .. | ||
| CTranslate2@4978339f30 | ||
| cmake | ||
| include | ||
| src | ||
| .gitignore | ||
| CMakeLists.txt | ||
| Cargo.toml | ||
| build.rs | ||