* feat: add stop words encoding offset for ctranslate model config * feat: set default suffix to \n * add special treatment for bytefallback tokens |
||
|---|---|---|
| .. | ||
| ctranslate2.cc | ||
| dummy.cc | ||
| lib.rs | ||
* feat: add stop words encoding offset for ctranslate model config * feat: set default suffix to \n * add special treatment for bytefallback tokens |
||
|---|---|---|
| .. | ||
| ctranslate2.cc | ||
| dummy.cc | ||
| lib.rs | ||