tabby/website/docs/programming-languages.md

47 lines
2.8 KiB
Markdown
Raw Normal View History

---
sidebar_position: 6
---
# 🧑‍💻 Programming Languages
Most models nowadays support a large number of programming languages (thanks to [The Stack](https://huggingface.co/datasets/bigcode/the-stack), which has collected 358 programming languages).
In Tabby, we need to add configuration for each language to maximize performance and completion quality.
Currently, there are two aspects of support that need to be added for each language.
**Stop Words**
Stop words determine when the language model can early stop its decoding steps, resulting in better latency and affecting the quality of completion. We suggest adding all top-level keywords as part of the stop words.
**Repository Context**
We parse languages into chunks and compute a token-based index for serving time Retrieval Augmented Code Completion. In Tabby, we define these repository contexts as [treesitter queries](https://tree-sitter.github.io/tree-sitter/using-parsers#query-syntax), and the query results will be indexed.
For an actual example of an issue or pull request adding the above support, please check out https://github.com/TabbyML/tabby/issues/553 as a reference.
## Supported Languages
* [Rust](https://www.rust-lang.org/)
* [Python](https://www.python.org/)
* [JavaScript](https://developer.mozilla.org/en-US/docs/Web/JavaScript)
* [TypeScript](https://www.typescriptlang.org/)
* [Golang](https://go.dev/)
* [Ruby](https://www.ruby-lang.org/)
* [Java](https://www.java.com/)
* [Kotlin](https://www.kotlinlang.org/)
## Languages Missing Certain Support
| Language | Stop Words (time to contribute: <5 min) | Repository Context (time to contribute: <1 hr) |
| :------: | :-------------------------------------: | :--------------------------------------------: |
| C/C++ | 🚫 | 🚫 |
| C# | 🚫 | 🚫 |
| CSS | 🚫 | 🚫 |
| Haskell | 🚫 | 🚫 |
| Julia | 🚫 | 🚫 |
| Lua | 🚫 | 🚫 |
| PHP | 🚫 | 🚫 |
| Perl | 🚫 | 🚫 |
| Scala | 🚫 | 🚫 |
feat: add support for Kotlin language (#813) * feat: add support for Kotlin * Update programming-languages.md * Update Cargo.toml * fix: kotlin missing TAGGING_QUERY * docs: remove 0.5.0 warning as tabby adapt same format for remote / local storage * feat: make --model optional, so user can start a chat only instance (#817) * fix: mark tabby-ui as yarn managed * fix: rust build caused by 1.74.0 release (#819) * chore: exclude experimental directory for language stats * chore: exclude ee/tabby-webserver/ui to language stats * fix: move db path, fix compile error (#820) * feat(vscode): Add dont-show-again for warnings. Add online help links. (#821) * refactor: delete contrib/jetson/Dockerfile as it's no longer compatible post 0.5 feel free to sending pr of adding it back if you get it work with llama.cpp based docker image * feat(vim): update tabby-agent. (#824) * feat(intellij): Add dont-show-again for warnings. Add more online help links. (#823) * feat: implement basic dependency detection for python / rust (#825) * refactor: extract tags mod for dataset.rs * feat: implement basic dependency detection for python / rust * fix: keep only direct dependency, clean up path (#827) * feat: add support for Kotlin * Update Cargo.toml * fix: adjustment post rebase * fix: added kotlin.scm to cover query tags * fix: updated kotlin.scm * remove definition import Co-authored-by: Meng Zhang <meng@tabbyml.com> * remove typealias Co-authored-by: Meng Zhang <meng@tabbyml.com> --------- Co-authored-by: Meng Zhang <meng@tabbyml.com> Co-authored-by: Eric <illuminating.me@gmail.com> Co-authored-by: Zhiming Ma <codes.icy@gmail.com>
2023-11-23 01:31:23 +00:00
| Kotlin | 🚫 | 🚫 |