root
/
tabby
mirror of https://gitee.com/zhang_1334717033/tabby.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

docs: add instruction on adding new programming language support. (#556) 

* temp commit

* docs: add instruction on adding new programming language support.
dedup-snippet-at-index
Meng Zhang 2023-10-14 00:37:05 -07:00 committed by GitHub
parent 8390fff423
commit d22f51cc16
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
2 changed files with 48 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

4
website/docs/configuration.md
View File

@ -1,3 +1,7 @@
---
sidebar_position: 5
---
# ⚙️ Configuration
Tabby server will look for a configuration file at `~/.tabby/config.toml` for advanced features.

44
website/docs/programming-languages.md Normal file
View File

@ -0,0 +1,44 @@
---
sidebar_position: 6
---
# 🧑‍💻 Programming Languages
Most models nowadays support a large number of programming languages (thanks to [The Stack](https://huggingface.co/datasets/bigcode/the-stack), which has collected 358 programming languages).
In Tabby, we need to add configuration for each language to maximize performance and completion quality.
Currently, there are two aspects of support that need to be added for each language.
**Stop Words**
Stop words determine when the language model can early stop its decoding steps, resulting in better latency and affecting the quality of completion. We suggest adding all top-level keywords as part of the stop words.
**Repository Context**
We parse languages into chunks and compute a token-based index for serving time Retrieval Augmented Code Completion. In Tabby, we define these repository contexts as [treesitter queries](https://tree-sitter.github.io/tree-sitter/using-parsers#query-syntax), and the query results will be indexed.
For an actual example of an issue or pull request adding the above support, please check out https://github.com/TabbyML/tabby/issues/553 as a reference.
## Supported Languages
* [Rust](https://www.rust-lang.org/)
* [Python](https://www.python.org/)
* [JavaScript](https://developer.mozilla.org/en-US/docs/Web/JavaScript)
* [TypeScript](https://www.typescriptlang.org/)
## Languages Missing Certain Support
| Language | Stop Words (time to contribute: <5 min) | Repository Context (time to contribute: <1 hr) |
| :------: | :-------------------------------------: | :--------------------------------------------: |
| C/C++ | 🚫 | 🚫 |
| C# | 🚫 | 🚫 |
| CSS | 🚫 | 🚫 |
| Go | 🚫 | 🚫 |
| Haskell | 🚫 | 🚫 |
| Java | 🚫 | 🚫 |
| Julia | 🚫 | 🚫 |
| Lua | 🚫 | 🚫 |
| PHP | 🚫 | 🚫 |
| Perl | 🚫 | 🚫 |
| Ruby | 🚫 | 🚫 |
| Scala | 🚫 | 🚫 |