From 1cc1cba56c7c629f15c0b780c3204f4ef8deb76c Mon Sep 17 00:00:00 2001 From: Lucy Gao Date: Thu, 21 Sep 2023 20:54:20 +0800 Subject: [PATCH] docs: add wizardcoder models (#467) * Update README.md test * Update on 10k stars. * Update README.md on 10k stars. * doc: add wizardcoder models --- website/docs/models/index.md | 32 +++++++++++++++++++------------- 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/website/docs/models/index.md b/website/docs/models/index.md index be6e287..1d73800 100644 --- a/website/docs/models/index.md +++ b/website/docs/models/index.md @@ -5,20 +5,22 @@ sidebar_position: 4 # 🧑‍🔬 Models Directory We recommend using -* small models (less than 400M) for CPU devices. -* For 1B to 7B models, it's advisable to have at least NVIDIA T4, 10 Series, or 20 Series GPUs. -* For 7B to 13B models, we recommend using NVIDIA V100, A100, 30 Series, or 40 Series GPUs. +* **small models (less than 400M)** for **CPU devices**. +* For **1B to 7B models**, it's advisable to have at least **NVIDIA T4, 10 Series, or 20 Series GPUs**. +* For **7B to 13B models**, we recommend using **NVIDIA V100, A100, 30 Series, or 40 Series GPUs**. -| Model ID | License | -| --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | -| [TabbyML/CodeLlama-13B](https://huggingface.co/TabbyML/CodeLlama-13B) | [Llama2](https://github.com/facebookresearch/llama/blob/main/LICENSE) | -| [TabbyML/CodeLlama-7B](https://huggingface.co/TabbyML/CodeLlama-7B) | [Llama2](https://github.com/facebookresearch/llama/blob/main/LICENSE) | -| [TabbyML/StarCoder-7B](https://huggingface.co/TabbyML/StarCoder-7B) | [BigCode-OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) | -| [TabbyML/StarCoder-3B](https://huggingface.co/TabbyML/StarCoder-3B) | [BigCode-OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) | -| [TabbyML/StarCoder-1B](https://huggingface.co/TabbyML/StarCoder-1B) | [BigCode-OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) | -| [TabbyML/SantaCoder-1B](https://huggingface.co/TabbyML/SantaCoder-1B) | [BigCode-OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) | -| [TabbyML/J-350M](https://huggingface.co/TabbyML/J-350M) | [BSD-3](https://opensource.org/license/bsd-3-clause/) | -| [TabbyML/T5P-220M](https://huggingface.co/TabbyML/T5P-220M) | [BSD-3](https://opensource.org/license/bsd-3-clause/) | +| Model ID | License | +| ---------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | +| [TabbyML/CodeLlama-13B](https://huggingface.co/TabbyML/CodeLlama-13B) | [Llama2](https://github.com/facebookresearch/llama/blob/main/LICENSE) | +| [TabbyML/CodeLlama-7B](https://huggingface.co/TabbyML/CodeLlama-7B) | [Llama2](https://github.com/facebookresearch/llama/blob/main/LICENSE) | +| [TabbyML/StarCoder-7B](https://huggingface.co/TabbyML/StarCoder-7B) | [BigCode-OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) | +| [TabbyML/StarCoder-3B](https://huggingface.co/TabbyML/StarCoder-3B) | [BigCode-OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) | +| [TabbyML/StarCoder-1B](https://huggingface.co/TabbyML/StarCoder-1B) | [BigCode-OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) | +| [TabbyML/SantaCoder-1B](https://huggingface.co/TabbyML/SantaCoder-1B) | [BigCode-OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) | +| [TabbyML/WizardCoder-3B](https://huggingface.co/TabbyML/WizardCoder-3B)| [OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) | +| [TabbyML/WizardCoder-1B](https://huggingface.co/TabbyML/WizardCoder-1B)| [OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) | +| [TabbyML/J-350M](https://huggingface.co/TabbyML/J-350M) | [BSD-3](https://opensource.org/license/bsd-3-clause/) | +| [TabbyML/T5P-220M](https://huggingface.co/TabbyML/T5P-220M) | [BSD-3](https://opensource.org/license/bsd-3-clause/) | ### CodeLlama-7B / CodeLlama-13B @@ -28,6 +30,10 @@ Code Llama is a collection of pretrained and fine-tuned generative text models. StarCoder series model are trained on 80+ programming languages from The Stack (v1.2), with opt-out requests excluded. The model uses Multi Query Attention, a context window of 8192 tokens, and was trained using the Fill-in-the-Middle objective on 1 trillion tokens. +### WizardCoder-1B / WizardCoder-3B + +WizardCoder [(arXiv)](https://arxiv.org/abs/2306.08568) series model are finetuned on StarCoder models with the Evol-Instruct method to adapt to coding tasks. Note that WizardCoder models have used GPT-4 generated data for finetuning, and thus adhere to [OpenAI's limitations](https://openai.com/policies/terms-of-use) for model usage. + ### SantaCoder-1B SantaCoder is the smallest member of the BigCode family of models, boasting just 1.1 billion parameters. This model is specifically trained with a fill-in-the-middle objective, enabling it to efficiently auto-complete function parameters. It offers support for three programming languages: Python, Java, and JavaScript.