From 3d00cc5e87bb5b6b36a936638700c81962fc89eb Mon Sep 17 00:00:00 2001 From: Meng Zhang Date: Wed, 13 Sep 2023 17:19:28 +0800 Subject: [PATCH] docs: add v0.1.0 release blog (#440) * docs: add v0.1.0 release blog * remove unused files * add authors * Apply suggestions from code review Co-authored-by: Lucy Gao --------- Co-authored-by: Lucy Gao --- .../2023-09-13-release-0-1-0-metal/index.md | 21 +++++++++++++++++++ .../inference.png | 3 +++ website/docs/models/index.md | 4 ++-- 3 files changed, 26 insertions(+), 2 deletions(-) create mode 100644 website/blog/2023-09-13-release-0-1-0-metal/index.md create mode 100644 website/blog/2023-09-13-release-0-1-0-metal/inference.png diff --git a/website/blog/2023-09-13-release-0-1-0-metal/index.md b/website/blog/2023-09-13-release-0-1-0-metal/index.md new file mode 100644 index 0000000..22800fa --- /dev/null +++ b/website/blog/2023-09-13-release-0-1-0-metal/index.md @@ -0,0 +1,21 @@ +--- +authors: [ meng ] +--- +# Highlights of Tabby v0.1.0: Apple M1/M2 Support +We are thrilled to announce the release of Tabby v0.1.0πŸ‘πŸ». + +Thanks to [llama.cpp](https://github.com/ggerganov/llama.cpp), Apple M1/M2 Tabby users can now harness Metal inference support on Apple's M1 and M2 chips by using the `--device metal` flag. + +This enhancement leads to a significant inference speed upgradeπŸš€. It marks a meaningful milestone in Tabby's adoption on Apple devices. Check out our [Model Directory](/docs/models) to discover LLM models with Metal support! 🎁 + +
+ +![Inference](./inference.png) + +*An example inference benchmarking with [CodeLlama-7B](https://huggingface.co/TabbyML/CodeLlama-7B) on Apple M2 Max, takes ~600ms.* + +
+ +:::tip +Check out latest Tabby updates on [Linkedin](https://www.linkedin.com/company/tabbyml/) and [Slack community](https://join.slack.com/t/tabbycommunity/shared_invite/zt-1xeiddizp-bciR2RtFTaJ37RBxr8VxpA)! Our Tabby community is eager for your participation. ❀️ +::: diff --git a/website/blog/2023-09-13-release-0-1-0-metal/inference.png b/website/blog/2023-09-13-release-0-1-0-metal/inference.png new file mode 100644 index 0000000..ae9048c --- /dev/null +++ b/website/blog/2023-09-13-release-0-1-0-metal/inference.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:702e9b69b54a0b86731c23d199ffe454a2f03437b25f0fe8c25257e9c71b8877 +size 19495 diff --git a/website/docs/models/index.md b/website/docs/models/index.md index 4d2262b..757b8d9 100644 --- a/website/docs/models/index.md +++ b/website/docs/models/index.md @@ -8,9 +8,9 @@ We maintains a curated list of models varies from 200M to 10B+. | Model ID | License | Metal Support | | --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | :-------------------------------------------------: | -| [TabbyML/CodeLlama-13B](https://huggingface.co/TabbyML/CodeLlama-13B) | [Llama2](https://github.com/facebookresearch/llama/blob/main/LICENSE) | ❌ | +| [TabbyML/CodeLlama-13B](https://huggingface.co/TabbyML/CodeLlama-13B) | [Llama2](https://github.com/facebookresearch/llama/blob/main/LICENSE) | βœ… | | [TabbyML/CodeLlama-7B](https://huggingface.co/TabbyML/CodeLlama-7B) | [Llama2](https://github.com/facebookresearch/llama/blob/main/LICENSE) | βœ… | -| [TabbyML/StarCoder-1B](https://huggingface.co/TabbyML/StarCoder-1B) | [BigCode-OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) | ❌ | +| [TabbyML/StarCoder-1B](https://huggingface.co/TabbyML/StarCoder-1B) | [BigCode-OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) | πŸ”œ | | [TabbyML/SantaCoder-1B](https://huggingface.co/TabbyML/SantaCoder-1B) | [BigCode-OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) | ❌ | | [TabbyML/J-350M](https://huggingface.co/TabbyML/J-350M) | [BSD-3](https://opensource.org/license/bsd-3-clause/) | ❌ | | [TabbyML/T5P-220M](https://huggingface.co/TabbyML/T5P-220M) | [BSD-3](https://opensource.org/license/bsd-3-clause/) | ❌ |