diff --git a/website/blog/2023-09-13-release-0-1-0-metal/index.md b/website/blog/2023-09-13-release-0-1-0-metal/index.md new file mode 100644 index 0000000..22800fa --- /dev/null +++ b/website/blog/2023-09-13-release-0-1-0-metal/index.md @@ -0,0 +1,21 @@ +--- +authors: [ meng ] +--- +# Highlights of Tabby v0.1.0: Apple M1/M2 Support +We are thrilled to announce the release of Tabby v0.1.0πŸ‘πŸ». + +Thanks to [llama.cpp](https://github.com/ggerganov/llama.cpp), Apple M1/M2 Tabby users can now harness Metal inference support on Apple's M1 and M2 chips by using the `--device metal` flag. + +This enhancement leads to a significant inference speed upgradeπŸš€. It marks a meaningful milestone in Tabby's adoption on Apple devices. Check out our [Model Directory](/docs/models) to discover LLM models with Metal support! 🎁 + +
+ +![Inference](./inference.png) + +*An example inference benchmarking with [CodeLlama-7B](https://huggingface.co/TabbyML/CodeLlama-7B) on Apple M2 Max, takes ~600ms.* + +
+ +:::tip +Check out latest Tabby updates on [Linkedin](https://www.linkedin.com/company/tabbyml/) and [Slack community](https://join.slack.com/t/tabbycommunity/shared_invite/zt-1xeiddizp-bciR2RtFTaJ37RBxr8VxpA)! Our Tabby community is eager for your participation. ❀️ +::: diff --git a/website/blog/2023-09-13-release-0-1-0-metal/inference.png b/website/blog/2023-09-13-release-0-1-0-metal/inference.png new file mode 100644 index 0000000..ae9048c --- /dev/null +++ b/website/blog/2023-09-13-release-0-1-0-metal/inference.png @@ -0,0 +1,3 @@ +version https://git-lfs.github.com/spec/v1 +oid sha256:702e9b69b54a0b86731c23d199ffe454a2f03437b25f0fe8c25257e9c71b8877 +size 19495 diff --git a/website/docs/models/index.md b/website/docs/models/index.md index 4d2262b..757b8d9 100644 --- a/website/docs/models/index.md +++ b/website/docs/models/index.md @@ -8,9 +8,9 @@ We maintains a curated list of models varies from 200M to 10B+. | Model ID | License | Metal Support | | --------------------------------------------------------------------- | ------------------------------------------------------------------------------------------- | :-------------------------------------------------: | -| [TabbyML/CodeLlama-13B](https://huggingface.co/TabbyML/CodeLlama-13B) | [Llama2](https://github.com/facebookresearch/llama/blob/main/LICENSE) | ❌ | +| [TabbyML/CodeLlama-13B](https://huggingface.co/TabbyML/CodeLlama-13B) | [Llama2](https://github.com/facebookresearch/llama/blob/main/LICENSE) | βœ… | | [TabbyML/CodeLlama-7B](https://huggingface.co/TabbyML/CodeLlama-7B) | [Llama2](https://github.com/facebookresearch/llama/blob/main/LICENSE) | βœ… | -| [TabbyML/StarCoder-1B](https://huggingface.co/TabbyML/StarCoder-1B) | [BigCode-OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) | ❌ | +| [TabbyML/StarCoder-1B](https://huggingface.co/TabbyML/StarCoder-1B) | [BigCode-OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) | πŸ”œ | | [TabbyML/SantaCoder-1B](https://huggingface.co/TabbyML/SantaCoder-1B) | [BigCode-OpenRAIL-M](https://huggingface.co/spaces/bigcode/bigcode-model-license-agreement) | ❌ | | [TabbyML/J-350M](https://huggingface.co/TabbyML/J-350M) | [BSD-3](https://opensource.org/license/bsd-3-clause/) | ❌ | | [TabbyML/T5P-220M](https://huggingface.co/TabbyML/T5P-220M) | [BSD-3](https://opensource.org/license/bsd-3-clause/) | ❌ |