root
/
tabby
mirror of https://gitee.com/zhang_1334717033/tabby.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

docs: Update faq.mdx on multiple GPU utilization

release-notes-05
Meng Zhang 2023-10-29 01:22:13 -07:00 committed by GitHub
parent 7bd99d14c0
commit 68a7b3628c
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 2 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

6
website/docs/faq.mdx
View File

@ -24,11 +24,7 @@ import CodeBlock from '@theme/CodeBlock';
<details> <details>
<summary>How to utilize multiple NVIDIA GPUs?</summary> <summary>How to utilize multiple NVIDIA GPUs?</summary>
<div> <div>
<p>Tabby supports replicating models on multiple GPUs to increase throughput. You can specify the devices for model replication by using the <b>--device-indices</b> option.</p> <p>Tabby only supports the use of a single GPU. To utilize multiple GPUs, you can initiate multiple Tabby instances and set CUDA_VISIBLE_DEVICES accordingly.</p>
<CodeBlock language="bash">
# Replicate model to GPU 0 and GPU 1.{'\n'}
tabby serve ... --device-indices 0 --device-indices 1
</CodeBlock>
</div> </div>
</details> </details>