docs: add faq on utilize multipl NV GPUs (#423)

* docs: add faq on utilize multipl NV GPUs

* update
release-0.2
Meng Zhang 2023-09-10 22:16:46 +08:00 committed by GitHub
parent 12a37e2330
commit 491e295a48
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 13 additions and 0 deletions

View File

@ -1,3 +1,5 @@
import CodeBlock from '@theme/CodeBlock';
# ⁉️ Frequently Asked Questions
<details>
@ -18,3 +20,14 @@
</p>
</div>
</details>
<details>
<summary>How to utilize multiple NVIDIA GPUs?</summary>
<div>
<p>Tabby supports replicating models on multiple GPUs to increase throughput. You can specify the devices for model replication by using the <b>--device-indices</b> option.</p>
<CodeBlock language="bash">
# Replicate model to GPU 0 and GPU 1.{'\n'}
tabby serve ... --device-indices 0 --device-indices 1
</CodeBlock>
</div>
</details>