docs: improve

improve-workflow
Meng Zhang 2023-06-06 19:53:40 -07:00
parent 09cd8a38ed
commit ffe777e519
3 changed files with 80 additions and 59 deletions

View File

@ -1,59 +0,0 @@
# Docker
There is a supplied docker image to make deploying a server as a container easier.
## CPU
**Command line**
```bash
docker run \
-p 8080:8080 -v $HOME/.tabby:/data \
tabbyml/tabby serve --model TabbyML/SantaCoder-1B
```
**Docker Compose**
```yaml
version: '3.5'
services:
tabby:
restart: always
image: tabbyml/tabby
command: serve --model TabbyML/SantaCoder-1B
volumes:
- "$HOME/.tabby:/data"
ports:
- 8080:8080
```
## CUDA (requires NVIDIA Container Toolkit)
**Command line**
```bash
docker run \
--gpus all -p 8080:8080 -v $HOME/.tabby:/data \
tabbyml/tabby \
serve --model TabbyML/SantaCoder-1B --device cuda
```
**Docker Compose**
```yaml
version: '3.5'
services:
tabby:
restart: always
image: tabbyml/tabby
command: serve --model TabbyML/SantaCoder-1B --device cuda
volumes:
- "$HOME/.tabby:/data"
ports:
- 8080:8080
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
```

View File

@ -0,0 +1,76 @@
# Docker
There is a supplied docker image to make deploying a server as a container easier.
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
## CPU
<Tabs>
<TabItem value="shell" label="Shell" default>
```bash title="run.sh"
docker run \
-p 8080:8080 -v $HOME/.tabby:/data \
tabbyml/tabby serve --model TabbyML/SantaCoder-1B
```
</TabItem>
<TabItem value="compose" label="Docker Compose">
```yaml title="docker-compose.yml"
version: '3.5'
services:
tabby:
restart: always
image: tabbyml/tabby
command: serve --model TabbyML/SantaCoder-1B
volumes:
- "$HOME/.tabby:/data"
ports:
- 8080:8080
```
</TabItem>
</Tabs>
## CUDA (requires NVIDIA Container Toolkit)
<Tabs>
<TabItem value="shell" label="Shell" default>
```bash title="run.sh"
docker run \
--gpus all -p 8080:8080 -v $HOME/.tabby:/data \
tabbyml/tabby \
serve --model TabbyML/SantaCoder-1B --device cuda
```
</TabItem>
<TabItem value="compose" label="Docker Compose">
```yaml title="docker-compose.yml"
version: '3.5'
services:
tabby:
restart: always
image: tabbyml/tabby
command: serve --model TabbyML/SantaCoder-1B --device cuda
volumes:
- "$HOME/.tabby:/data"
ports:
- 8080:8080
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
```
</TabItem>
</Tabs>

View File

@ -7,3 +7,7 @@ Thanks to Apple's Accelerate and CoreML frameworks, we can now run Tabby on edge
2. Run `tabby --help` to verify successful installation. 2. Run `tabby --help` to verify successful installation.
3. Start the server with `tabby serve model --models TabbyML/T5P-220M`. 3. Start the server with `tabby serve model --models TabbyML/T5P-220M`.
:::tip
The compute power of M1/M2 is limited and is likely to be sufficient only for individual usage. If you require a shared instance for a team, we recommend considering Docker hosting with CUDA. You can find more information about Docker [here](./docker).