From aa7ed053ecba01459a558840585c91f07e11b00c Mon Sep 17 00:00:00 2001 From: Meng Zhang Date: Thu, 6 Apr 2023 00:25:16 +0800 Subject: [PATCH] Update README.md --- README.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/README.md b/README.md index 027d3d0..be94d47 100644 --- a/README.md +++ b/README.md @@ -42,13 +42,6 @@ docker run \ tabbyml/tabby ``` -You can then query the server using `/v1/completions` endpoint: -```bash -curl -X POST http://localhost:5000/v1/completions -H 'Content-Type: application/json' --data '{ - "prompt": "def binarySearch(arr, left, right, x):\n mid = (left +" -}' -``` - To use the GPU backend (triton) for a faster inference speed: ```bash docker run \ @@ -65,6 +58,13 @@ docker run \ ``` Note: To use GPUs, you need to install the [NVIDIA Container Toolkit](https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/install-guide.html). We also recommend using NVIDIA drivers with CUDA version 11.8 or higher. +You can then query the server using `/v1/completions` endpoint: +```bash +curl -X POST http://localhost:5000/v1/completions -H 'Content-Type: application/json' --data '{ + "prompt": "def binarySearch(arr, left, right, x):\n mid = (left +" +}' +``` + We also provides an interactive playground in admin panel [localhost:8501](http://localhost:8501) ![image](https://user-images.githubusercontent.com/388154/227792390-ec19e9b9-ebbb-4a94-99ca-8a142ffb5e46.png)