root
/
tabby
mirror of https://gitee.com/zhang_1334717033/tabby.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
tabby/development
History
Meng Zhang d5d58fbbec
Improve documentations. (#20) 
* Improve help message of model preload

* Update development/scripts/triton.sh

* Improve documents

* Update deployment.md

* Update deployment.md
 		2023-03-27 11:46:18 +08:00
..
config Add gptj converter (#19) 2023-03-27 11:12:52 +08:00
scripts Improve documentations. (#20) 2023-03-27 11:46:18 +08:00
.gitignore Add gptj converter (#19) 2023-03-27 11:12:52 +08:00
Makefile Add gptj converter (#19) 2023-03-27 11:12:52 +08:00
README.md Add gptj converter (#19) 2023-03-27 11:12:52 +08:00
docker-compose.dev.yml Add gptj converter (#19) 2023-03-27 11:12:52 +08:00
docker-compose.triton.yml Add gptj converter (#19) 2023-03-27 11:12:52 +08:00
docker-compose.yml Add gptj converter (#19) 2023-03-27 11:12:52 +08:00

README.md

Deployment

  1. Start service 
    docker-compose up
  2. Test API endpoint with curl 
    curl -X POST http://localhost:5000/v1/completions -H 'Content-Type: application/json' --data '{
    "prompt": "def binarySearch(arr, left, right, x):\n    mid = (left +"
}'