root
/
tabby
mirror of https://gitee.com/zhang_1334717033/tabby.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
tabby/deployment
History
Meng Zhang a0b438da06
Add python transformer backend for tabby (mainly used for local dev / test in non-cuda environment) (#6) 
* Add python backend

* Split docker-compose.triton.yml

* update makefile
 		2023-03-23 14:14:33 +08:00
..
config Create config dir for config files in deployment 2023-03-22 23:58:55 +08:00
.gitignore Add python transformer backend for tabby (mainly used for local dev / test in non-cuda environment) (#6) 2023-03-23 14:14:33 +08:00
Dockerfile Add admin panel (w/ streamlit) and logging (w/ vectordev) (#4) 2023-03-22 23:18:12 +08:00
README.md Create README.md 2023-03-22 23:57:26 +08:00
docker-compose.dev.yml Add admin panel (w/ streamlit) and logging (w/ vectordev) (#4) 2023-03-22 23:18:12 +08:00
docker-compose.triton.yml Add python transformer backend for tabby (mainly used for local dev / test in non-cuda environment) (#6) 2023-03-23 14:14:33 +08:00
docker-compose.yml Add python transformer backend for tabby (mainly used for local dev / test in non-cuda environment) (#6) 2023-03-23 14:14:33 +08:00

README.md

Deployment

  1. Start service 
    docker-compose up
  2. Test API endpoint with curl 
    curl -X POST http://localhost:5000/v1/completions -H 'Content-Type: application/json' --data '{
    "prompt": "def binarySearch(arr, left, right, x):\n    mid = (left +"
}'