tabby

History

Meng Zhang 92eb2d54f5 Add LoRA Fine-tuning for private code repository (#22 ) * Add bitandsands * Fix cudart in Dockerfile * Add ConstantLengthDataset in trainer * Add train_lora * Remove bnb * Remove useless imports		2023-03-28 15:57:13 +08:00
..
config	Add gptj converter (#19 )	2023-03-27 11:12:52 +08:00
scripts	Improve documentations. (#20 )	2023-03-27 11:46:18 +08:00
.gitignore	Add gptj converter (#19 )	2023-03-27 11:12:52 +08:00
Makefile	Add LoRA Fine-tuning for private code repository (#22 )	2023-03-28 15:57:13 +08:00
README.md	Add gptj converter (#19 )	2023-03-27 11:12:52 +08:00
docker-compose.dev.yml	Add gptj converter (#19 )	2023-03-27 11:12:52 +08:00
docker-compose.triton.yml	Add gptj converter (#19 )	2023-03-27 11:12:52 +08:00
docker-compose.yml	Add gptj converter (#19 )	2023-03-27 11:12:52 +08:00

README.md

Deployment

Start service
```
docker-compose up
```

Test API endpoint with curl

curl -X POST http://localhost:5000/v1/completions -H 'Content-Type: application/json' --data '{
    "prompt": "def binarySearch(arr, left, right, x):\n    mid = (left +"
}'