22fbaefbd4
* feat: add deployment script on lambda cloud with skypilot * docs: adjust API documentation level * fix: move docker-compose install before the docker-compose pull * Fix documentation * Update replica updating |
||
|---|---|---|
| .. | ||
| README.md | ||
| default.yml | ||
README.md
Run Tabby server on any cloud with one click
Background
SkyPilot is an open-source framework for seamlessly running machine learning on any cloud. With a simple CLI, users can easily launch many clusters and jobs, while substantially lowering their cloud bills. Currently, Lambda Labs (low-cost GPU cloud), AWS, GCP, and Azure are supported. See docs to learn more.
Steps
-
Install SkyPilot and check that cloud credentials exist:
pip install "skypilot[aws,gcp,azure,lambda]" # pick your clouds sky check
-
Get the deployment folder:
git clone https://github.com/TabbyML/tabby cd tabby/deployment/skypilot -
run:
sky launch -c tabby default.yml -
Open another terminal and run:
ssh -L 8501:localhost:8501 -L 5000:localhost:5000 -L 8080:localhost:8080 tabby -
Open http://localhost:8501 in your browser and start coding!
Cleaning up
When you are done, you can stop or tear down the cluster:
- To stop the cluster, run
You can restart a stopped cluster and relaunch the chatbot (thesky stop tabby # or pass your custom name if you used "-c <other name>"runsection in YAML) with
Note thesky launch default.yml -c tabby --no-setup--no-setupflag: a stopped cluster preserves its disk contents so we can skip redoing the setup. - To tear down the cluster (non-restartable), run
sky down tabby # or pass your custom name if you used "-c <other name>"