tabby/deployment/skypilot/README.md

Run Tabby server on any cloud with one click

Background

SkyPilot is an open-source framework for seamlessly running machine learning on any cloud. With a simple CLI, users can easily launch many clusters and jobs, while substantially lowering their cloud bills. Currently, Lambda Labs (low-cost GPU cloud), AWS, GCP, and Azure are supported. See docs to learn more.

Steps

1. Install SkyPilot and check that cloud credentials exist:

   pip install "skypilot[aws,gcp,azure,lambda]"  # pick your clouds
   sky check
   

   

2. Get the deployment folder:

   git clone https://github.com/TabbyML/tabby
   cd tabby/deployment/skypilot
   

3. run:

   sky launch -c tabby default.yml
   

4. Open another terminal and run:

   ssh -L 5000:localhost:5000 tabby
   

5. Open http://localhost:5000 in your browser and start coding!

Cleaning up

When you are done, you can stop or tear down the cluster:

• To stop the cluster, run

  sky stop tabby # or pass your custom name if you used "-c <other name>"
  

  You can restart a stopped cluster and relaunch the chatbot (the run section in YAML) with

  sky launch default.yml -c tabby --no-setup
  

  Note the --no-setup flag: a stopped cluster preserves its disk contents so we can skip redoing the setup.
• To tear down the cluster (non-restartable), run

  sky down tabby # or pass your custom name if you used "-c <other name>"