diff --git a/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/building.png b/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/building.png new file mode 100644 index 0000000..62daa77 Binary files /dev/null and b/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/building.png differ diff --git a/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/curl.png b/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/curl.png new file mode 100644 index 0000000..af8d094 Binary files /dev/null and b/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/curl.png differ diff --git a/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/docker-space.png b/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/docker-space.png new file mode 100644 index 0000000..1bb67e1 Binary files /dev/null and b/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/docker-space.png differ diff --git a/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/edit-dockerfile.png b/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/edit-dockerfile.png new file mode 100644 index 0000000..0580040 Binary files /dev/null and b/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/edit-dockerfile.png differ diff --git a/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/edit-readme.png b/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/edit-readme.png new file mode 100644 index 0000000..9611f9a Binary files /dev/null and b/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/edit-readme.png differ diff --git a/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/empty-readme.png b/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/empty-readme.png new file mode 100644 index 0000000..bf76b85 Binary files /dev/null and b/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/empty-readme.png differ diff --git a/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/empty-space.png b/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/empty-space.png new file mode 100644 index 0000000..cbebcc6 Binary files /dev/null and b/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/empty-space.png differ diff --git a/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/index.md b/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/index.md new file mode 100644 index 0000000..9ad6454 --- /dev/null +++ b/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/index.md @@ -0,0 +1,133 @@ +--- +slug: deploy-tabby-to-huggingface-space.md +title: Deploying a Tabby Instance in Hugging Face Spaces +authors: + - name: Rand Xie + url: https://github.com/randxie + image_url: https://github.com/randxie.png + - meng + - gyxlucy +tags: [deployment] +--- + +[Hugging Face Spaces](https://huggingface.co/spaces) offers an easy-to-use Nvidia GPU hosting runtime, allowing anyone to host their machine learning models or AI applications. + +In this blog post, we are going to show you how to deploy a Tabby instance in Hugging Face Spaces. If you have not heard of Tabby, it’s an open-source Github Copilot alternative that supports code completion. +Check out more details [here](https://github.com/TabbyML/tabby). + +### How it works + +Let’s firstly take a look at what steps are needed to deploy a Tabby instance in Hugging Face. It’s super easy and you don’t need much coding knowledge. Buckle up and let’s get started. + +**Step 1:** Create a new Hugging Face Space ([link](https://huggingface.co/new-space)). Spaces are code repositories that host application code. + +**Step 2:** Create a Dockerfile to capture your machine learning models' logic, and bring up a server to serve requests. + +**Step 3:** After space is built, you will be able to send requests to the APIs. + +That's it! With the hosted APIs, now you can connect Tabby's [IDE extensions](/docs/getting-started) to the API endpoint. Next, we will deep dive into each step with screenshots!! +**Everything will be done in the Hugging Face UI. No local setup is needed.** + +:::tip +Looking to quickly start a Tabby instance? You can skip the tutorials entirely and simply [duplicate the space](https://huggingface.co/spaces/randxie/tabbyml?duplicate=true). +::: + +### Deep Dive + +#### Create a new Space + +After you create a Hugging Face account, you should be able to see the following page by clicking this [link](https://huggingface.co/new-space). +The owner name will be your account name. Fill in a Space name, e.g. "tabbyml", and select Docker as Space SDK. Then click "Create Space" at the bottom. + +In this walkthrough we recommend using **Nvidia T4** instance to deploying a model of ~1B parameter size. + +![Create a new Space](./new-space.png) + +#### Uploading Dockerfile + +For advanced users, you can leverage the Git workspace. In this blog post, we will show you the UI flow instead. After you click the "Create a Space" in the last step, you will be directed to this page. Just ignore the main text and click the "Files" on the top right corner. + +![Docker Space](./docker-space.png) + +After clicking on the "Files", you will be able to see a "Add file" button, click that, then click on "Create a new file" + +![Empty Space](./empty-space.png) + +Then you will be redirected to the page below. Set the filename to “Dockerfile” and copy the content to the “Edit” input box. You can copy the code from the [appendix](#dockerfile) here to bring up the [SantaCoder-1B](https://huggingface.co/TabbyML/SantaCoder-1B) model. Once ready, click the button “Commit new file to main” on the bottom. + +![Edit Dockerfile](./edit-dockerfile.png) + +#### Edit Readme + +You also need to add a new line the README.md. Click the "edit" button in the README.md file. + + +![Empty README](./empty-readme.png) + +Add this line "app_port: 8080" after "sdk: docker" + +![Edit README](./edit-readme.png) + +Click the button "Commit to main" to save the changes. + +#### Verify Tabby is running + +Click on the "App" button, you should be able to see the container is building: + +![Space Building](./building.png) + +If the App is up successfully, you should see this page: + +![Tabby Swagger](./tabby-swagger.png) + +#### Call code completion API + +Now, you are able to call the completion API. The full URL is https://{YOUR-ACCOUNT-NAME}-tabbyml.hf.space/v1/completions. In this post, the URL is https://randxie-tabbyml.hf.space/v1/completions. + +To test if your APIs are up and running, use [this online tool](https://reqbin.com/curl) to send curl commands: + +![curl](./curl.png) + +The complete curl command can also be located in the [appendix](#curl-command). Ensure that you have adjusted the URL to align with your Hugging Face Spaces settings! + +(If you are setting the space to private, you will need to fill in your Huggingface Access Token as bearer token in HTTP Headers, like `Authorization: Bearer $HF_ACCESS_TOKEN`.) + +### Conclusion +In this post, we covered the detailed steps for deploying a Tabby instance to Hugging Face Spaces. By following these steps, anyone is able to bring up their own code completion APIs easily. + +### Appendix + +#### Dockerfile +```Dockerfile +FROM tabbyml/tabby + +RUN useradd -m -u 1000 user + +USER user + +ENV HOME=/home/user \ + PATH=/home/user/.local/bin:$PATH + +ENV TABBY_ROOT=$HOME/app + +WORKDIR $HOME/app + +COPY --chown=user . $HOME/app + +CMD ["serve", ”--device", "cuda", "--model", "TabbyML/SantaCoder-1B"] + +``` + +#### CURL Command +```bash +curl -L 'https://randxie-tabbyml.hf.space/v1/completions' \ +-H 'Content-Type: application/json' \ +-H 'Accept: application/json' \ +-d '{ + "language": "python", + "segments": { + "prefix": "def fib(n):\n ", + "suffix": "\n return fib(n - 1) + fib(n - 2)" + } +}' +``` diff --git a/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/new-space.png b/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/new-space.png new file mode 100644 index 0000000..384c389 Binary files /dev/null and b/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/new-space.png differ diff --git a/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/tabby-swagger.png b/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/tabby-swagger.png new file mode 100644 index 0000000..7a247d6 Binary files /dev/null and b/website/blog/2023-09-05-deploy-tabby-to-huggingface-space/tabby-swagger.png differ diff --git a/website/blog/authors.yml b/website/blog/authors.yml index 0540b6f..4df7b28 100644 --- a/website/blog/authors.yml +++ b/website/blog/authors.yml @@ -1,5 +1,9 @@ meng: name: Meng Zhang - title: Maintainer of Tabby url: https://github.com/wsxiaoys image_url: https://github.com/wsxiaoys.png + +gyxlucy: + name: Lucy Gao + url: https://github.com/gyxlucy + image_url: https://github.com/gyxlucy.png