tabby

History

Meng Zhang e92a8c8005 Release 0.7.0-dev http-api-bindings@0.7.0-dev juniper-axum@0.7.0-dev llama-cpp-bindings@0.7.0-dev tabby@0.7.0-dev tabby-common@0.7.0-dev tabby-download@0.7.0-dev tabby-inference@0.7.0-dev tabby-scheduler@0.7.0-dev tabby-webserver@0.7.0-dev Generated by cargo-workspaces		2023-11-27 14:58:58 +08:00
..
src	refactor: make language optional in TextGenerationOptions (#897 )	2023-11-26 11:17:31 +08:00
Cargo.toml	Release 0.7.0-dev	2023-11-27 14:58:58 +08:00
README.md	feat: add support vertex-ai http bindings (#419 )	2023-09-09 11:22:58 +00:00

README.md

Examples

export MODEL_ID="code-gecko"
export PROJECT_ID="$(gcloud config get project)"
export API_ENDPOINT="https://us-central1-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/us-central1/publishers/google/models/${MODEL_ID}:predict"
export AUTHORIZATION="Bearer $(gcloud auth print-access-token)"

cargo run --example simple

Usage

export MODEL_ID="code-gecko"
export PROJECT_ID="$(gcloud config get project)"
export API_ENDPOINT="https://us-central1-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/us-central1/publishers/google/models/${MODEL_ID}:predict"
export AUTHORIZATION="Bearer $(gcloud auth print-access-token)"

cargo run serve --device experimental-http --model "{\"kind\": \"vertex-ai\", \"api_endpoint\": \"$API_ENDPOINT\", \"authorization\": \"$AUTHORIZATION\"}"