root
/
tabby
mirror of https://gitee.com/zhang_1334717033/tabby.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
tabby/crates/http-api-bindings
History
Meng Zhang 54ed003757 Release 0.5.1-rc.3 
http-api-bindings@0.5.1-rc.3
llama-cpp-bindings@0.5.1-rc.3
tabby@0.5.1-rc.3
tabby-common@0.5.1-rc.3
tabby-download@0.5.1-rc.3
tabby-inference@0.5.1-rc.3
tabby-scheduler@0.5.1-rc.3

Generated by cargo-workspaces
 		2023-11-05 12:28:10 -08:00
..
src fix: align with fastchat mainstream (#670) 2023-10-29 19:31:46 -07:00
Cargo.toml Release 0.5.1-rc.3 2023-11-05 12:28:10 -08:00
README.md feat: add support vertex-ai http bindings (#419) 2023-09-09 11:22:58 +00:00

README.md

Examples

export MODEL_ID="code-gecko"
export PROJECT_ID="$(gcloud config get project)"
export API_ENDPOINT="https://us-central1-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/us-central1/publishers/google/models/${MODEL_ID}:predict"
export AUTHORIZATION="Bearer $(gcloud auth print-access-token)"

cargo run --example simple

Usage

export MODEL_ID="code-gecko"
export PROJECT_ID="$(gcloud config get project)"
export API_ENDPOINT="https://us-central1-aiplatform.googleapis.com/v1/projects/${PROJECT_ID}/locations/us-central1/publishers/google/models/${MODEL_ID}:predict"
export AUTHORIZATION="Bearer $(gcloud auth print-access-token)"

cargo run serve --device experimental-http --model "{\"kind\": \"vertex-ai\", \"api_endpoint\": \"$API_ENDPOINT\", \"authorization\": \"$AUTHORIZATION\"}"