llmserve
Any model. Any backend. One TUI to serve them all.
curl -fsSL https://llmserve.axjns.dev/install.sh | sh
brew tap AlexsJones/llmserve && brew install llmserve
cargo install llmserve
Sister project: llmfit — find which models fit your hardware