Awesome-Rust-MachineLearning icon indicating copy to clipboard operation
Awesome-Rust-MachineLearning copied to clipboard

Add shimmy - Python-free Rust inference server

Open Michael-A-Kuykendall opened this issue 3 months ago • 0 comments

Adds shimmy to the Interface & Pipeline & AutoML section.

Project Details:

  • Repository: https://github.com/Michael-A-Kuykendall/shimmy
  • Crate: https://crates.io/crates/shimmy
  • Stars: 2,918 ⭐
  • Description: Python-free Rust inference server — OpenAI-API compatible

Key ML Features:

  • OpenAI-compatible API for seamless integration
  • GGUF + SafeTensors format support
  • Hot model swapping for dynamic serving
  • Auto-discovery of available models
  • Single binary deployment (no Python dependencies)
  • GPU acceleration support
  • LLaMA, ChatML, and various model architectures

Why this belongs in Awesome Rust ML:

  • Pure Rust implementation for ML inference serving
  • Production-ready inference server with significant adoption
  • Fills important gap for OpenAI-compatible Rust inference solutions
  • Zero-dependency deployment suitable for ML production environments
  • Active development with regular feature additions

Placement: Added alphabetically to Interface & Pipeline & AutoML section alongside other inference servers like orkhon, wonnx, and tract.

Following contribution guidelines: "Please just update the README.md"

Michael-A-Kuykendall avatar Oct 18 '25 17:10 Michael-A-Kuykendall