3 posts tagged with "Rust"

Building production ML inference services that run anywhere—from Raspberry Pi to cloud edge—requires a different approach. This article walks through a complete implementation of a text embedding API using WasmEdge, GGML, and Rust, delivering a 136KB WASM module paired with a 1.8MB async HTTP server that processes embeddings in ~100-200ms per request.

Full implementation: github.com/porameht/wasmedge-ggml-llama-embedding

When scraping websites or testing APIs from multiple IPs, you need a proxy that can rotate source addresses automatically. This article explores how to build a production-ready IP rotation proxy using Cloudflare's Pingora framework, achieving lock-free rotation with atomic operations.

Full implementation: github.com/porameht/pingora-forward-proxy

In ML inference servers, choosing the right concurrency pattern can make the difference between 200 RPS and 20,000 RPS. This article analyzes why Arc<RwLock<Option<T>>> is often the optimal choice for shared model state.