Live page · Day archive

Serving GLM-5.3 at 607.8 tok/s across two Nvidia DGX stations with vLLM

Read the original at Hacker News

Summary

vLLM serves GLM-5.3 at 607.8 tokens per second across two Nvidia DGX stations using ConnectX-8 RDMA.

Carried by: Hacker News. First seen: .