Excited to share that our accompanying paper for the Inference Perf project (under Kubernetes SIGs) has been published in the Journal of Open Source Software!
Co-authors: Ashok Chandrasekar, Sachin Mathew Varghese, Jason Kramberger, Brendan Slabe, Chen Wang
The project was created as a part of the inference benchmarking and metrics standardization effort in the Kubernetes Serving Working Group, and seeks to standardize tooling and metrics for measuring inference performance across the Kubernetes and model server communities.
Paper:
Repo: