📚 The doc issue In both examples for logging custom metrics, emit_metrics is imported but not called. Is this correct? To me, it's not intuitive that simply importing it would be necessary. The two examples: L546 in metrics.md L562 in metrics.md I would be grateful for any assistance. Suggest a po...

Commit 4ac7940 daily scheduled CI run failed, please check why

🐛 Describe the bug I have benchmarked GRPC vs HTTP expecting to achieve certain improvements on my application but I was surprised to find that GRPC can be a lot slower compared to HTTP. Basically GPRC performance is highly dependent on model response. This is probably due to .proto declaration: mes...

Commit 1b10513 daily scheduled CI run failed, please check why

🚀 The feature To track PT2.0 better transformer performance, nightly benchmark is moved tp g5 instances Motivation, pitch Track PT2.0 better transformer performance. Alternatives No response Additional context No response

🚀 The feature Today, handler pre and post processes batch in sequential. This is not efficient for large batch and large payload. To improve performance, handler needs to support pre and post processing in parallel. Motivation, pitch This task is to improve performance for batching. Alternatives No...

Summary The Torchserve model zoo, i.e the S3 bucket (https://torchserve.s3.amazonaws.com/) will need to be migrated across AWS accounts. Since S3 bucket names are required to be unique across accounts and regions, the torchserve bucket will need to be deleted in the source account and then created i...

🚀 The feature Ignore url query parameters when registering/accessing/deleting models. Instead of saving .mar files as filename.mar&queryparams, save them as filename.mar Motivation, pitch I'm using S3 presigned URL's to access models stored on S3. I'm aware of the Encrypted model serving option...

Commit 221da4e daily scheduled CI run failed, please check why

🚀 The feature this migration includes the following tasks: reserve gpu hosts for regression test update host label in regression_tests_gpu.yml Motivation, pitch deprecate regression test on aws cloud-build Alternatives No response Additional context No response