feat: Expose warmup thread pool to model server - #3660

Open

corona10 wants to merge 2 commits into

tensorflow:masterfrom

corona10:warmup-thread-pool

corona10 commented Jan 3, 2025

Please let me know if I misunderstood the original intention.

Parallel warm-up options were exposed a very long time ago, but AFAIK, there is no way to set it on the model server side.
- 79ac1fd
- 79f9d28
So it would be great if we can set this, and even if we use one thread, it would be beneficial to separate the specific thread pool ("Warmup_ThreadPool") if the user specify the number of thread explicitly.

Please let me know if I missed something or if I understand wrongly.
I just try to understand TensorFlow serving internally.

corona10 added 2 commits

January 4, 2025 00:22


          feat: Expose warmup thread pool to model server

60ad1fc

fix

de30142

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment