Change theme
Help
Press space for more information.
Show links for this issue (Shortcut: i, l)
Copy issue ID
Previous Issue (Shortcut: k)
Next Issue (Shortcut: j)
Sign in to use full features.
Vote: I am impacted
Notification menu
Refresh (Shortcut: Shift+r)
Go home (Shortcut: u)
Use Markdown for this comment
Set severity, which reflects how much the issue affects the use of the product
Assign issue to yourself
Pending code changes (auto-populated)
[ID: 82937]
Primary programming language affected, if applicable [ID: 82936]
[ID: 82935]
[ID: 82940]
[ID: 82941]
Set the version(s) of the product affected by this issue (comma-separated list)
Set the version(s) of the product in which the issue should be fixed (comma-separated list)
Set the version(s) of the product in which the issue fix was verified (comma-separated list)
Set if this issue occurs in production
Set Reporter
Set Type
Set priority, which reflects how soon the issue should be fixed
Set Status
Set Assignee
Set Verifier
Remove item
View or edit staffing
View issue level access limits(Press Alt + Right arrow for more information)
Description
What you would like to accomplish:
In order to reduce the latency of online predictions, customer considers necessary implementing some customization options to the autoscaling algorithm
How this might work:
• Option to specify the idle time: Currently, the documentation states that the service scales down to zero after several minutes without a prediction request. Customer wants to specify how many minutes they want to keep the workers up before it scales down.
• Option to have both a minimum node count and the scaling down to zero: Currently, the minNodes option disables the scaling down to zero, customer wants that when the service scaled down to zero and then it gets a new request, the service pushes up a minimum number of nodes.
• Options for aggressively adding more nodes: For example, customer wants to specify that if the utilization is above 50%, then add more nodes.
If applicable, reasons why alternative solutions are not sufficient:
Low latency is business critical for the customer, but they want to avoid unused resources as well for when their customers are not likely to submit requests, so those are some options the customer wants in order to guarantee a low latency for the most of the time.
Other information (workarounds you have tried, documentation consulted, etc):