Why are we experiencing huge latency on one autoscaled Google App Engine instance when several others are available?
Our autoscaling parameters in app.yaml are as follows: automatic_scaling: min_idle_instances: 3 max_idle_instances: automatic max_pending_latency: 30ms max_concurrent_requests: 20 The result is 3 resident instances and typically 2-6 dynamic instances (depending on traffic), but the load distribution among the instances seems inefficient. In the screenshot below we see 1 instance with the vast majority of requests, and a massive 21s latency (in last minute). To me this indicates there must be something wrong with our setup to explain these high latencies. Has anyone experienced issues like this with GCP or App Engine?
Idle instances aren't used to balance current load. They bridge the gap while new dynamic instances are spinning up. In your setup it might be worth trying just one or two idle instances and fiddle with min and max pending latency. Pending latency is measured by how long a request stays in the queue before it is handled by an instance. The latency you see in your screenshot is the time between request and response. If any single request takes 21 seconds it would look like this. The pending latency could still be below 30ms though. You should check your logs and see which request takes so long and probably break them up into smaller chunks of work. Many small jobs scale much better than huge jobs. Pending latency will also go up with lots of small jobs and will cause your app to scale properly.
Google App Engine Log - ms and cpu_ms [duplicate]
unable to deploy after upgrading to 1.7.4
Model.get(list_of_keys) in a transaction, filtering a single entity group instead of throwing BadRequestError
Google app engine and JPA with Eclipse plug-in
Many-To-Many Relationships in Google App Engine Datastore (ndb)
Is it possible to run Google Omaha on Google App Engine?
Loading request start-up time variation in google app engine?
GAE 1.7.3 - How to debug appengine-web.xml XML error validating
How to setup data fixtures in Google App Engine for Go
How to write a JDO Query to sort result based on long attribute?
GAE doesn't import gflags
Real time notification system on Google App Engine(GAE)
How can I easily get a list of the indexed terms from a search index in the google appengine full text search api in java?
jpa #version on google appengine with #OneToMany: appengine bug or usage error?
How can I delete old backup via cron?
Error on itext-gae.jar