google-app-engine


Why are we experiencing huge latency on one autoscaled Google App Engine instance when several others are available?


Our autoscaling parameters in app.yaml are as follows:
automatic_scaling:
min_idle_instances: 3
max_idle_instances: automatic
max_pending_latency: 30ms
max_concurrent_requests: 20
The result is 3 resident instances and typically 2-6 dynamic instances (depending on traffic), but the load distribution among the instances seems inefficient. In the screenshot below we see 1 instance with the vast majority of requests, and a massive 21s latency (in last minute).
To me this indicates there must be something wrong with our setup to explain these high latencies.
Has anyone experienced issues like this with GCP or App Engine?
Idle instances aren't used to balance current load. They bridge the gap while new dynamic instances are spinning up. In your setup it might be worth trying just one or two idle instances and fiddle with min and max pending latency.
Pending latency is measured by how long a request stays in the queue before it is handled by an instance. The latency you see in your screenshot is the time between request and response. If any single request takes 21 seconds it would look like this. The pending latency could still be below 30ms though.
You should check your logs and see which request takes so long and probably break them up into smaller chunks of work. Many small jobs scale much better than huge jobs. Pending latency will also go up with lots of small jobs and will cause your app to scale properly.

Related Links

Google App Engine Log - ms and cpu_ms [duplicate]
unable to deploy after upgrading to 1.7.4
Model.get(list_of_keys) in a transaction, filtering a single entity group instead of throwing BadRequestError
Google app engine and JPA with Eclipse plug-in
Many-To-Many Relationships in Google App Engine Datastore (ndb)
Is it possible to run Google Omaha on Google App Engine?
Loading request start-up time variation in google app engine?
GAE 1.7.3 - How to debug appengine-web.xml XML error validating
How to setup data fixtures in Google App Engine for Go
How to write a JDO Query to sort result based on long attribute?
GAE doesn't import gflags
Real time notification system on Google App Engine(GAE)
How can I easily get a list of the indexed terms from a search index in the google appengine full text search api in java?
jpa #version on google appengine with #OneToMany: appengine bug or usage error?
How can I delete old backup via cron?
Error on itext-gae.jar

Categories

HOME
search
twitter-bootstrap-3
c++11
ggplot2
3d
battery
package
solver
replace
bitbucket-api
soa
rom
3d-reconstruction
web-hosting
bing-maps
qpython
automatic-ref-counting
eip
libusb
google-picker
imessage
google-maps-autocomplete
named-entity-recognition
react-toolbox
tiki-wiki
opentk
jackson-modules
android-things
wcag
corenlp-server
scatter3d
r-grid
mailkit
sammy.js
openwhisk
hibernate-search
navigator
connection-pool
filesystemwatcher
loading
signal-strength
memory-mapping
stylecop
uiimage
dapper-simplecrud
flask-restful
fwrite
ntvs
dbcontext
manova
java-bytecode-asm
appdomain
spring-data-couchbase
java-6
quicktime
android-instrumentation
cvxpy
stripe-connect
chromium-os
cubism.js
iotivity
spreadjs
turbo-prolog
nio2
toast
jnlua
state-restoration
toran-proxy
tfs-workitem
finalbuilder
pyobjc
node-serialport
xmltype
phpldapadmin
qtwebengine
adehabitathr
iiop
big-ip
raygun
ray
sbcl
nebula
merb
wp7test
alice
android-loadermanager
thttpd
exponentiation
cloudfiles
commerceserver2007
tui
wspbuilder

Resources

Encrypt Message