google-app-engine


Best approach for caching lists of objects in memcache


Our Google AppEngine Java app involves caching recent users that have requested information from the server.
The current working solution is that we store the users information in a list, which is then cached.
When we need a recent user we simply grab one from this list.
The list of recent users is not vital to our app working, and if it's dropped from the cache it's simply rebuilt as users continue to request from the server.
What I want to know is: Can I do this a better way?
With the current approach there is only a certain amount of users we can store before the list gets to large for memcache (we are currently limiting the list to 1000 and dropping the oldest when we insert new). Also, the list is going to need updating very quickly which involves retrieving the full list from memcache just to add a single user.
Having each user stored in cache separately would be beneficial to us as we require the recent user to expire after 30 minutes. At the moment this is a manual task we do to make sure the list does not include expired users.
What is the best approach for this scenario? If it's storing the users separately in cache, what's the best approach to keeping track of the user so we can retrieve it?
You could keep in the memcache list just "pointers" that you can use to build individual memcache keys to access user entities separately stored in memcache. This makes the list's memcache size footprint a lot smaller and easy to deal with.
If the user entities have parents then pointers would have to be their keys, which are unique so they can be used as memcache keys as well (well, their urlsafe versions if needed).
But if the user entities don't have parents (i.e. they're root entities in their entity groups) then you can use their datastore key IDs as pointers - typically shorter than the keys. Better yet, if the IDs are numerical IDs you can even store them as numbers, not as strings. The IDs are unique for those entities, but they might not be unique enough to serve as memcache keys, you may need to add a prefix/suffix to make the respective memcache keys unique (to your app).
When you need a user entity data you 1st obtain the "pointer" from the list, build the user entity memcache key and retrieve the entity with that key.
This, of course, assumes you do have reasons to keep that list in place. If the list itself is not mandatory all you need is just the recipe to obtain the (unique) memcache keys for each of your entities.
If you use NDB caching, it will take care of memcache for you. Simply request the users with the key using ndb.Key(Model, id).get() or Model.get_by_id(id), where id is the User id.

Related Links

'Model is not immutable' TypeError
How to store data from SmartGWT ListGrid?
BigQuery OAuth access token on Java App Engine dev server
Google App Engine ancestor query in JPA
Google App Engine Go memory management
How to build Many-to-Many relations in Google App-engine using JPA
Currently experiencing frequent DeadlineExceededExceptions when it was fine a few hours ago
How to get related to db.Model entity from datastore - what API allows it?
GAE-Python: Can Appstats record RPC made in background thread?
Blank Page after granting OAuth2 permission in BigQuery GAE Sample
GAE get Data using JDO with key
Setup GAE project with Resteasy using Maven in Eclipse
Working Maven3 configuration for AppEngine + DataNucleus + JPA 2
Spring Security ACL on App Engine Datastore
JDO 1:N issue (retriving data)
Endpoints Bad Api Configuration

Categories

HOME
oracle12c
paraview
time-complexity
jmeter
mstest
gradient
replace
xbee
frontend
odoo-8
nfs
accessibility
appcelerator-studio
comsol
internationalization
azure-iot-hub
dimensional-modeling
mapstruct
delphi-xe5
replication
selinux
ember-cli
ios10.2
qliksense
highstock
comparator
draft-js-plugins
xcode-ui-testing
jackson-modules
progress-bar
rpgle
mql
eviews
complex-numbers
utorrent
testbed
jbutton
quickfixj
code-snippets
linkedhashmap
mnist
sharepoint-workflow
lint
railstutorial.org
p4merge
unordered-map
branching-and-merging
parser-generator
procfile
pom.xml
flask-restful
fwrite
facebook-pixel
rserve
manova
insight
google-cse
avx
css-transforms
gets
chromium-os
simplepie
meteor-packages
redbeard-ios
preprocessor
hclust
android-dateutils
firebase-security
appscale
http-status-code-401
halcon
realsense
ascii-art
libvlc
.bash-profile
doctype
viola-jones
sql-server-data-tools
linked-tables
avahi
jquery-forms-plugin
diff3
java-security
boost-program-options
spring-remoting
synapse
multitasking
spring-3
rolify
cffile
observium
css-reset
jquery-1.9
font-awesome-4.0.0
windows-98
booksleeve
curb
android-loadermanager
virtualpathprovider
automationelement
umra
luajava
isapi-extension
zend-rest
opml
build-environment
linfu-dynamicproxy

Resources

Encrypt Message