google-app-engine


AppEngine entity modeling - minimizing entity groups and achieving atomic cascading update/delete


Am learning AppEngine and have started developing new app and want to clarify something.
I understood that
a. To achieve atomicity of update/delete of several entities we need to do it in a transaction and hence all should fall under same entity group
b. Having big entity groups is not scalable as it causes contention.
(Q1: Correct?)
So here is an entity model of an online examination system for sake of discussion:
Entities:
Subject
Exam
Page
Question
Answer
As you can see from top, each entity 1 - many relationship with the immediate bottom one i.e 1 Subject can have many exams, 1 exam -> many pages, 1 page can have many questions...
As you can see, i would like to establish cascading update/delete relationship among these entities (JPA datanucleus appengine implemention supports this (under the hood) by putting all entities under same entity group (Q2: Correct?) though AppEngine natively doesn't support this constraint) so naturally all would go under same entity group so that
a. i can delete a Page (if my user does) in a transaction and be sure that all pages, questions, answers are all deleted
b. or i can delete a subject altogether in a transaction all clear all stuff underneath it
So when i extend this to my real app, i see that all of my (or atleast most) entities are interrelated and fit into same entity group to be able to transact them altogether - making my model inefficient.
Q3: Please advice on how to rethink this design (and the best practice) and still achieve what i need. Ask me more if needed.
Would be great if you could point me to relevant examples.
p.s. 1 solution i could think of is having each entity in a separate entity group and a separate persistent field in each entity (say Exam) named 'IS_DELETED' defaulting to FALSE (value 0). Once a user deletes an Exam, i will set the field to 1 (TRUE) and that i don't load them anymore. I shall write a Cron job which clears all related entities in separate separate transaction in the backend which will retry upon failures if needed. But am sure this is not elegant and not sure whether this will work out..
Thanks all for your responses,
Hari
One of the simplest ways to improve things is to just have fewer entities in the first place. I can't really think of a terribly good reason why pages, questions and answers need to be separate entities. I suspect you normally display all of the questions on a single page in the same request, without exception. If that's really the case, just keep them in one entity.
It does make a lot of sense to use the Exam entities as the parent for pages; for one thing, each exam is probably limited to a reasonable, small number of pages, so scaling this up probably won't hurt much.
On the other hand, there probably are a great many exams per subject, and for that reason, subjects should not appear in the ancestry of exams (and by extension, pages).
If, for some reason you needed to delete all of the exams in the subject of math, even if they were in the same entity group, you'd probably be unable to complete the whole delete in one transaction without timing out. You might even have trouble completing the delete in a single request.
That suggests that you should be using the Task Queue for this operation. When a cascading change on a subject occurs, the request handler needs to insert a new task and then just return successfully. don't forget to just update the subject entity right there in the request handler.
The task queue pulls a block of affected entities from the datastore, updates them, and then checks the time. If there is still more time available for continued updates, it pulls another block of entities, and so on, until none remain. If time is almost up, the task just adds itself back to the queue so it can restart where it left off when it respawns.
It's a good idea to schedule the first task at least a few seconds into the future of the initial request, so that if, for instance, the subject was deleted, the delete can propagate to future requests and no new exams in that subject can be created by the time the task starts.

Related Links

GAE printing same log statement multiple times
Adding multiple accounts for “You do not have permission to modify this app” error
Logs are Not Nested Under Requests in Flexible VM
How to create an equivalent of a background thread for an auto-scaling instance
Communication between modules locally
Technology for realtime messaging to mobile apps
how to apply date filter on ancestor query
How to enable speech api higher quota
Google App Engine standard environment to Cloud-SQL Second Generation instance
GAE: Restore ah-builtin-python-bundle and ah-builtin-datastoreservice
Google Cloud Storage Force Download
Importing data into 2nd generation google cloud sql
App Engine: What is the Maximum URLFetch Timeout Deadline in a Taskqueue / Backend
Using URL-safe keys in Google Cloud Datastore
AttributeError on google cloud datastore entity object
Is there any way to handle Google Datastore Kind Property names with spaces in Golang?

Categories

HOME
spark-streaming
codenvy
amp-html
angular-formly
oracle-apex-5
sh
requirements
pclxl
tail
square
limit
jetty
x264
lanczos
endeca
android-securityexception
google-form
endpoint
flatpak
firefox-addon
asp.net-mvc-viewmodel
mapstruct
shortest-path
jmp
connector
windows-xp
sar
qliksense
tosca
xcode-ui-testing
appirater
tpm
paperjs
jaws
event-flow
git-squash
spring-security4
dynamic-jasper
sql-delete
xajax
git-rewrite-history
connection-pool
demo
xv6
doc
placeholder
photos
httr
x++
openrasta
wcftestclient
reportservice2010
rserve
openblas
zenhub
tinkerpop
yii2-user
android-testing
ngmaterial
seq
rackspace-cloud
django-1.10
android-instrumentation
stripe-connect
custom-lists
segment-io
mta
cfchart
mediametadataretriever
boost-serialization
phpwebsocket
halcon
realsense
exim4
blitline
zen-cart
dock
theos
jbake
sip-server
spy++
kissfft
vbe
cfcache
cnf
time.h
pagedown
stacky
dopostback
pstack
fragmenttransaction
veracity
zope.interface
cuda-gdb
relative
pinch
winmain
mtu
globals
silverlight-oob
product-management
change-management

Resources

Encrypt Message