django


Django and Celery - re-loading code into Celery after a change


If I make a change to tasks.py while celery is running, is there a mechanism by which it can re-load the updated code? or do I have to shut Celery down a re-load?
I read celery had an --autoreload argument in older versions, but I can't find it in the current version:
celery: error: unrecognized arguments: --autoreload
You could try SIGHUP on the parent worker process, it restarts the worker, but I'm not sure if it picks up new tasks. Worth a shot, thought :)
Unfortunately --autoreload doesn't work and it is deprecated.
You can use Watchdog which provides watchmedo a shell utilitiy to perform actions based on file events.
pip install watchdog
You can start worker with
watchmedo auto-restart -- celery worker -l info -A foo
By default it will watch for all files in current directory. These can be changed by passing corresponding parameters.
watchmedo auto-restart -d . -p '*.py' -- celery worker -l info -A foo
If you are using django and don't want to depend on watchdog, there is a simple trick to achieve this. Django has autoreload utility which is used by runserver to restart WSGI server when code changes.
The same functionality can be used to reload celery workers. Create a seperate management command called celery. Write a function to kill existing worker and start new worker. Now hook this function to autoreload as follows.
import shlex
import subprocess
from django.core.management.base import BaseCommand
from django.utils import autoreload
def restart_celery():
cmd = 'pkill -9 celery'
subprocess.call(shlex.split(cmd))
cmd = 'celery worker -l info -A foo'
subprocess.call(shlex.split(cmd))
class Command(BaseCommand):
def handle(self, *args, **options):
print('Starting celery worker with autoreload...')
autoreload.main(restart_celery)
Now you can run celery worker with python manage.py celery which will autoreload when codebase changes.
This is only for development purposes and don't use it in production.

Related Links

How to get the same date result from Model.objects.create and Model.objects.get in Django?
how to use django restframework as a backend for mobile apps
Validate nested form in Django
Django management command cannot see arguments?
creating list of lists alimented with data from database gives empty list
How to run management command in test file on Travis?
How do i get the control back to my app from the popup which comes up in quickbooks oauth
Django Rest Framework remove csrf
How to set Bootstrap theme for django-endless-pagination?
Asserting for presence of added objects in Django ManyToMany relation
Local templates not loading in django-oscar
How to create a Person(models.Model): to store data submitted through a Survey Form?
How to run Django in development on EC2 so that it is accessible to the internet?
Django syncdb mysql error on localhost -> (1045, "Access denied for user 'ODBC'#'localhost')
Get all transactions on monday between 10 and 11am
setFont in ReportLab, Django

Categories

HOME
algorithm
fuelux
codenvy
merge
fpga
pdfbox
opendj
seo
dojo
babeljs
lanczos
pkcs#11
p2p
fhir
rtf
php-5.6
favicon
altera
google-data-studio
smartclient
production
googlebot
href
named-entity-recognition
react-toolbox
mongoid6
displaytag
delimiter
amazon-quicksight
iis-express
autodesk-designautomation
dotspatial
aws-codecommit
event-flow
zend-debugger
database-comparison
code-snippets
hreflang
hard-drive
activexobject
sql-delete
baidu-map
video-capture
openerp-7
git-rewrite-history
z-notation
railstutorial.org
visa
branching-and-merging
postgresql-8.4
http-request
oim
strftime
builder
netbeans6.8
context-sensitive-grammar
compiler-design
java-6
avx
boo
reshape
java-websocket
filenet
choice
taocp
slackware
latin
android-dateutils
xdoclet
firebase-security
kendo-upload
toast
concept-insights
rasterize
maybe
opl
microsoft-reporting
video-player
rdflib
nutiteq
rrule
public-html
document-repository
cbind
reflector
draw2d-js
paste
cnf
process-explorer
superscrollorama
resource-management
redis-py
onmousemove
frameworkelementfactory
correlated-subquery
predicatebuilder
floating-point-exceptions
agility.js
mysql-error-1205
mtu
undefined-index
ie-compatibility-mode
interprocess
opml
data-mapping
feasibility
google-wave
complex-event-processing
software-lifecycle

Resources

Mobile Apps Dev
Database Users
javascript
java
csharp
php
android
MS Developer
developer works
python
ios
c
html
jquery
RDBMS discuss
Cloud Virtualization
Database Dev&Adm
javascript
java
csharp
php
python
android
jquery
ruby
ios
html
Mobile App
Mobile App
Mobile App