django


Django and Celery - re-loading code into Celery after a change


If I make a change to tasks.py while celery is running, is there a mechanism by which it can re-load the updated code? or do I have to shut Celery down a re-load?
I read celery had an --autoreload argument in older versions, but I can't find it in the current version:
celery: error: unrecognized arguments: --autoreload
You could try SIGHUP on the parent worker process, it restarts the worker, but I'm not sure if it picks up new tasks. Worth a shot, thought :)
Unfortunately --autoreload doesn't work and it is deprecated.
You can use Watchdog which provides watchmedo a shell utilitiy to perform actions based on file events.
pip install watchdog
You can start worker with
watchmedo auto-restart -- celery worker -l info -A foo
By default it will watch for all files in current directory. These can be changed by passing corresponding parameters.
watchmedo auto-restart -d . -p '*.py' -- celery worker -l info -A foo
If you are using django and don't want to depend on watchdog, there is a simple trick to achieve this. Django has autoreload utility which is used by runserver to restart WSGI server when code changes.
The same functionality can be used to reload celery workers. Create a seperate management command called celery. Write a function to kill existing worker and start new worker. Now hook this function to autoreload as follows.
import shlex
import subprocess
from django.core.management.base import BaseCommand
from django.utils import autoreload
def restart_celery():
cmd = 'pkill -9 celery'
subprocess.call(shlex.split(cmd))
cmd = 'celery worker -l info -A foo'
subprocess.call(shlex.split(cmd))
class Command(BaseCommand):
def handle(self, *args, **options):
print('Starting celery worker with autoreload...')
autoreload.main(restart_celery)
Now you can run celery worker with python manage.py celery which will autoreload when codebase changes.
This is only for development purposes and don't use it in production.

Related Links

Retrieving value from a form in a template
django admin with django-treebeard and parler
Edit all attributes of all instances of a ForeignKey in Model Admin Django
Django customized AdminFileWidget not working for readonly field
Combining queries into one result (rest_views)
How to access file after upload in Django?
Replacement for autocomplete_light.modelform_factory()
How to create a created_by field in django rest framework model?
Custom Authorization class in Django Tastypie
Django - retrieve the form input values
Django Rest Framework - CreateAPIView doesn't let use POST method
Django Validate Bitcoin Address
django RuntimeError: maximum recursion depth exceeded on using model manager
Different django processes same log
How to correctly use template tags in this specific Many-to-Many relationship in Django?
Django push to Heroku: “Unknown command: 'compress'”?

Categories

HOME
android
microsoftgraph
facebook-oauth
fpga
shiro
access
rds
square
accessibility
arangodb
xcodebuild
frequency-distribution
flash-player
selectize.js
symfony-2.8
html5-appcache
symfony-console
production
asp.net-core-webapi
breadth-first-search
squashfs
cgbitmapcontext
parcelable
resourcebundle
syntax-highlighting
amazon-quicksight
apktool
mql
sqlpackage
record
database-comparison
magento-1.9.2.4
docfx
watchman
sessionstorage
diacritics
hackintosh
comparison-operators
doc
trigonometry
preloader
cnc
openblas
convex-optimization
posixct
zenhub
video-conferencing
semantic-logging
colorbar
backbone-collections
payload
shinobi
reachability
choice
cubism.js
parfor
xcode-server
iotivity
clipboard.js
shopping
http-status-code-401
project-organization
grass
kendo-upload
piracy-prevention
xenomai
xvim
eoferror
amiga
nutiteq
fail2ban
code-documentation
mass-assignment
document-repository
apache-spark-1.3
showdialog
beatsmusic
universal-image-loader
raygun
ss7
reflector
functional-java
ray
square-bracket
gd-graph
cffile
octal
diem
mysql-error-2013
chart-director
preload
robotlegs
boost-msm
iboutlet
subproject
window-decoration
failing-tests
virtualpathprovider
jquery-address
google-maps-mobile
screen-size
jquery-tools
android-holo-everywhere
mysql-error-1205
socketserver
lxr
product-management
wspbuilder

Resources

Mobile Apps Dev
Database Users
javascript
java
csharp
php
android
MS Developer
developer works
python
ios
c
html
jquery
RDBMS discuss
Cloud Virtualization
Database Dev&Adm
javascript
java
csharp
php
python
android
jquery
ruby
ios
html
Mobile App
Mobile App
Mobile App