Discussion:
Run Crypto.Random.atfork() when new worker process is initialized
Alex S-B
2013-04-11 00:21:30 UTC
Permalink
I'm using django-celery to run some tasks that make use of PyCrypto (via
python-keyczar). PyCrypto requires that Crypto.Random.atfork() be
executed whenever os.fork() is invoked. I discovered this because it raises
an exception[1] when I try to use it in my celery task. How can I run this
whenever celery initializes a newly forked process? Is there a standard
hook or signal to use to insert this code?

Thanks!
Alex

[1] "AssertionError: PID check failed. RNG must be re-initialized after
fork(). Hint: Try Random.atfork()"
--
You received this message because you are subscribed to the Google Groups "celery-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to celery-users+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to celery-users-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
Visit this group at http://groups.google.com/group/celery-users?hl=en-US.
For more options, visit https://groups.google.com/groups/opt_out.
Rich Rauenzahn
2013-04-11 18:17:22 UTC
Permalink
Your question seems a little similar to
http://stackoverflow.com/questions/2129820/how-can-i-set-up-celery-to-call-a-custom-initialization-function-before-running

Otherwise, I was going to suggest putting it into your global settings,
similar to putting djcelery.setup_loader() into the django settings.py
--
You received this message because you are subscribed to the Google Groups "celery-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to celery-users+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to celery-users-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
Visit this group at http://groups.google.com/group/celery-users?hl=en-US.
For more options, visit https://groups.google.com/groups/opt_out.
Alex S-B
2013-04-12 09:51:12 UTC
Permalink
That particular example demonstrates how to initialize the worker before
forking the child process, but it looks like the same principle of creating
a custom loader might allow me to execute the necessary code in each child
process of the worker by writing a custom on_worker_process_init
(http://docs.celeryproject.org/en/latest/_modules/celery/loaders/base.html#BaseLoader.on_worker_process_init).
I'll give that a try. Thanks Rich!
Post by Rich Rauenzahn
Your question seems a little similar to
http://stackoverflow.com/questions/2129820/how-can-i-set-up-celery-to-call-a-custom-initialization-function-before-running
Otherwise, I was going to suggest putting it into your global settings,
similar to putting djcelery.setup_loader() into the django settings.py
--
You received this message because you are subscribed to the Google Groups "celery-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to celery-users+unsubscribe-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
To post to this group, send email to celery-users-/JYPxA39Uh5TLH3MbocFF+G/***@public.gmane.org
Visit this group at http://groups.google.com/group/celery-users?hl=en-US.
For more options, visit https://groups.google.com/groups/opt_out.
Loading...