TimeLimitExceeded exception problem in case of using eventlet concurrency pool

[lssergey]

Hello,

I have tasks.py file:


from celery.task import task
from eventlet.green import urllib2

@task
def get_slow_response(delay):
print urllib2.urlopen('http://slowapi.herokuapp.com/delay/%s' %
delay).read()


I run Celery with '-P eventlet' option, enter next lines in python shell:

from celery.task import control
from tasks import get_slow_response
control.time_limit("tasks.get_slow_response", hard=5, reply=True)
get_slow_response.delay(10).wait()

and receive a response from slowapi.com in 10 seconds, although I expect
TimeLimitExceeded exception in 5 seconds.

Why Celery does not kill eventlet green thread when time limit is exceeded?

(If Celery is launched without eventlet, everything is fine, exception is
raised in 5 seconds.)

Thanks

[David Markey]

As timeouts are signal based.. I don't think they would work with
eventlets.

Post by lssergey
Hello,
from celery.task import task
from eventlet.green import urllib2
@task
print urllib2.urlopen('http://slowapi.herokuapp.com/delay/%s' %
delay).read()
from celery.task import control
from tasks import get_slow_response
control.time_limit("tasks.get_slow_response", hard=5, reply=True)
get_slow_response.delay(10).wait()
and receive a response from slowapi.com in 10 seconds, although I expect
TimeLimitExceeded exception in 5 seconds.
Why Celery does not kill eventlet green thread when time limit is exceeded?
(If Celery is launched without eventlet, everything is fine, exception is
raised in 5 seconds.)
Thanks
--
You received this message because you are subscribed to the Google Groups
"celery-users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/celery-users/-/UHASdH9EnQgJ.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/celery-users?hl=en.

[Andrew Novoselsky]

David, if you are right - adding note about that to docs is the best option
I suppose. Now there is nothing about time limits and eventlet -
http://ask.github.com/celery/userguide/workers.html#time-limits

BTW - eventlet has it's own timeout infrastructure(
http://eventlet.net/doc/modules/timeout.html) probably it may be integrated
with celery if eventlet pool is used.

Ask, is it correct that eventlet pool has no timelimits?

Post by David Markey
As timeouts are signal based.. I don't think they would work with
eventlets.

Post by lssergey
Hello,
from celery.task import task
from eventlet.green import urllib2
@task
print urllib2.urlopen('http://slowapi.herokuapp.com/delay/%s' %
delay).read()
from celery.task import control
from tasks import get_slow_response
control.time_limit("tasks.get_slow_response", hard=5, reply=True)
get_slow_response.delay(10).wait()
and receive a response from slowapi.com in 10 seconds, although I expect
TimeLimitExceeded exception in 5 seconds.
Why Celery does not kill eventlet green thread when time limit is exceeded?
(If Celery is launched without eventlet, everything is fine, exception is
raised in 5 seconds.)
Thanks
--
You received this message because you are subscribed to the Google Groups
"celery-users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/celery-users/-/UHASdH9EnQgJ.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/celery-users?hl=en.

--
You received this message because you are subscribed to the Google Groups
"celery-users" group.
To unsubscribe from this group, send email to
For more options, visit this group at
http://groups.google.com/group/celery-users?hl=en.

[Ask Solem]

Post by lssergey
Hello,
Why Celery does not kill eventlet green thread when time limit is exceeded?
(If Celery is launched without eventlet, everything is fine, exception is raised in 5 seconds.)

Time limits does not currently work with eventlet/gevent.

You could use eventlet.Timeout:


with Timeout(5):
....


I would guess celery could also implement time limits for eventlet this way,
but this has not been done so far. One limitation is that eventlet.Timeout
doesn't help if you have blocking calls, maybe using sigalrm could be a solution.


I updated the docs in the repo to tell what supports what.