python - In celery how to get task position in queue? -


i'm using celery redis broker , can see queue redis list serialized task items.

my question is, if have asyncresult object result of calling <task>.delay(), there way determine item's position in queue?

update:

i'm able position using:

from celery.task.control import inspect = inspect() i.reserved() 

but bit slow since needs communicate workers.

the inspect.reserved()/scheduled() mention may work, not accurate since take account tasks workers have prefetched.

celery not allow out of band operations on queue, removing messages queue, or reordering them, because not scale in distributed system. messages may not have reached queue yet, can result in race conditions , in practice not sequential queue transactional operations, stream of messages originating several locations. is, celery api based around strict message passing semantics.

it possible access queue directly on of brokers celery supports (like redis or database), not part of public api, , discouraged doing so, of course if not planning on supporting operations @ scale should whatever convenient , discard advice.

if give user idea when job completed, i'm sure come algorithm predict when task executed, if had length of queue , time @ each task inserted.

the first redis.len("celery"), , latter add listening task_sent signal:

from celery.signals import task_sent  @task_sent.connect def record_insertion_time(id, **kwargs):    redis.zadd("celery.insertion_times", id) 

using sorted set here: http://redis.io/commands/zadd

for pure message passing solution use dedicated monitor consumes celery event stream , predicts when tasks finish. http://docs.celeryproject.org/en/latest/userguide/monitoring.html#event-reference

(just noticed task-sent missing timestamp field in documentation, timestamp sent event fix it).

the events contain "clock" field logical clock (see http://en.wikipedia.org/wiki/lamport_timestamps), can used detect order of events in distributed system without depending on system time on each machine in sync (which ~impossible achieve).


Comments

Popular posts from this blog

java - Play! framework 2.0: How to display multiple image? -

gmail - Is there any documentation for read-only access to the Google Contacts API? -

php - Controller/JToolBar not working in Joomla 2.5 -