RQ is Python specific, messages use
Redis is a key value store, not a highly available message broker.
RQ Workers fork new child processes for every task.
But is quick and easy!
Take any Python function
import requests def count_words_at_url(url): resp = requests.get(url) return len(resp.text.split())
Create a queue and enqueue the function reference and arguments
from redis import Redis from rq import Queue from count import count_words_at_url q = Queue(connection=Redis()) q.enqueue( count_words_at_url, 'http://uberhip.com' )
Run the worker on the command line
Run the client
Worker logs show
Remember, all we did was enqueue a job, specifying only the function and its arguments.
q = Queue(connection=Redis()) q.enqueue( count_words_at_url, 'http://uberhip.com' )
No queue names, no priorities, no handling of the return values.
To enqueue something on a specific queue, instantiate Queue() with the queue name as an argument.
low_q = Queue('low', connection=Redis()) low_q.enqueue( count_words_at_url, 'http://uberhip.com' )
You can use any queue name.
Workers can be started on specific queues by providing queue names as command line arguments to
Queue keyword arguments include:
at_font. See http://python-rq.org/docs/ for details.
The task function passed to
enqueue can be EITHER a function reference OR a string. So the Workers can be implemented in a separate code base from the
Queue() has the following methods and attributes:
len(q)- Number of jobs in queue
q.job_ids- List of enqueued job IDs
q.jobs- List of enqueued job instances
job = q.fetch_job('my_id')- Returns job having ID: my_id
Prep Exec ->
Read the docs, we don't have time for this!
Want results from your job?
enqueue returns a job reference, save it, ask it for results before the default
results_ttl of 500s expires.
q = Queue(connection=Redis()) job = q.enqueue( count_words_at_url, 'http://uberhip.com' ) time.sleep(2) print job.result
What, if we don't have time for Workers, we certainly don't have time for jobs!
Well, other than to emphasize there are two
result_ttl- result time to live
ttl- job time to live
And ... there's a
meta property to which you can attach arbitrary data using dictionary syntax.
(sounds a little fancy, maybe you want to try a different queueing system)
rq-dashboardis handy for reviewing this.
SimpleWorkerotherwise you might encounter trouble with