Message Queues
Added by Gerhard Gonter (uni) over 10 years ago
- http://www.ibm.com/developerworks/opensource/library/os-mongodb-work-queues/
- example programs written in Ruby that implement a distributed(?) work queue for video encoding
- mentions ActiveMQ, redis, FFmpeg
- Theora: Open Source Video Codec
- Inotify: (inode notify) is a Linux kernel subsystem that acts to extend filesystems to notice changes to the filesystem, and report those changes to applications. It replaces an earlier facility, dnotify, which had similar goals. http://en.wikipedia.org/wiki/Inotify
- Resque is a Redis-backed Ruby library for creating background jobs, placing them on multiple queues, and processing them later. http://github.com/blog/542-introducing-resque
Replies (3)
introducing resque - Added by Gerhard Gonter (uni) over 10 years ago
Article https://github.com/blog/542-introducing-resque
github itself seems to be using resque, this article discusses that and mentions other queueing systems they tried and abandoned again, e.g.- SQS, Starling, ActiveMessaging, BackgroundJob, DelayedJob, and beanstalkd.
- Starling was mentioned in an article about "Kestrel" which is a "pilot job frame work".
- SQS: Amazon's simple queueing system
- ActiveMessaging
- http://code.google.com/p/activemessaging/ is an attempt to bring the simplicity and elegance of Rails development to the world of messaging. Messaging, (or event-driven architecture) is widely used for enterprise integration, with frameworks such as Java's JMS, and products such as ActiveMQ, Tibco, IBM MQSeries etc.
- Installation: get ActiveMessaging, STOMP, and ActiveMQ
- re-evaluation: Kestrel, AMQP, beanstalkd
resque with perl - Added by Gerhard Gonter (uni) over 10 years ago
modules:
see also:
- Mojolicious::Plugin::Resque Mojolicious helper for sending jobs to a Resque queue.
- http://search.cpan.org/~nuclon/Qless-0.06/lib/Qless.pm also uses redis and is inspired by resque
Ubuntu packages¶
| redis-server | Persistent key-value database with network interface |
| libredis-perl | persistent key-value database with network interface (Perl client library) |
| python-redis | Persistent key-value database with network interface (Python library) |
| ruby-moneta | Ruby interface to multiple key/value stores |
| libanyevent-redis-perl | Perl module that implements a non-blocking Redis client |
apt-get install redis-server libredis-perl python-redis ruby-moneta libanyevent-redis-perl
further reading - Added by Gerhard Gonter (uni) over 10 years ago
T2R:
- http://en.wikipedia.org/wiki/Message-oriented_middleware
- http://en.wikipedia.org/wiki/Flow-based_programming
- http://www.ibm.com/developerworks/library/os-springredis/index.html Build Spring-based applications with Redis as the datastore.
- http://zguide.zeromq.org/page%3aall ZeroMQ - The Guide
- https://blog.serverdensity.com/queueing-mongodb-using-mongodb/ using MongoDB as a queueing system Done
- https://blog.serverdensity.com/mongodb-schema-design-pitfalls/
- https://blog.serverdensity.com/using-celery-for-queuing-requests/
- https://blog.serverdensity.com/how-we-handle-on-call-schedules/
- https://blog.serverdensity.com/does-everyone-hate-mongodb/
- https://blog.serverdensity.com/choosing-a-non-relational-database-why-we-migrated-from-mysql-to-mongodb/
- http://qdb.io/ QDB - Persistent Message Queues With Replay
- http://kr.github.io/beanstalkd/ beanstalkd
- https://github.com/pda/pheanstalk PHP library for beanstalkd