Advanced Message Queuing Protocol (AMQP)
AMQP (Advanced Message Queuing Protocol) is a messaging protocol that enables conforming client applications to communicate with conforming messaging middleware brokers.
Brokers and their roles
Messaging brokers receive messages from publishers (applications that publish them, also known as producers) and route them to consumers (applications that process them). Since it is a network protocol, the publishers, consumers, and the broker can all reside on different machines.
AMQP Model
The AMQP 0-9-1 Model has the following view of the world: messages are published to exchanges (which can be compared to post offices or mailboxes). Exchanges then distribute message copies to queues using rules called bindings. Then the broker either delivers messages to consumers subscribed to queues, or consumers fetch/pull messages from queues on demand.
When publishing a message, publishers may specify various message attributes (message meta-data). Some of this meta-data may be used by the broker, however, the rest of it is completely opaque to the broker and is only used by applications that receive the message.
Networks are unreliable and applications may fail to process messages hence the AMQP model has a notion of message acknowledgments: when a message is delivered to a consumer the consumer notifies the broker, either automatically or as soon as the application developer chooses to do so. When message acknowledgments are in use, a broker will only completely remove a message from a queue when it receives a notification for that message (or group of messages).
In certain situations, for example, when a message cannot be routed, messages may be returned to publishers, dropped, or, if the broker implements an extension, placed into a so-called "dead letter queue". Publishers choose how to handle situations like this by publishing messages using certain parameters. Queues, exchanges and bindings are collectively referred to as AMQP entities.
Programmable protocol
AMQP is a programmable protocol in the sense that its entities and routing schemes are primarily defined by applications themselves, not a broker administrator. Accordingly, provision is made for protocol operations that declare queues and exchanges, define bindings between them, subscribe to queues, and so on.
This gives application developers a lot of freedom but also requires them to be aware of potential definition conflicts. In practice, definition conflicts are rare and often indicate a misconfiguration.
Applications declare the AMQP entities that they need, define necessary routing schemes, and may choose to delete AMQP entities when they are no longer used.
Exchanges and Exchange Types
Exchanges are AMQP entities where messages are sent. Exchanges take a message and route it into zero or more queues. The routing algorithm used depends on the exchange types and binding rules. AMQP brokers provide four exchange types:
Direct exchange
Fanout exchange
Topic exchange
Headers exchange
Besides the exchange type, exchanges are declared with a number of attributes, the most important of which are:
Name
Durability (exchanges survive broker restart)
Auto-delete (exchange is deleted when the last queue is unbound from it)
Arguments (optional, used by plugins and broker-specific features)
Let us have a look at the different types of exchanges in the section below:
Direct Exchange
A direct exchange delivers messages to queues based on the message routing key. A direct exchange is ideal for the unicast routing of messages. Here is how it works:
A queue binds to the exchange with a routing key R
When a new message with routing key K arrives at the direct exchange, the exchange routes it to the queue if R = K
Direct exchanges are often used to distribute tasks between multiple workers (instances of the same application) in a round-robin manner. When doing so, it is important to understand that, in AMQP 0-9-1, messages are load-balanced between consumers and not between queues.
A direct exchange can be represented graphically as follows:
Fanout exchange
A fanout exchange routes messages to all of the queues that are bound to it and the routing key is ignored. If N queues are bound to a fanout exchange when a new message is published to that exchange a copy of the message is delivered to all N queues. Fanout exchanges are ideal for the broadcast routing of messages.
A fanout exchange can be represented graphically as follows:
Topic Exchange
Topic exchanges route messages to one or many queues based on matching between a message routing key and the pattern that was used to bind a queue to an exchange. The topic exchange type is often used to implement various publish/subscribe pattern variations. Topic exchanges are commonly used for the multicast routing of messages.
Topic exchanges have a very broad set of use cases. Whenever a problem involves multiple consumers/applications that selectively choose which type of messages they want to receive, the use of topic exchanges should be considered.
Headers Exchange
A header exchange is designed for routing on multiple attributes that are more easily expressed as message headers than a routing key. Header exchanges ignore the routing key attribute. Instead, the attributes used for routing are taken from the headers attribute. A message is considered matching if the value of the header equals the value specified upon binding.
It is possible to bind a queue to a headers exchange using more than one header for matching. In this case, the broker needs one more piece of information from the application developer, namely like should it consider messages with any of the headers matching or all of them?
This is what the "x-match" binding argument is for. When the "x-match" argument is set to "any", just one matching header value is sufficient. Alternatively, setting "x-match" to "all" mandates that all the values must match.
Note that headers beginning with the string x- will not be used to evaluate matches.
Queues
Queues in the AMQP model are very similar to queues in other messages and task-queueing systems: they store messages that are consumed by applications. Queues share some properties with exchanges, but also have some additional properties:
Name
Durable (the queue will survive a broker restart)
Exclusive (used by only one connection and the queue will be deleted when that connection closes)
Auto-delete (queue that has had at least one consumer is deleted when the last consumer unsubscribes)
Arguments (optional; used by plugins and broker-specific features such as message TTL, queue length limit, etc)
Before a queue can be used it has to be declared. Declaring a queue will cause it to be created if it does not already exist. The declaration will have no effect if the queue does already exist and its attributes are the same as those in the declaration.
Bindings
Bindings are rules that exchanges use (among other things) to route messages to queues. To instruct an exchange E to route messages to a queue Q, Q has to be bound to E. Bindings may have an optional routing key attribute used by some exchange types. The purpose of the routing key is to select certain messages published to an exchange to be routed to the bound queue. In other words, the routing key acts like a filter.
To draw an analogy:
Queue is like your destination in New York city
Exchange is like JFK airport
Bindings are routes from JFK to your destination. There can be zero or many ways to reach it
Having this layer of indirection enables routing scenarios that are impossible or very hard to implement using publishing directly to queues and also eliminates a certain amount of duplicated work application developers have to do.
If a message cannot be routed to any queue (for example, because there are no bindings for the exchange it was published to) it is either dropped or returned to the publisher, depending on the message attributes the publisher has set.
Consumers
Storing messages in queues is useless unless applications can consume them. In the AMQP Model, there are two ways for applications to do this:
Subscribe to have messages delivered to them ("push API"): this is the recommended option
Polling ("pull API"): this way is highly inefficient and should be avoided in most cases
With the "push API", applications have to indicate interest in consuming messages from a particular queue. When they do so, we say that they register a consumer or, simply put, subscribe to a queue. It is possible to have more than one consumer per queue or to register an exclusive consumer (excludes all other consumers from the queue while it is consuming).
Each consumer (subscription) has an identifier called a consumer tag. It can be used to unsubscribe from messages. Consumer tags are just strings.
Last updated
