MassTransit Send vs. Publish

MassTransit is capable of a lot, and one important feature is Send vs Publish. If you read the official documentation section “creating a contract“, it talks about commands (send) and events (publish). This makes sense, because commands you want to only be executed once, however events can be observed by one or more listeners.

This tutorial will look at what is happening with the two different message types. To follow along, download the completed project here. The only thing you will need to do is create a virtual host and user in RabbitMQ. Other tutorials for MassTransit typically use the default RabbitMQ guest/guest account, but I prefer to separate mine into virtual hosts. So open the solution and look in either of the three services App.config or the Mvc Web.config. You’ll see:

<add key="RabbitMQHost" value="rabbitmq://localhost/tutorial-send-vs-publish" />
<add key="RabbitMQHostUsername" value="tutorial-send-vs-publish" />
<add key="RabbitMQHostPassword" value="chooseapassword" />

So browse to http://localhost:15672, login as admin and configure. I make the virtual host and username the same. Then choose a password you want. After it’s setup run the solution and try it out!

Tutorial

I started with the Mvc StarterKit and created 3 services. One service is for the events (in practice it could be 1..n). The other two services for commands have artificially fast and slow processing speeds. This was done to illustrate that although X quantity of commands are queued, they are only processed once (well, once or more to be honest, as RabbitMQ is a ‘at least once’ message broker).

Note #1: Commands and events are both usually defined as interfaces. Therefore using descriptive names helps discern commands from events.

Publish (Events)

Lets use a post office and mailbox to help describe publish. An event in MassTransit is akin to flyer’s that we receive in the mail. They aren’t directly addressed to us, but everybody who has a mailbox gets them. So flyer = event, and mailbox = endpoint. When building your bus and registering an endpoint like so: sbc.ReceiveEndpoint(…), one has to be sure that the queueName parameter is unique. This is emphasized in the “common gotcha’s” part of the documentation. Now although it’s called queueName, it actually makes an Exchange and Queue pair, both named after the queueName parameter.

The below diagram shows the MyEvent flow in the sample project.

send-vs-publish_2r2

Although the sample doesn’t have multiple event services, I added my_events[N] just to show you each new service (and receive endpoint) will have a corresponding exchange+queue pair made.

Note #2: Now when I’ve used MassTransit and RabbitMQ, I’ve never worked with high traffic projects that result with queue or service bottlenecks. I believe RabbitMQ can be thrown quite a lot and keep up, but if you are encountering thresholds for RabbitMQ or the service, then you will need to think about RabbitMQ clustering, or adding more services (perhaps even scaling out services). But these are highly specialized and more advanced topics. My advice would be to use the MassTransit and RabbitMQ community discussion boards or mailing lists.

Send (Commands)

Commands should only be executed once (although you should be aware if making commands idempotent, but we will not cover this more complex topic in this tutorial). So in our scenario above (publish), if we have multiple services (ie. multiple endpoints) for the same contract type, we will have a single command being executed by many consumers. So rather than publish, we send a command to a specific queue+exchange pair. Then, our services can connect to the same receive endpoint’s queueName so long as they have the same consumers registered. I’m paraphrasing this important point from the documentation.

Here’s the MyCommand flow in the sample project.

send-vs-publish_2

The one main difference is the Message is sent directly to the my_commands endpoint exchange, rather than the MyCommand contract exchange. This avoids the contract exchange fan-out. In the sample, we also chose our services to subscribe to the same endpoint (as I made sure they both had the same consumers registered). I cheated a little and added a different sleep time to each consumer.

Code Comparison

To publish:

await _bus.Publish<MyEvent>(...);

Pretty simple, nothing to explain here

To send:

var sendEndpoint = await _bus.GetSendEndpoint(new Uri(ConfigurationManager.AppSettings["MyCommandQueueFullUri"]));
await sendEndpoint.Send<MyCommand>(...);

The big difference, we have to get our send endpoint first by passing in the Full Uri.

I hope this has helped clarify commands vs events, and when to use each. The next post I have planned will look at Transient vs Persistent exchanges+queues.

Starter Kit for ASP.NET MVC, MassTransit, and Autofac


Chris has done a great job providing Samples for simple and complex use cases with MassTransit. I recommend looking at the Shopping Cart sample and my complimentary post.

I want to write more SOA+MassTransit tutorials covering a wide range of scenarios for scalable and robust web applications, but I need a good jumping off point for the reader and myself to start from. So I’ll be making some Starter Kits available on Github to help provide an equal and familiar starting point for my tutorials. This is a learning process for both you and me, and I want to give the best and most accurate advice.

I like to construct my projects roughly following Jeffery Palermo’s Onion Architecture. The onion architecture might seem like overkill to use for the simple example of the Starter Kit. However I will be writing more tutorials, and when they get more complex, this organizational architecture will help keep things concise, and should (I hope) make it easier for the reader (you) to follow along.

Onion Architecture Approach (Quickly Explained)

starterkit-onion-architecture_1

One important concept is that layers can only have dependencies on their current layer, or more Inner Layers (or external nuget dependencies if necessary). So for example the Contracts and Domain Models should really only reference .NET Framework, nothing else.

Centre: This layer will typically contain our Contracts (Interfaces) and Domain Models or POCO (for ORM). There should be no business logic or type verification. This should be done in another (more outer) layer.

Middle: This layer houses services that contain implementation for our contracts. This layer could be further split into Domain Services and Application Services, but for simplicity we will just leave them as the Middle layer for now. I’ve provided some examples in the diagram such as; MassTransit Consumers, SignalR Hubs, CQRS services (eg. something like Mediatr).

Outer: The outer most layer typically has a variety of items, which end up tying together the services with concrete assemblies and infrastructure. For example, MassTransit (and projects I’ve worked on in the past) use a logging mechanism called LibLog. I am a huge fan of this, because it doesn’t restrict you to a specific logging implementation. So in the outer most layer, all you have to do is add your dependency to NLog, EntLib, or log4net, and configure it (using code or app.config), and your off to the races. None of your inner layers have any dependency on any logging system (LibLog works by simply adding a *.cs file to your projects). The same idea applies for other infrastructure you wire up such as; what ORM you use, the Storage type, the DB provider, the unit testing (and mocking) libraries, which CQRS pattern, or what logging framework to use.

Starter Kit – Autofac, MassTransit, Mvc

The Starter Kit can be found here, and below is the architecture.

Update Oct. 24, 2015: I’ve added another starter kit here, which includes SignalR.

starterkit-architecture_2

The Starter Kit compiles and runs a simple message (publish and subscribe) example. You can see the message sent from the webpage in the console service running.

Logging Explained

I want to take a moment to explain the logging option used. I’ve been around the block when it comes to logging for .NET, and by far the best technique I’ve encountered so far is LibLog. It lets you choose your favorite, and there is no DLL dependency in your libraries. Only the final executable that wires up the logger of your choice (log4net, nlog, entlib, etc…). So in the Starter Kit, I chose log4net, and if you look in StarterKit.Web (Web.config), or StarterKit.Service (App.config), you will see log4net’s configuration.

Logging Level in StarterKit

Because MassTransit, and Topshelf both use LibLog, our logs can become crowded if we set the global log level to debug. So you’ll notice in the *.config, that I set the root level to info and I set the StarterKit.* namespace level to debug.

<root>
  <level value="INFO" />
  <appender-ref ref="RollingFile" />
</root>
<logger name="StarterKit.*">
  <level value="DEBUG" />
</logger>

This keeps the logs clean if you are trying to debug your code. And of course, if you are having trouble with MassTransit, and need to post logs in the google discuss group to get help, the library contributors might ask for logs so they can help diagnose your issue. In that case you would want to change the root level to debug and run + capture the logs.

So go clone the Starter Kit and start experimenting!