When it comes to connectivity in IoT, does it make a big difference in performance and battery management to use MQTT instead of HTTP? This question has been bugging me for a while now, so I decided to test it out to see the difference...

The set-up

How to test

Firstly, I had to figure out a way to generate traffic and then measure it. Instead of using some fancy network profiling tool, I kept it simple by using Wireshark on my computer to monitor the outgoing traffic and a Python script to generate it. For the cloud part, I used AWS IoT Core, which is a managed MQTT broker that can also accept traffic via HTTPS (authentication works via Amazon's SigV4. Source.

The test code

I wrote a very basic Python script that takes a couple of arguments and generates messages with different sizes and sends them different counts. Be warned, this is not a production-grade code but feel free to use any part of it.

Monitoring

As mentioned earlier, I used Wireshark to monitor the traffic. After capturing the traffic, I exported it into CSV with the following headers:

"No.","Time","Source","Destination","Protocol","Length","Info"

The file name was the experiment number. Then I used another small Python script to get the information I need out of these files.

The results

I was rather surprised by the result! I expected that MQTT would be better under any circumstances. Well, I can confirm, it isn't. Let's see the request size and the response time (note: the response time was measured based on the time difference between the first packet and the last packet) when we send 1 message only. This behaviour is similar to basically sending 1 message every once in a while without any connection reuse. 

As you can see, HTTP has a somewhat smaller size and a lot quicker response time. This is likely to be because MQTT is quite heavy on headers at the initialization of the connection compared to HTTP. Once the connection has been established, MQTT is a lot lighter on the payload. This is notable as you increase the message count. For example for 10 messages and the same message size, I got the following results:

And if we increase the count to 100 and then to 1000, MQTT scales much better than HTTP:

I also undertook some research around message sizes, but that doesn't seem to be very conclusive. As you would expect, increasing the message size increases the overall size more for MQTT than for HTTP. This is also because of how the protocol was built. Larger message size increases the size relative to the overall message size more in MQTT than in HTTP:

If you are interested in the exact measurements, you can access them here.

TLDR

HTTP seems to have a ~10% (10,683B / 9,616B) smaller message size if you are only sending a single message every once in a while. However, as soon as you can reuse the connection for multiple messages MQTT proves to be drastically better. For 1000 messages, the message size for HTTP was ~2200% (415KB / 9,381KB) more. It seems that MQTT is the safest option and going for it can provide you with more reusable code across multiple applications unless you really need that extra 10% and you could not reuse your connection.