Writing a simple async message queue server in C++, Python and JavaScript — Summary

Introduction

I started writing the asynchronous memory queue series of articles in 2023, 2 years ago. My original goal was to get back to doing some fun systems programming, especially in C++. I am currently a hands-off manager at work, but I like to write code on the weekends in an attempt to keep myself sharp. I was also curious how difficult it is to make the implementations wire-compatible. Finally, back in 2023 there was a lot of hype around Rust, so my plan was to write a version in Rust as well. As the hype around Rust (and Go) has decreased, so has my interest; another factor here has been C++'s continued dominance as the de facto standard systems programming language, and my excitement over new language features coming every 3 years. From a personal perspective, I have limited time to write the blog, and lots of interest, and Rust just kept getting deprioritized. These 2 years have also coincided with LLMs becoming useful coding assistants.

So far there have been 10 posts in this series:

• Python:
  • Writing a simple Python async message queue server - Part I
  • Writing a simple Python async message queue server - Part II
  • Writing a simple Python async message queue server - Part III
• JavaScript:
  • Writing a simple JavaScript async message queue server - Part I
  • Writing a simple JavaScript async message queue server - Part II
  • Writing a simple JavaScript async message queue server - Part III
• C++:
  • Writing a simple C++ async message queue server - Part I
  • Writing a simple C++ async message queue server - Part II
  • Writing a simple C++ async message queue server - Part III
• Unit tests:
  • Writing unit tests for the async message queue server

The code for all three language implementations is in this Github repo.

Takeaways

Overall, the cleanest and shortest implementation is in Python. In terms of lines of code (LOC):

The Python standard library is remarkably well designed and useful. Putting aside debugging and typing, the only imports needed were: sys, collections, asyncio, random, json, and no fundamental data structures needed to be implemented by hand. Compare this with JavaScript, where I had to implement a custom DefaultDict and Deque:

class DefaultDict {
    constructor(defaultInit) {
        return new Proxy({}, {
            get: (target, name) => name in target ?
                target[name] :
                (target[name] = typeof defaultInit === 'function' ?
                    defaultInit() : // Call the function directly
                    defaultInit)
        });
    }
}

class Deque extends Array {
    constructor(maxlen) {
        super();
        this.maxlen = maxlen;
    }

    push(...args) {
        if (args.length + this.length > this.maxlen) {
            this.splice(0, args.length + this.length - this.maxlen);
        }
        return super.push(...args);
    }
}

let topics = new DefaultDict(() => new Set());
let topics_reverse = new DefaultDict(() => new Set());
let caches = new DefaultDict(() => new Deque(cacheSize));
let indexes = new DefaultDict(() => 0);

C++ is a different level of abstraction and trade-offs, so a direct comparison is not really fair. The following #includes were required:

#include <boost/asio.hpp>
#include <boost/json.hpp>
#include <boost/locale/encoding_utf.hpp>
#include <deque>
#include <iostream>
#include <memory>
#include <random>
#include <set>
#include <string>
#include <unordered_map>

Overall, although modern C++ supports abstractions similar to languages like Python (auto keyword, for (auto& item: list) style iteration, etc.) and is remarkably nicer to use than 15 years ago (when I wrote ScalienDB in C++), in a production environment I would invest a lot of thought and time to try to keep as much of my code in higher-level languages like Python or JavaScript, and not C++. The raw low-level nature of C/C++ still leaks through, very quickly when trying to compile code. An example, above I wrote for (auto& item: list), this works if list is an std::vector<int>, but not if it's an std::vector<bool>; then it has to be for (auto&& item: list), and good luck figuring this out from the compiler error.

Another surprise for me was that the JavaScript version ended up being almost as many LOC as the C++ version. Having been a programmer since the 90s, like many people in my age group, I've written a lot of client-side JavaScript code over the years. I have also written server-side production Node code 10-15 years ago, but not recently. I was surprised that the language hasn't evolved to be more competitive with Python. I don't see why I would pick JavaScript over Python on the server-side, with clunky C/C++-style syntax, relative verbosity, and a smaller ecosystem than Python. Noisy code like the snippet below just doesn't feel competitive for a language similar to Python:

let topics = new DefaultDict(() => new Set());
let topics_reverse = new DefaultDict(() => new Set());
let caches = new DefaultDict(() => new Deque(cacheSize));
let indexes = new DefaultDict(() => 0);

I've also been pleasantly surprised that all 3 versions are able to handle 10,000 concurrent connections. This requirement used to be known as the 'C10k problem' 15-20 years ago, when it was not so trivial to achieve it — it even has a Wikipedia page: C10k problem. Back then many server implementations used threads for handling concurrent connections, and context switching between 10k threads often broke operating systems back then. With asynchronous, event-driven IO being conveniently available in all 3 languages, handling 10k connections now is easy even in interpreted languages.

Unit tests have been a great help, esp. to test and enforce wire-compatibility. Historically I've not always been a great proponent of writing tests, but with the advent of LLMs writing unit tests is a no-brainer.

Moving on to the topic of LLMs, as mentioned in the introduction, the 2 years of these articles coincided with LLMs becoming practically useful (from GPT-3.5 to o3), especially for programming. In my experience as a weekend programmer (and seeing individual contributors in teams I manage), LLMs are having a huge impact and influence on programming and programmers.

LLMs are trained on publicly available Internet data, so whatever language has more coverage online in terms of source code and Stack Overflow, the LLMs will be better at this language. So, LLMs will always be better Python programmers than D programmers. This will have a feedback effect, because people will notice which language their LLMs are better at, and gravitate to those languages. So I foresee the major languages like Python gaining even more market share as programmers increasing use LLMs in their workflow — especially new programmers.

LLMs have become part of a programmer's toolchain over the past 2 years — however, in my experience, even the latest LLMs such as o3 regularly make mistakes, also in small-scale situations, when dealing with 100s LOC. Blindly accepting the code that LLMs produce is not yet feasible — maybe in another couple of years. As an example, when writing the final version of the C++ code, I showed the Python version to o3, and asked it to give me the C++ version. It gave me mostly working C++ code, but with many features (command-line flags and logical features like controlling caching behaviour) missing. Having said that, LLMs have become magic for generating small, 10-100 LOC self-contained chunks of code like a function or a class that achieves something. I also found that LLMs are great at writing flat code without too much complexity, for example unit tests. This is one of the great ways to use LLMs "against each other" in a constructive way: use LLMs to build unit tests, also use LLMs to write the code itself, and see if the code passes the unit tests.

Conclusion

Writing these implementations (with the help of LLMs) has been a good learning experience, and a way to retain some of my programming sharpness. One follow-up project idea is to implement some core distributed algorithms like Paxos, Blockchain, BitTorrent, perhaps as part of a book, possibly in multiple languages.