Its two-layer buffer design prioritizes producer hot-path performance. Batch publication and transfer let producers use small buffers while consumers use larger buffers, reducing the effect of message transfer on business execution. Supports single or multiple producers and consumers. Currently requires C++20.
简体中文 · Full guide · API guide · Examples · Benchmarks
The queue has room and sends never block, so everything appears normal. Yet after sending a continuous batch, the same business code can take several times longer to execute. If the producer never waited, where did the extra time go?
Copying an 8 or 16 B message should be quick. The smaller the message, the more the other work involved in sending deserves attention. With the same amount of data and the same number of messages, how much can performance differ between individual and batch submission?
Consumption slows down, messages accumulate and the buffers eventually fill. Waiting interrupts business execution; dropping messages can lose important records. Could there be another place for pending messages while keeping memory use bounded?
Most records may be only a few dozen bytes, with an occasional snapshot, sample batch or data block measuring several MiB. A single send then writes a broad memory range. How can it transfer quickly while minimizing its effect on subsequent business work?
Small buffers may not cover bursts and can make the producer wait. Large buffers may retain memory for peaks that happen only occasionally. When normal traffic and peak traffic differ greatly, how can burst capacity and memory use be balanced?
These questions guide the design and continued improvement of TwoBufQueue. See Problems and solutions for the approaches and planned features. Continue below for the performance measurements.
The four charts compare producer submission throughput in million messages/s; higher is better. TwoBufQueue uses WriteSession; consumers run concurrently.
Full Session results · Ordinary write, batch publication · Ordinary write, per-record publication
Every queue follows the same sequence: normal business work → send a continuous message batch → immediately repeat the same business work. The chart shows post-send cost / normal cost; lower is better.
Normal business work costs about 13 ns/call on this host. After a 16 MiB batch, TwoBufQueue costs 18.2 ns/call, ReaderWriterQueue 58.2, rigtorp 42.5 and atomic_queue 99.8. Full experiment and parameters
These are separate experiments on Apple M3. Submission uses 256 MiB per TwoBufQueue layer and callback loop count 0. Business impact uses a 16 KiB producer pool, a 256 MiB consumer pool and callback loop count 64. Native queues use 256 MiB of message storage. Three-run means, high QoS and no hard CPU affinity; see the reports for timing scope and limitations.
If the questions in chapter 1 interest you, or you have encountered similar problems, consider TwoBufQueue.
In practical terms: the producer runs on the hot-path thread and the consumer runs on a supporting thread. You want the hot path to remain stable and efficient while letting the producer send messages at high speed, as in low-latency applications such as robotics, sensors and quantitative trading.
I will continue improving this component. You are welcome to share related requirements and ideas at any time. I will do my best to implement requests that are broadly useful to the community.
Two-layer transfer adds copying and background work. The single-layer queues mentioned earlier may be a better fit when individual send-to-receive time matters most, or when traffic and processing are light. Also, the producer and consumer threads must not be pinned to the same CPU core; otherwise, two-layer buffering effectively behaves like single-layer buffering.
Submit through a Writer and receive messages in the on_record callback.
#include <TwoBufQueue.hpp>
#include <cstdio>
#include <string_view>
int main() {
//---------------1. Initialization--------------------------------------
auto writer = TwoBufQueue::make_writer<TwoBufQueue::Writer<>>();
// Callback that receives each message.
auto on_record = [](std::int32_t, const std::byte* data,
std::size_t size) noexcept {
std::printf("%.*s\n", static_cast<int>(size),
reinterpret_cast<const char*>(data));
};
TwoBufQueue::TwoBufQueue queue(on_record, writer);
//---------------2. Producer writes-------------------------------------
bool ok = true;
for (std::string_view text : {"hello", "telemetry", "goodbye"})
ok &= writer.write_mem(text.data(), static_cast<std::uint16_t>(text.size()));
writer.flush(); // Publish and notify; does not wait for delivery.
//---------------3. Finish----------------------------------------------
writer.finish(); // Stop production and hand off the tail block.
queue.stop(); // Drain copying and callbacks before leaving this scope.
//---------------4. Check the result------------------------------------
return ok && queue.error_code() == TwoBufQueue::ReadError::none ? 0 : 1;
}Requires CMake 3.20+, a C++20 compiler and thread support. No third-party runtime dependency. macOS requires 14.4+ and a supporting SDK; detailed tests additionally require Python 3.9+.
cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --config Release --parallel
ctest --test-dir build -C Release --output-on-failureLink the CMake target TwoBufQueue::TwoBufQueue. Installation and integration · More examples
| What you need | Start here |
|---|---|
| Two-layer design, use cases and roadmap | Full guide README_DETAIL |
| Writing, publication, callbacks, threads and parameters | API guide |
| Session, multiple producers and synchronous callbacks | Examples |
| Measurement scope, parameters and reproduction | Benchmark methodology |
| Application logging through the queue | Async logging guide |
Apache-2.0. License · Contributing