Logging system

Authors: Xein

Created on: 26 Mar 2026

The logging system provides an easy way to write logs and visualize them after the program execution has finished. It contains a C++ implementation for producing messages, a Rust server for receiving, processing and storing messages, and a Rust TUI for easily reading and searching for information after reading.

To use this system you can simply just include <Toast/Log.hpp> and call any of the toast macros:

TOAST_TRACE(TestClass, "This is a regular debug message, you can also use fmt like this: {}", count);
TOAST_INFO(TestClass, "This is an informational message");
TOAST_WARN(TestClass, "This is a warning message");
TOAST_ERROR(TestClass, "This is an error message");
TOAST_CRITICAL(TestClass, "Oh no... something exploded");

This is a quick guide of how to chose the severity of your messages:

• Trace: Your regular debug message, most of the stuff you will log will be this type. An example of this is loading a texture or sending an event.
• Info: Something that is of great importance just happened, marks a before and after on the state of the engine. An example of this is loading a scene or changing an input layout.
• Warning: Suggests a possible missuse of a system or unwanted behaviour. An example of this is a find function not finding anything and returning a nullptr
• Error: Indicates something has failed on code execution, the engine is considered to be in an undefined behaviour after this. An example of this is failing to read a file from the disk.
• Critical error: Indicates an error that we cannot recover from has just happened. The program will abort its execution right after calling a critical error. An example of this is not being able to create a window.

Motivation

This system was created to address two issues that existed on Toast Engine v1 regarding the logging system:

• On a large project, searching for a message on the log trace was messy as a lot of systems were writing a lot of messages to the console
• The previous system was not crash-ready since an unexpected termination would result in the log file not being written and valuable information would get lost as soon as you closed the terminal that ran the application

With this in mind, I took the approach of separating the log parser logic into a different process so that when the engine decides to terminate ungratefully, we could detect that event and handle it acordingly. This also liberates the main thread for having to parse or evaluate any logging logic. I decided to handle inter-process communication as a TCP/IP connection since it could be easily asyncronous, would allow a lot of space for our log reader implementation, and would allow me to learn about networking.

A naïve implementation of sending a log to our server could be like this:

void Logger::log(std::string_view message) {
    // get our logger instance
    auto& logger = get();
 
    // create a new log and assign the information
    logging::LogData log;
    log.set_message(message);
 
    // send the log through TCP/IP
    toast::ThreadPool::queueJob([&logger, log]() {
        std::vector<uint8_t> buffer(log.ByteSizeLong());
        log.SerializeToArray(log.data(), log.size());
        asio::write(m.socket, asio::buffer(buffer));
    });
}

where we just send the information every time we want to send a message. However, this will get convoluted as soon as the information we send on a message increases. We could create a struct that contains the information about our log, but the moment we need to introduce interoperability or support multiple compilers, memory alignment could be a potential issue. I fixed this with the addition of a protocol buffer that we can then just convert into a byte stream and send to our server to decode it. In the example above, logging::LogData is not a trivial struct but one that has been autogenerated by a protocol buffer, and the function SerializeToArray() (implemented by the protobuf autogeneration) just converts our message to a stream of bytes.

This method, however, comes with a drawback: big log messages could - potentially - saturate our TCP connection and stall all the threads in our pool. In order to fix this issue, we enqueue the logs rather than send them as soon as they get created, and send a batch of messages and we send all of them as once as soon as the TCP connection is idle. Our implementation of the log function would then look like this:

void Logger::log(std::string_view file, unsigned line, char severity, std::string_view sink, std::string_view message) {
    // get our logger instance
    auto& logger = get();
 
    // create a new log and assign the information
    logging::LogData log;
    log.set_timestamp(std::chrono::duration_cast<std::chrono::nanoseconds>(
        std::chrono::system_clock::now().time_since_epoch()).count());
    log.set_filepath(file);
    log.set_line_number(line);
    log.set_severity(static_cast<logging::LogData_Severity>(severity));
    log.set_sink(sink);
    log.set_message(message);
 
    // add the event to a queue
    {
        std::lock_guard lock(logger.m.queue_mutex);
        logger.m.log_queue.emplace_back(std::move(log));
    }
 
    // tell the engine that the queue needs processing
    // We use an atomic exchange to claim a "drain slot"
    // This ensures that even if 100 threads log at once, only one background task is queued
    if (!logger.m.drain_pending.exchange(true)) {
        toast::ThreadPool::queueJob([&logger]() { logger.drain(); });
    }
}

The implementation of drain() is trivial so it is left to the reader's imagination. Notice as well the std::string_view sink parameter, which just stores information about which class created the log. This will be used for easy filtering later on.

With this, we already have the messages on our server and saving to a CSV is implemented by using a simple library - there is no need to reinvent the wheel - when the TCP connection gets terminated.

While until now I have focused on our second motivation, this implementation also allows for an easy fix of our first concern by creating a client application that connects to our server. On connection, we will first send a batch of all the messages received until that point - easily done since I am already storing every message to serialize it to a CSV file on disconnect -, and we can then send the rest of the messages as soon as received. We do not really care about saturating the send thread of our logging server, since delaying the appearance of our logs in the TUI is not a big concern. This can be implemented as:

async fn handleTuiConnection(socket: &mut TcpStream, storage: LogStorage) -> Result<(), Box<dyn std::error::Error>> {
    // send initial batch
    let existing_logs = storage.get_all().await;
    let batch = LogBatch { logs: existing_logs };
    let mut buf = Vec::new();
    batch.encode(&mut buf)?;
    send_framed(socket, &batch).await?;
 
    // subscribe to new logs
    let mut rx = storage.subscribe();
    loop {
        match rx.recv().await {
            Ok(log) => {
                let update_batch = LogBatch { logs: vec![log] };
                send_framed(socket, &update_batch).await?;
            }
            Err(broadcast::error::RecvError::Lagged(_)) => {
                // handle lag
            }
            Err(broadcast::error::RecvError::Closed) => {
                break;
            }
        }
    }
    Ok(())
}

As the TUI now has all the logs stored in a list, we can now just create a frontend to the information we have received. This are the base features I implemented:

• Displaying all the messages on a scrollable table, ordered by timestamp and changing the text colors for easy recognition of different severities
• Filtering by severity and sink (toggle on and off the display of certain messages)
• Search anywhere on the table by any information showcased on the screen
• Load a CSV file to check logs offline
• Connecting to an engine in another computer (as long as the other computer has opened their 12801 port, trivial to implement since we are already using TCP for communication)

By using this tool, you can much easily read logs and understand what is happening during runtime.

Uses

Initialization

The Logger can easily be initialized by using the create() function. Any log done after initialization will just work.

#include <Toast/Logger.hpp> // the logger class is private and is not exposed outside
 
std::unique_ptr<logging::Logger> logger;
 
void myInitFunction() {
    // you will be responsible for the lifespan of the logger after creating it
    logger = logging::Logger::create();
}

Note that the program will terminate if the logger is unable to create or connect to a server, as you will never want to use an application wihout a logging system.

Logging a message

As described above, the following macros are available for logging:

#define TOAST_TRACE(sink, ...)
#define TOAST_INFO(sink, ...)
#define TOAST_WARN(sink, ...)
#define TOAST_ERROR(sink, ...)
#define TOAST_CRITICAL(sink, ...)

where sink is a valid alias type and the variadic ... is the fmt formatted message. The macro will automatically fill the rest of the parameters required on Logger::log()

In addition to this, the following macro replacing std::assert() exists

#define TOAST_ASSERT(condition, sink, ...)

where condition is an evaluation that returns a bool and the assert will trigger only when that condition is not met. Think about it as you are declaring that not fulfilling the condition is impossible and you must terminate the program in the case that happens. This is why TOAST_ASSERT() - following the same route as std::assert() - will not be evaluated in Release builds, as an assert condition failing is considered erroneous behaviour and is optimized away.

Using Kenzo

Kenzo is the official TUI used to read the messages from the log server. It is named after one of my cats. You are allowed to ask for a picture of Kenzo if you are curious. The program will automatically try to connect to localhost by default. You can also connect to a different IP or read from a CSV file if the connection is not established. You can terminate your current connection at any moment by pressing q. Pressing q again on the screen will close Kenzo.

On the program, you can use hjkl to move around and shift + hjkl to change the current window. The program mostly follows vim motions so you can, for example, expect that / is used for searching and n and p to move across selections. Pressing f will open the filters panel, where you can toggle visibility by sink or by severity. In addition to this, you can also use the shorcuts 12345 to toggle severities without having to open the filters panel. You can use s to lock or unlock the automatic scrolling of the logs. Note that when unlocking the scroll, you will automatically select the latest message.

For easily opening CSV files for offline reading, you can use the --csv [path] flag when opening Kenzo.

Performance

Surprisingly, the logger is more performant than std::println() added on C++23, with the following times when stress tested:

Count: 1,000
  Toast Logger:       4ms
  std::println:       5ms
Count: 100,000
  Toast Logger:     361ms
  std::println:     810ms
Count: 10,000,000
  Toast Logger:   24034ms
  std::println:   26812ms

My guess is that this happens because the actual processing of the log, and writing it to the buffer - which is the most expensive part of logging anyways - is happening server-side, so the game thread doesn't have to do all of that computation.

Anyways, this is the code I used for the stress test:

auto run_bench = [](int count) -> std::pair<size_t, size_t> {
    auto start = std::chrono::high_resolution_clock::now();
    for (int i = 0; i < count; ++i) {
        TOAST_TRACE(BenchSink, "Benchmark log message {}", i);
    }
    auto end = std::chrono::high_resolution_clock::now();
    auto toast_duration = std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count();
 
    start = std::chrono::high_resolution_clock::now();
    for (int i = 0; i < count; ++i) {
        std::println("Benchmark log message {}", i);
    }
    end = std::chrono::high_resolution_clock::now();
    auto std_duration = std::chrono::duration_cast<std::chrono::milliseconds>(end - start).count();
 
    return {toast_duration, std_duration};
};
 
auto small = run_bench(1000);
auto medium = run_bench(100000);
auto big = run_bench(10000000);
 
std::cout << "Count: 1,000" << "\n";
std::cout << "  Toast: " << small.first << "ms\n";
std::cout << "  std::println: " << small.second << "ms\n";
std::cout << "Count: 100,000" << "\n";
std::cout << "  Toast: " << medium.first << "ms\n";
std::cout << "  std::println: " << medium.second << "ms\n";
std::cout << "Count: 10,000,000" << "\n";
std::cout << "  Toast: " << big.first << "ms\n";
std::cout << "  std::println: " << big.second << "ms\n";

As a funny note, the CSV file generated after the session was over 800MiB - yummy

[xein@kaveh] ~/code/engine/logs > ls -l
.rw-r--r-- 807M xein 26 Mar 13:37  2026-03-26_13-37-26.csv

Expansion

One of the implementation I will do at some point is create another client other than Kenzo, for example, a "Logs" window inside the editor.

This can easily be created by just connecting to localhost:12801 and implementing, at minimum, the following features:

• Displaying all messages, being able to see at least the message, sink, severity and timestamp
• Being able to filter based on sink
• Being able to filter based on severity

Any program that fulfills those requirements is a valid client for the Logging server