T01: Systems and performance engineering

A. Prerequisites and observable goals#

Level: Advanced-entry track. Suggested first specialization for engine-related work, not a permanent career assignment.

B. The central question: what resource actually limits this workload?#

Performance work begins with a defined task and a budget: frame time, request latency, throughput, memory capacity, energy, or recovery time. These objectives can conflict. Batching persistence may improve throughput but increase the amount of recent work waiting to be saved. Caching may reduce reads while consuming memory and creating invalidation complexity. A fast average does not establish an acceptable tail or deadline behavior.

Count useful work before choosing low-level tricks. Replacing a quadratic scan with an appropriate spatial index may remove far more work than vectorizing the scan. Once an algorithm is appropriate, representation, allocation, branching, synchronization, and data movement matter. A profiler can locate time or resource consumption; a microbenchmark can isolate one hypothesis. Neither automatically identifies the change that improves the whole application.

For an engine, a useful question is not “is ECS faster?” but “which components does this update read and write, how many entities are active, how are they laid out, and which other systems share their lifetime?” An architecture name does not determine cache traffic or synchronization cost. J05 applies this habit to an immutable spatial-query snapshot with a brute-force oracle.

C. Layout changes what the machine must fetch#

An array of structures (AoS) places all fields of one entity together. A structure of arrays (SoA) places one field from many entities together. Suppose a particle contains eight 4-byte floating-point fields, but one kernel reads only velocity $v_x$ and updates position $x$. The logical useful data per entity is one velocity read plus one position read and write. AoS may fetch cache lines containing unused fields; SoA can make the needed arrays contiguous.

AoS can be attractive when most fields of a small number of entities are used together. SoA can help streaming kernels over a few fields and may facilitate vectorization. Hybrid blocked layouts can balance concerns. The correct choice depends on access patterns, not a universal superiority rule. Alignment and padding change actual object sizes; measure sizeof and inspect a target rather than calculating layout solely from field names.

Allocation also has a lifetime dimension. Reusing a scratch buffer can reduce allocator work when capacity and ownership are controlled. An arena can free a group of objects together, but pointers into it become invalid when it resets. Pooling without a clear lifetime protocol can turn a performance optimization into use-after-free or stale-handle bugs. A generation counter can detect some stale handles, but wraparound, reuse, and concurrency still require a contract.

D. Reason about bottlenecks and synchronization#

A lower-bound style model compares compute work and memory traffic. If a kernel needs $F$ operations and moves $B$ bytes, its time cannot realistically beat both the machine’s compute and bandwidth constraints; a rough model is $T\gtrsim\max(F/P,B/W)$ for effective compute rate $P$ and bandwidth $W$. Those effective rates are measured or bounded quantities, not marketing peak numbers. Cache reuse, write allocation, vectorization, and dependencies can change the relevant traffic and attainable rates.

Parallelism adds coordination. A shared atomic counter updated for every entity can serialize otherwise independent work. False sharing occurs when independent variables share a coherence unit and different cores repeatedly write them, causing coherence traffic despite no logical data dependence. Separating hot counters or batching updates may help, but alignment padding also consumes memory and depends on the target. A data race is a correctness violation; false sharing is a performance interaction between otherwise properly synchronized or distinct data.

Work partitioning must account for load balance and overhead. Static equal-size ranges are simple for uniform work; a bounded work queue can balance variable jobs but adds synchronization. Immutable snapshots allow readers to avoid mutation races while paying build and publication costs. A snapshot is not free: memory can temporarily contain both old and new versions, and readers must keep the version they use alive.

E. A reproducible experiment can reject your favorite hypothesis#

The lab updates $x\gets x+v_x\Delta t$ for $200{,}000$ particles, 20 passes per timed invocation, using AoS and SoA representations. It allocates and initializes outside the timed region, performs one excluded warm-up for each, alternates order over seven repetitions, and validates matching final checksums. The object size reported in this Linux/GCC build is 32 bytes.

The reference run recorded AoS median $0.719918$ ms and SoA median $0.718477$ ms. These are nearly equal in this small shared-container experiment. They do not support a dramatic SoA speedup. The compiler may transform loops, the working set and cache hierarchy matter, and a single environment does not isolate all effects. The appropriate conclusion is that this experiment did not demonstrate a practically large difference, not that layout never matters. Exact commands and environment are in the verification ledger.

A more complete investigation would vary entity count, fields touched, pass structure, and thread count; inspect generated code; collect hardware counters where permitted; and test representative application traces. Those are continuation experiments, not measurements claimed to have been performed here. Keep the original baseline and validate every modified implementation before trusting its timing.

F. Worked budget and system-level interpretation#

Suppose a frame takes $16$ ms: $9$ ms rendering, $4$ ms simulation, and $3$ ms other work. A verified optimization halves only simulation, yielding $9+2+3=14$ ms, a speedup of $16/14\approx1.143$, not $2$. The local improvement is real but bounded by the untouched work. If the optimization also introduces a $3$ ms synchronization stall, the frame worsens to $17$ ms. Measurements must include the actual integration boundary.

For a service, tail behavior can matter more than average work. A queue that is stable under ordinary load can grow during bursts; bounded capacity makes overload explicit. Reliability work includes timeouts, cancellation, graceful shutdown, recovery tests, and observability. A fast path that corrupts state on shutdown is not a successful optimization. Review C10 closure and J03 transaction recovery before optimizing either.

G. Independent practice#

F foundation C standard course work T transfer S optional stretch L runnable lab or project. Attempt the complete set before opening its solutions.

H. Complete solutions: T01#

Compare the first step where your work differs. A corrected answer is not yet an independent success.

Open every solution

#include <algorithm>
#include <chrono>
#include <cmath>
#include <cstddef>
#include <iostream>
#include <numeric>
#include <vector>
struct Particle { float x,y,z,vx,vy,vz,mass,spare; };
using Clock = std::chrono::steady_clock;
template<class F> double timed(F f) {
    const auto start = Clock::now();
    f();
    return std::chrono::duration<double,std::milli>(Clock::now()-start).count();
}
double median(std::vector<double> samples) {
    std::sort(samples.begin(),samples.end());
    return samples[samples.size()/2];
}
int main() {
    constexpr std::size_t n=200000;
    constexpr int passes=20, repetitions=7;
    std::vector<Particle> aos(n);
    std::vector<float> x(n), vx(n);
    for (std::size_t i=0;i<n;++i) {
        aos[i]={static_cast<float>(i%1024),0,0,0.25F,0,0,1,0};
        x[i]=aos[i].x; vx[i]=aos[i].vx;
    }
    auto a=[&] { for(int p=0;p<passes;++p) for(auto& e:aos) e.x+=e.vx*0.01F; };
    auto b=[&] { for(int p=0;p<passes;++p) for(std::size_t i=0;i<n;++i) x[i]+=vx[i]*0.01F; };
    a(); b(); // warm-up excluded
    std::vector<double> ta,tb;
    for(int r=0;r<repetitions;++r) {
        if(r%2==0) { ta.push_back(timed(a)); tb.push_back(timed(b)); }
        else { tb.push_back(timed(b)); ta.push_back(timed(a)); }
    }
    double ca=0,cb=0;
    for(std::size_t i=0;i<n;++i) { ca+=aos[i].x; cb+=x[i]; }
    if(ca!=cb) { std::cerr<<"checksum mismatch\n"; return 1; }
    std::cout<<"n="<<n<<" passes="<<passes<<" repeats="<<repetitions
             <<" sizeof_particle="<<sizeof(Particle)<<"\n"
             <<"aos_median_ms="<<median(ta)<<" soa_median_ms="<<median(tb)
             <<" checksum="<<ca<<"\n";
}

I. Retention and readiness#

Retrieve C09 locality, C10 ownership and synchronization, and A04 uncertainty. Readiness requires independently diagnosing one bottleneck and defending a reproducible comparison, including a result that fails to support your initial idea. Continue through J05, then deeper C++ object-model study, OS internals, profiling, allocators, storage/I/O, and production reliability under mentorship or substantial project review.