The roofline model
Relate useful operations to the bytes needed to feed them.
A faster compute engine cannot help much if it spends its time waiting for bytes.
Useful counted operations.
Find the bottleneck
Low arithmetic intensity hits the memory ceiling. More data reuse can move the workload toward the compute ceiling.
What this model includes
An ideal upper bound with fixed peak compute and bandwidth. Ignores latency, overhead, cache-level traffic, and instruction mix.
What happens inside
Compute arithmetic intensity
Arithmetic intensity is useful operations divided by bytes moved at a specified memory level. The roofline bounds throughput by the lesser of peak compute and bandwidth times intensity. Define the operation format and memory boundary explicitly; DRAM intensity and L1 intensity are different quantities.
Move toward the useful ceiling
Tiling, fusion, and reuse can raise intensity by reducing traffic. Vectorization and suitable kernels can improve utilization of the compute ceiling. Real performance falls below the bound because of latency, instruction mix, synchronization, and imperfect access. The model helps classify a bottleneck, not predict an exact runtime.
throughput ≤ min(peak_compute, bandwidth × intensity)What this means for your code
Low-level engineer
Measure traffic and achieved operation rate. Use separate roofs for precisions and memory levels when the workload requires it.
Software developer
Eliminate redundant copies and intermediate materialization. Reusing data can help more than buying a device with a higher peak arithmetic number.
Read the actual specifications
These references supply the underlying contracts and implementation details. The diagrams here are simplified teaching models.