Speeding Up Paging: TLB & EMAT

Why Is the TLB Essential? The Array Bottleneck

To understand why a TLB is critical to modern computer architecture, consider what happens during everyday programming routines—such as summing an array of 1,024 integers (4,096 bytes, exactly 1 standard 4 KB page):

int total = 0;
for (int i = 0; i < 1024; i++) {
    total += data[i]; // All 1,024 integers reside on the same 4 KB virtual page!
}
        

Assume a typical system: CPU execution core access takes 1 ns, but an off-chip DRAM lookup takes 50 ns. On a 64-bit architecture with 4-level paging (PML4 → PDPT → PD → PT):

Architecture WITHOUT a TLB: For every single element in the loop, the CPU must resolve the virtual address of data[i]. It reads Level 4 (50 ns), Level 3 (50 ns), Level 2 (50 ns), and Level 1 (50 ns), then finally reads data[i] itself (50 ns).
Cost per loop iteration: 250 ns.
Total cost for 1,024 integers: 1,024 × 250 ns = 256,000 ns (0.256 ms). The CPU spends 80% of its execution time stalled on redundant pointer reads for the exact same page!
Architecture WITH a TLB:
Iteration 0: Compulsory TLB Miss → Walk 4 levels of DRAM (200 ns) → Cache translation in TLB → Fetch data (50 ns) = 250 ns.
Iterations 1 to 1,023: TLB Hit! The associative register matches the Virtual Page Number in < 0.5 ns. The 4-level DRAM walk is bypassed completely! Fetch data from L1 cache (~1 ns) = ~1.5 ns per element.
Total cost for 1,024 integers: 250 ns + (1,023 × 1.5 ns) ≈ 1,784 ns (0.00178 ms).

Speedup = 256,000 ns / 1,784 ns ≈ 143x Faster!
Without a TLB, modern 3.0 GHz processors would spend nearly all their cycles waiting on memory bus lookups. The TLB cuts translation overhead to zero for >98% of instructions.

What is EMAT (Effective Memory Access Time)?

EMAT stands for Effective Memory Access Time. In plain English, it is the average time it takes for a CPU to access memory when you average out lightning-fast successes (TLB hits) and slow penalties (TLB misses).

To calculate EMAT, we look at three hardware variables:

α (Alpha): The Hit Ratio—the percentage of time the virtual address is successfully found in the TLB (e.g., 0.98 for 98%).
t_tlb: The time it takes to look up a translation in the TLB (e.g., 1 ns).
t_mem: The time it takes to access physical DRAM (e.g., 100 ns).

EMAT = α × (t_tlb + t_mem) + (1 - α) × (t_tlb + 2 × t_mem)

The Two Paths:
1. Hit Path (α): You check the TLB (t_tlb), find your match instantly, and read the data from DRAM (t_mem). Total cost = t_tlb + t_mem.
2. Miss Path (1 - α): You check the TLB (t_tlb) and miss, so you must walk the page table in DRAM (t_mem), and then read your actual data from DRAM (t_mem again). Total cost = t_tlb + 2 × t_mem.

Demonstration: 1,024-Element Array Traversal Race

Simulate running the array summation loop with vs. without a TLB in real time.

Without TLB (DRAM Walk Every Iteration) Idle

Elements Processed: 0 / 1024

DRAM Page Table Lookups: 0 reads

Elapsed Simulated Time: 0.00 μs

With Hardware TLB (Associative Cache) Idle

Elements Processed: 0 / 1024

DRAM Page Table Lookups: 0 reads

Elapsed Simulated Time: 0.00 μs

Textbook Diagram: Figure 3-12 (A TLB to speed up paging)

Figure 3-12. A typical 8-entry hardware TLB snapshot showing mapped virtual pages and execution permissions.

Part 1: Step 1 of 4 Interactive "Uh-Huh!" Walkthrough

1. The Cold Miss (Iteration 0)

When our array loop starts, the TLB has never seen Virtual Page 140. Click the button below to execute data[0] and watch the hardware experience a cold miss.

Part 2: Interactive TLB & EMAT Workbench

Edit entries below, step through instruction streams, or customize hardware speeds.

Hardware Instruction Stepper

Editable Hardware TLB Registers (Click to Edit)

Valid	Virtual Page (VPN)	Dirty	Perm	Frame (PFN)	Action

Hardware Timing Parameters

TLB Latency: DRAM Latency:

Live EMAT Performance Telemetry

Total Requests: 0

TLB Hits / Misses: 0 / 0

Hit Ratio (α): 0.0%

Effective Memory Access Time (EMAT): -- ns

MMU Hardware Trace Real-time log