How a 5-stage pipeline works

In a single-cycle processor each instruction runs entirely in one clock cycle, so the cycle must be as long as the slowest instruction. Pipelining splits that work into stages separated by registers, so that while one instruction is in EX the next one is already being decoded and another is being fetched from memory.

Latency and throughput

Two measures must be kept apart. Latency is how long one instruction takes from fetch until it writes its result: in a 5-stage pipeline that is 5 cycles, and it does not improve. Throughput is how many instructions finish per unit of time: in the ideal case, one per cycle. Pipelining improves throughput, not latency.

How many cycles a sequence takes

The first instruction needs 5 cycles to go through the stages; after that one finishes every cycle. For N instructions and k stages, without hazards:

cycles = k + (N − 1) = N + 4   (k = 5)

With N = 5 that gives 9 cycles, as in the diagram. The CPI (cycles per instruction) of this sequence is 9 / 5 = 1.8, but it approaches 1 as N grows, because the 4 fill cycles are shared among more instructions.

Why the speedup does not reach 5

Assume these delays for each stage (they are example values, not those of a specific processor):

The single-cycle processor needs an 850 ps cycle. In the pipelined one, the cycle is set by the slowest stage: 200 ps. For one million instructions, the single-cycle design takes 850 µs and the pipelined one about 200 µs (1,000,004 cycles × 200 ps). The improvement is 850 / 200 = 4.25, not 5, because the stages are not balanced: ID and WB waste part of their cycle. In practice the registers between stages also add a small delay to every cycle.

What each stage needs

• IF reads the instruction at the PC and computes PC + 4.
• ID decodes the fields and reads rs1 and rs2 from the register file.
• EX performs the ALU operation, computes a memory address or evaluates a branch condition.
• MEM reads or writes data memory (loads and stores only).
• WB writes the result into rd.

Everything an instruction needs in later stages (the register values it read, the number of rd, the ALU result…) travels with it in the pipeline registers IF/ID, ID/EX, EX/MEM and MEM/WB.

What breaks the ideal rhythm

The pipeline only finishes one instruction per cycle if nothing forces it to stop. There are three kinds of hazard:

• Structural: two stages need the same resource in the same cycle. With separate instruction and data memories, as the simulator assumes, they do not happen.
• Data: an instruction needs a result the previous one has not written yet. They are covered in the RAW hazards guide.
• Control: after a branch, the pipeline has already fetched instructions that may not have to run.

This is the cost of a branch in the simulator’s model, which resolves branches in EX and treats them as taken:

When bne reaches EX, the next two instructions are already in ID and IF and are discarded (FLUSH): 4 instructions that should take 8 cycles take 10. Real processors reduce this penalty by resolving the branch earlier or by predicting whether it will be taken.