PEARL: Credit-Aware Minimal-Adaptive Routing for Mesh NoC

We present PEARL (Policy Extracted via Adaptive Reinforcement Learning), a credit-aware adaptive routing sub-policy for mesh-based Networks-on-Chip (NoC). PEARL is framed as a controlled ablation study that explicitly isolates which architectural choice contributes most to the gains attributed to “deep RL for NoC routing.” The central technical contribution is a simple rule: among productive output ports (those that reduce Manhattan distance to the destination), select the one with the most downstream space (lowest used credit count). PEARL is a one-line modification to the classical Min-Adaptive deadlock-free routing function and requires no hardware support beyond the existing IQRouter credit interface.

We discovered PEARL by training a Dueling Double Deep Q-Network (D3QN) agent with a 20-dimensional mesh-size-invariant state representation; the agent’s learned action converged to this credit-aware tie-break. We then evaluate PEARL standalone (no DRL inference at runtime) against five classical NoC baselines (XY/DOR, Odd-Even, DyAD, Min-Adaptive, and Q-routing HARAQ) and the original D3QN agent itself, on BookSim 2.0 across 1,575 BookSim runs total.

Findings. (i) PEARL beats the standard Min-Adaptive in 26/45 cells (58%, paired t-test), with significant gains on transpose traffic at moderate injection rates (−79%, −69%, −56% on 4×4, 8×8, 16×16 meshes). (ii) Across the seven-method comparison, the rule appears in the tied-best group on 12/45 cells and the full D3QN agent on 14/45; in 12/12 rule-tied cells the agent has converged to the rule and the two produce identical latencies. (iii) Head-to-head, the rule beats the agent on 10/45 cells and the agent beats the rule on 7/45 (rest tied within 0.5 cycle), so the agent’s exploration noise on net hurts more than it helps. The takeaway: the rule is the contribution, DRL a discovery tool.

Limitations. On hotspot traffic at low injection rate on 8×8 and 16×16 meshes, classical adaptive baselines retain a 10–24% advantage. We discuss this as a structural limit of PEARL when downstream credit alone is an incomplete proxy for hotspot avoidance.