Title: RISC-V MICROARCHITECTURE EXPLORATION VIA REINFORCEMENT LEARNING. Abstract: Microarchitecture determines a processor's detailed structure, affecting the processor's performance, power, and area (PPA).
Deciding on a microarchitecture to achieve a good balance between the PPA values is a non-trivial problem.
Previous arts mainly require expert knowledge.
The solution becomes inefficient as nowadays processors become increasingly complicated.
Machine learning has solved problems automatically with high-quality results via reduced access to domain knowledge.
In this paper, we formulate the problem as a Markov decision process and propose an end-to-end solution framework via reinforcement learning.
Firstly, a dynamically-weighted reward design is proposed to accommodate the optimization of multiple negatively-correlated objectives.
Secondly, local heuristic search is adopted in the action design with prior knowledge of microarchitectures.
Thirdly, lightweight calibrated PPA models are incorporated to accelerate the learning process.
Experimenting with electronic design automation (EDA) tools on famous RISC-V processors demonstrate that our methodology can learn from experience and outperform human implementations and previous arts' solutions in PPA and overall running time.