The Policy-gradient Placement and Generative Routing Neural Networks for Chip Design

Ruoyu ChengXianglong LyuYang LiJunjie YeJianye HaoJunchi Yan

article2022NeurIPS59 citations

As modern microchips grow increasingly complex and dense, automated physical design faces major bottlenecks. Arranging circuit components (placement) and wiring their connections (routing) are critical steps to prevent congestion and minimize total wirelength. However, traditional heuristic software tools and emerging machine learning methods struggle with real-world complexities. Specifically, existing learning-based placers oversimplify varied component dimensions, causing severe overlaps, while learning-based routers connect points sequentially, leading to high computational costs and scalability issues.

The article develops and evaluates a fully learnable, end-to-end artificial intelligence pipeline—termed PRNet—for joint component placement and routing. The objective is to demonstrate that a pure neural framework can handle mixed-size components, dynamically learn the order in which connections are routed, and generate global routes in one shot without relying on traditional heuristic solvers.

To achieve this, the authors designed a reinforcement learning placement model that incorporates actual component dimensions and fixed initial layouts. They combined this with an image-based conditional generative network featuring an input-adapting architecture, an enhanced loss function, and dual evaluators targeting path connectivity and visual realism. The framework also uses an adaptive reinforcement learning agent to dynamically determine routing sequence. The system was validated against standard industry benchmarks, including ISPD-2005 for placement and ISPD-98 and ISPD-07 for routing, using hundreds of thousands of net instances on high-performance graphics processing units.

The evaluation produced four key findings. First, on mixed-size placement benchmarks, the model reduced component overlap area roughly fourfold (about 75%) compared to the leading learning baseline DeepPlace, while maintaining nearly identical wirelengths (within 1.3%). Second, the one-shot generative routing model achieved roughly double the correctness rate and up to a 14.7% reduction in wirelength over prior generative routers. Third, dynamically learning the net routing order substantially reduced routing congestion across full circuit benchmarks compared to static heuristics. Finally, while the sequential routing model achieved zero overflow on standard test circuits, running nets concurrently accelerated throughput by roughly 3.4 to 8 times at the cost of modest increases in wirelength and overflow.

These findings demonstrate that replacing rule-based physical design tools with an integrated, pure deep learning pipeline is feasible and effective. Drastically reducing placement overlaps eliminates costly post-processing fixes, while dynamic routing ordering lowers congestion and improves routability. However, because the sequential neural router is computationally slower than highly tuned, traditional rule-based tools, deploying such models requires balancing execution speed against optimization quality.

Organizations evaluating this approach should consider a hybrid deployment: using concurrent batching when rapid design iterations are required, and sequential execution for final, high-precision layout optimization. Before moving to production environments, development teams should pilot unsupervised or semi-supervised training pipelines to remove reliance on traditional tools for training data, and extend the model from global routing to detailed routing constraints.

The findings are supported by consistent results across recognized public benchmarks, though confidence should be weighed against notable boundaries. The neural router's performance ceiling is currently tied to the quality of the classical router used to generate its training labels, and runtime remains higher than conventional heuristic tools. Continued work on model compression and broader industrial test cases will be essential to establish fully autonomous, production-grade chip design flows.

Huawei Shanghai Artificial Intelligence Laboratory Shanghai Jiao Tong University routing scheme network architecture recurrent neural networks generative routing neural networks policy-gradient placement chip design

Cheng et al (2022).pdf

Paper: Neural Combinatorial Optimization with Reinforcement Learning, Irwan Bello et al. (2016). Introduces the core framework of using policy-gradient reinforcement learning to solve combinatorial optimization problems without handcrafted heuristics, underpinning the placement learning formulation.
Paper: Pointer networks, Oriol Vinyals et al. (2015). Pioneers the attention-based Pointer Network architecture that enables neural models to sequentially select and place variable numbers of combinatorial elements.
Paper: Attention, Learn to Solve Routing Problems!, Wouter Kool et al. (2018). Develops the attention-driven encoder-decoder reinforcement learning method for routing and sequencing decisions that informs the net-ordering and routing modules.
Paper: Learning Combinatorial Optimization Algorithms over Graphs, Elias Boutros Khalil et al. (2017). Demonstrates how graph neural representations can be coupled with deep reinforcement learning to solve complex graph-structured combinatorial optimization problems.
Paper: Machine Learning for Combinatorial Optimization: a Methodological Tour d'Horizon, Yoshua Bengio et al. (2018). Provides a comprehensive methodological foundation for replacing traditional heuristics with learned optimization policies in discrete combinatorial problems.

Paper: Versatile Multi-stage Graph Neural Network for Circuit Representation, Shuwen Yang et al. (2022). Advances electronic design automation representations by integrating both topological netlists and physical geometric placement layouts into a versatile circuit graph neural network.
Paper: Combinatorial Optimization and Reasoning with Graph Neural Networks, Quentin Cappart et al. (2023). Synthesizes and surveys advancements in using graph neural networks and deep learning pipelines to solve complex combinatorial optimization and electronic design automation challenges.
Paper: CktGNN: Circuit Graph Neural Network for Electronic Design Automation, Zehao Dong et al. (2023). Extends learning-based electronic design automation by formulating hierarchical graph neural networks to jointly generate circuit topologies and size parameters for analog circuits.

Abstract

Placement and routing are two critical yet time-consuming steps of chip design in modern VLSI systems. Distinct from traditional heuristic solvers, this paper on one hand proposes an RL-based model for mixed-size macro placement, which differs from existing learning-based placers that often consider the macro by coarse grid-based mask. While the standard cells are placed via gradient-based GPU acceleration. On the other hand, a one-shot conditional generative routing model, which is composed of a special-designed input-size-adapting generator and a bi-discriminator, is devised to perform one-shot routing to the pins within each net, and the order of nets to route is adaptively learned. Combining these techniques, we develop a flexible neural pipeline, which to our best knowledge, is the first joint placement and routing network without involving any traditional heuristic solver. Experimental results on chip design benchmarks showcase the effectiveness of our approach. Source code will be made publicly available at: https://github.com/Thinklab-SJTU/EDA-AI

Topics

routing scheme network architecture recurrent neural networks

Keywords

generative routing neural networks policy-gradient placement chip design neural macro placement global routing detailed routing Proximal Policy Optimization net order learning variational autoencoders

1 Introduction
2 Related Work
3 Methodology
3.1 Reinforcement Learning for Mixed-size Placement
3.2 Conditional Generative Learning for Pin Routing in Net-Grid-based Conditional Generative Routing
3.2.1 Layout Input-size-Adapting Generator
3.2.2 Bi-Discriminator to Consider both Realness and Connectivity
3.2.3 Enhanced Model Loss
3.3 Neural Macro Placement and Routing Pipeline
3.3.1 Reward Adaptation between Coarse HPWL and Router's WL
3.3.2 Learning Net Order to Route
4 Experiments
4.1 Protocols and Setup
4.2 Results on Mixed-size Placement
4.3 Results on Routing
4.4 Comparison with a Concurrent Version of Our Router and Other Classic Routers
4.5 Results of Overall Placement and Routing with Ablation Study on Net Order Learning
5 Conclusion and Outlook
Acknowledgement
References

Knowls

Knowl 1 — PRNet neural placement-and-routing pipeline
model/method
The paper proposes PRNet, a neural pipeline that combines three learned components: (1) a PPO-based agent that sequentially places mixed-size macros, (2) gradient-based GPU optimization for placing standard cells after macro placement, and (3) a conditional generative router that produces the complete route for one selected net in a single forward pass. A separate reinforcement-learning agent selects the order in which nets are routed. The pipeline feeds the resulting routing wirelength and congestion back to the placement and net-order policies, avoiding a conventional heuristic placer or router as the main optimization engine. The authors present this as, to their best knowledge, the first fully neural placement-and-global-routing pipeline, although disconnected generated routes are refined with maze routing during route post-processing.
Knowl 2 — Mixed-size macro-placement Markov decision process
model/method
The macro placer represents a chip layout by a binary occupancy image $I \in \{0,1\}^{n\times n}$ and a netlist graph $H$ containing the positions of already placed macros. Unlike a zero-initialized image, the initial image marks every pre-occupied location, including fixed macros, with $I_{xy}=1$ . At step $t$ , the action places the current macro of height $h$ and width $w$ at center $(x_o,y_o)$ . The action is feasible only when every point in the macro region
$R=\{(x,y): |x-x_o|\leq h/2,\ |y-y_o|\leq w/2\}$
is unoccupied, meaning $I_p=0$ for all $p\in R$ . The terminal reward is a negative weighted sum of total wirelength $L_{\mathrm{wl}}$ , routing congestion $L_{\mathrm{cg}}$ , and overlapping area $L_{\mathrm{ol}}$ :
$R_E=-L_{\mathrm{wl}}-\lambda_1L_{\mathrm{cg}}-\lambda_2L_{\mathrm{ol}},$
where $\lambda_1,\lambda_2\geq 0$ are reward weights. The policy uses a CNN for the layout image and a three-layer GCN with 16, 32, and 16 feature channels for the netlist graph; PPO updates the policy.
Knowl 3 — One-net global-routing representation
definition
The global-routing grid is a graph $\mathcal{G}=(V,E)$ in which each vertex represents a rectangular global-routing cell and each edge represents the shared boundary between adjacent cells. For an edge $e\in E$ , $c_e$ is its allowed wire capacity, $u_e$ is its assigned-wire usage, and the overflow is $o_e=\max(0,u_e-c_e)$ . Each net is routed as a rectilinear Steiner-tree-like structure. For one net, the conditional routing problem is represented as an image-to-image mapping $x\mapsto y$ : $x$ has three channels encoding pin locations, available horizontal edges, and available vertical edges, while the single-channel target $y$ marks whether each grid point belongs to the route. The generator outputs a route likelihood at every grid point; points above a threshold are collected as the predicted route. If the thresholded output is disconnected, maze routing is applied to connect its components. The model does not use a random latent-noise input because the routing task is treated as essentially deterministic rather than stochastic.
Knowl 4 — Input-size-adapting routing generator
model/method
The conditional generator handles both grids no larger than $64\times64$ and larger grids. For inputs of size at most $64\times64$ , smaller layouts are padded to $64\times64$ and processed by $G_{\mathrm{base}}$ , which contains a convolutional front end, residual blocks, transposed convolutions, and a final convolutional output layer. For larger layouts, $G_{\mathrm{large}}$ combines a guiding network $G_i$ with a filling network $G_o$ . The filling network first extracts a feature map from the original large input; a downsampled version of the input is passed through the guiding network to obtain a second feature map. These feature maps are added elementwise and injected into later filling-network layers before route generation. The guiding network supplies global information for long-range dependencies, while convolutional locality captures neighboring grid connectivity. The sub-networks are first pretrained separately and then jointly fine-tuned; additional sub-networks can be stacked for still larger inputs.
Knowl 5 — Bi-discriminator conditional routing objective
equation
The routing GAN uses two discriminators: $D_1$ evaluates whether a generated route connects all pins, and $D_2$ evaluates its visual or distributional realness. Connectivity labels for generated and reference routes are computed and used to train the first branch; ordinary real/fake labels train the second. The weighted adversarial objective is
$\mathcal{L}_{\mathrm{adv}}(G,D)=\sum_{i=1}^{2}\lambda_i\left(\mathbb{E}_{x,y}[\log D_i(x,y)]+\mathbb{E}_{x}[\log(1-D_i(x,G(x)))]\right),$
where $G$ is the route generator, $x$ is a conditional routing image, $y$ is its reference route, $D_i$ is discriminator branch $i$ , and $\lambda_1,\lambda_2\geq0$ satisfy $\lambda_1+\lambda_2=1$ . Both branches share a convolutional front end followed by three convolutional/residual blocks, then split into connectivity and realness heads. The connectivity branch addresses the hard constraint that every pin of a net must be connected, while the realness branch discourages implausible route patterns.
Knowl 6 — Focal, geometric, and adversarial losses for route synthesis
equation
To prevent the generator from predicting an empty route because most grid points are easy negative examples, the paper adds a focal pixel loss:
$\mathcal{L}_{\mathrm{FL}}(G)=-\mathbb{E}_{x,y}\left[\frac{1}{N}\sum_{i=1}^{N}\alpha\left(y_i(1-g_i)^\gamma\log g_i+(1-y_i)g_i^\gamma\log(1-g_i)\right)\right],$
where $N$ is the number of grid points, $y_i\in\{0,1\}$ is the reference route label at point $i$ , $g_i\in(0,1)$ is the generator's predicted route likelihood, and $\alpha$ and $\gamma$ are focal-loss weighting parameters. The objective also contains a pixelwise $L_2$ loss $\mathcal{L}_{L2}(G)$ between generated and reference route images. To control the wirelength increase that can result from emphasizing connectivity, it adds
$\mathcal{L}_r(G)=\mathbb{E}_x\left[\left\|l(G(x))-h(x)\right\|_1\right],$
where $l(G(x))$ is the generated route length and $h(x)$ is the net's half-perimeter wirelength, both measured in grid-wire units. The complete generator objective is
$\min_G\left(\max_D\mathcal{L}_{\mathrm{adv}}(G,D)+\mu_{\mathrm{FL}}\mathcal{L}_{\mathrm{FL}}(G)+\mu_{L2}\mathcal{L}_{L2}(G)+\mu_r\mathcal{L}_r(G)\right),$
where the three nonnegative $\mu$ coefficients control the focal, pixelwise, and wirelength regularizers.
Knowl 7 — Alternating end-to-end training with adaptive wirelength reward
algorithm
PRNet alternates routing-model training with reinforcement-learning training of placement and net ordering. First, the generative router is updated using routes induced by the current macro placement, analogous to an expectation step. Next, the macro-placement and net-order policies are jointly updated with PPO to reduce the wirelength returned by the trained router, analogous to a maximization step. This alternation is repeated.

Because early random placements produce pin distributions that are difficult for an untrained or partially trained router, the placement reward gradually changes from a coarse geometric estimate to the neural router's estimate. The smoothed wirelength is
$WL_s=\lambda WL_n+(1-\lambda)HPWL,$
where $WL_n$ is the neural router's wirelength, $HPWL$ is the half-perimeter wirelength of the net bounding box, and $\lambda$ is updated after iteration $n_{\mathrm{iter}}$ as
$\lambda=1-e^{-0.01n_{\mathrm{iter}}}.$
Thus $\lambda=0$ initially and the reward begins with HPWL; as training proceeds, the learned router increasingly determines the placement objective.
Knowl 8 — Reinforcement learning for adaptive net-order selection
model/method
The routing-order agent chooses which unrouted net to process next. Its state contains a three-channel routing image $R$ encoding already routed nets, horizontal-edge capacities, and vertical-edge capacities, together with a graph $G_N$ whose vertices are nets and whose edges connect nets that share a cell. This graph is the edge-to-vertex dual of the original netlist hypergraph: original netlist hyperedges become vertices, and shared original cells induce edges between those vertices. The action is the selection of the next net. The placement and net-order agents use the same policy-network design to produce task-specific feature embeddings and are trained jointly because both decisions affect total wirelength and congestion. Unlike a fixed heuristic order such as routing smaller nets first, the learned order can change in response to the current routing state.
Knowl 9 — Training and evaluation protocol
experimental setup
The placement experiments use preprocessed ISPD-2005 circuits, with most fixed macros converted to movable macros. Routing data come from ISPD-2007 instances and use routes from NCTU-GR 2.0 as supervision. From roughly 750,000 routing instances, the authors construct Route-small-4 with 30,000 $64\times64$ samples and at most four pins per net, Route-small with 80,000 $64\times64$ samples, and Route-large with 100,000 $128\times128$ samples. Each dataset uses an 80%/20% random train/test split. The best $64\times64$ model is then trained with 200,000 additional samples, and the $128\times128$ model with 400,000 additional samples, for ISPD-1998 evaluation.

PPO trains the placement and net-order policies; Adam uses learning rate $2.5\times10^{-4}$ for those policies. Routing models use Adam with learning rate $2\times10^{-4}$ , $\beta_1=0.5$ , $\beta_2=0.999$ , weight decay $0.01$ , batch size 64, and linear learning-rate decay. Experiments run with PyTorch on RTX 3090 GPUs and an AMD 3970X 32-core CPU. Placement is evaluated with HPWL and overlap area; complete placement-and-routing solutions use wirelength (WL) and routing congestion (RC). Generative routing uses correctness rate (CrrtR), the fraction of connected overflow-free predictions, and wirelength ratio (WLR), the predicted-to-reference wirelength ratio for connected overflow-free routes.

Knowl 10 — Mixed-size placement reduces macro overlap

data/table

The mixed-size placer is compared with DeepPlace on eight ISPD-2005 circuits. Both methods first generate macro placements with reinforcement learning and then use gradient-based optimization for the complete placement. The proposed method has slightly higher aggregate wirelength but substantially less overlap, showing the benefit of representing the true macro shape instead of treating every macro as a single coarse grid cell.

Circuit	# Cells	# Mov.	Mixed-size technique		DeepPlace
			Wirelength $\downarrow$	Overlap Area $\downarrow$	Wirelength $\downarrow$	Overlap Area $\downarrow$
adaptec1	211K	514	82783826	12606828	80117232	66608273
adaptec2	255K	542	123307824	19485631	123265964	47085963
adaptec3	451K	710	232373680	58588016	241072304	140272759
adaptec4	496K	1309	234008876	73075220	236391936	169853555
bigblue1	278K	551	141020208	2041890	140435296	3519755
bigblue2	558K	948	144803296	70702107	140465488	103663199
bigblue3	1097K	1227	468632064	39664931	450633360	574956948
bigblue4	2177K	659	1001315712	67794270	951984128	87630042
ratio	-	-	1.000	1.0	0.987	3.9

Across the eight circuits, the proposed method's wirelength is within a 1.3% average difference of DeepPlace, while its aggregate overlap-area ratio is approximately four times smaller. The reduced overlap also decreases the amount of collision-resolution post-processing required.

Knowl 11 — Bi-discriminator ResNet routing achieves the strongest benchmark accuracy

data/table

The routing models are evaluated on Route-small-4 and Route-small using correctness rate (CrrtR, higher is better) and wirelength ratio (WLR, lower is better). The full model combines a ResNet generator, the connectivity-and-realness bi-discriminator, and the enhanced loss. It obtains the best or near-best results in both datasets, reaching CrrtR $0.814\pm0.001$ and WLR $1.010\pm0.000$ on Route-small-4, and CrrtR $0.735\pm0.010$ and WLR $1.018\pm0.004$ on Route-small.

Model	Route-small-4		Route-small
	CrrtR $\uparrow$	WLR $\downarrow$	CrrtR $\uparrow$	WLR $\downarrow$
CVAE*(CNN)	0.414 $\pm$ 0.020	1.179 $\pm$ 0.033	0.397 $\pm$ 0.008	1.042 $\pm$ 0.006
CVAE*-cGAN(CNN)	0.557 $\pm$ 0.065	1.292 $\pm$ 0.108	0.439 $\pm$ 0.021	1.315 $\pm$ 0.015
CVAE*-bcGAN(CNN)	0.474 $\pm$ 0.048	1.525 $\pm$ 0.029	0.488 $\pm$ 0.007	1.241 $\pm$ 0.012
U-Net*	0.724 $\pm$ 0.001	3.306 $\pm$ 0.266	0.524 $\pm$ 0.005	1.232 $\pm$ 0.016
cGAN(U-Net*)	0.602 $\pm$ 0.009	1.028 $\pm$ 0.001	0.532 $\pm$ 0.011	1.286 $\pm$ 0.022
bcGAN(U-Net*)	0.721 $\pm$ 0.012	1.134 $\pm$ 0.055	0.552 $\pm$ 0.007	1.104 $\pm$ 0.054
ResNet	0.783 $\pm$ 0.002	1.023 $\pm$ 0.003	0.594 $\pm$ 0.004	1.030 $\pm$ 0.007
cGAN(ResNet)	0.698 $\pm$ 0.010	1.073 $\pm$ 0.011	0.568 $\pm$ 0.020	1.320 $\pm$ 0.151
bcGAN(ResNet)	0.804 $\pm$ 0.021	1.035 $\pm$ 0.013	0.738 $\pm$ 0.005	1.036 $\pm$ 0.002
bcGAN(ResNet)+EL	0.814 $\pm$ 0.001	1.010 $\pm$ 0.000	0.735 $\pm$ 0.010	1.018 $\pm$ 0.004

The bi-discriminator substantially improves the ResNet generator over the ordinary cGAN, while the enhanced loss further reduces WLR with only a small correctness-rate change. Relative to CVAE*(CNN), the full model gives approximately twice the correctness rate on Route-small-4 and improves its WLR by 14.7%; on Route-small, the WLR improvement is 2.4%. An attempted reproduction of the step-by-step reinforcement-learning router did not converge after two weeks of training.

Knowl 12 — End-to-end PRNet improves wirelength and congestion

data/table

On the eight ISPD-2005 circuits, the complete PRNet is compared with an RL-based placer alone and with the same placer augmented by the generative router but without learned net ordering. The full method has the lowest wirelength and routing congestion on every listed circuit, demonstrating that both generative routing and adaptive net ordering contribute to the integrated result.

Circuit	RL-based Placer		RL-based Placer + GR		RL-based Placer + GR + NOL
	WL $\downarrow$	RC $\downarrow$	WL $\downarrow$	RC $\downarrow$	WL $\downarrow$	RC $\downarrow$
adaptec1	6149	10.565	5940	10.464	5787	9.386
adaptec2	23659	46.278	23048	45.249	22977	35.504
adaptec3	30154	62.751	29711	73.324	29462	43.207
adaptec4	47933	128.257	47406	121.435	46964	65.796
bigblue1	7634	11.480	7385	12.391	7230	11.289
bigblue2	16775	26.318	16693	45.945	16617	25.498
bigblue3	42550	67.124	41617	70.187	40509	64.964
bigblue4	15847	34.903	15356	42.096	15283	24.223

Here GR denotes the proposed generative router and NOL denotes learned net-order selection. Removing NOL can substantially increase congestion, especially on adaptec3 and adaptec4; jointly training placement with the neural router further reduces wirelength.

Knowl 13 — ISPD-1998 routing is competitive but slower than heuristic routers

data/table

On four ISPD-1998 routing benchmarks, the sequential generative router produces zero overflow on every case and competitive wirelength, but requires substantially more runtime than established classical routers. The generative router processes the entire grid image for one net at a time, whereas the heuristic routers exploit more localized procedures.

Circuit	Our router		NTHU-Route 2.0		BoxRouter 2.0		FastRoute 3.0
	WL $\downarrow$	Time(s) $\downarrow$	WL $\downarrow$	Time(s) $\downarrow$	WL $\downarrow$	Time(s) $\downarrow$	WL $\downarrow$	Time(s) $\downarrow$
ibm01	62337	59.2	62498	1.54	62659	33	64221	0.64
ibm02	170270	179.9	169881	3.15	171110	36	172223	0.85
ibm03	146362	194.6	146458	1.49	146634	18	146753	0.49
ibm04	165874	254.4	166452	3.81	167275	116	170146	2.7

A concurrent inference variant routes 16 nets at a time rather than one net at a time. It achieves a reported $3.36$ -- $7.98\times$ runtime speedup, but the parallel routes can conflict in shared capacity, increasing wirelength and overflow. The sequential variant has the strongest route quality; the concurrent variant exposes a practical speed-quality tradeoff.

Knowl 14 — Supervised-router labels limit the routing model
limitation
The conditional generative router is trained on routes produced by a strong classical router, so its attainable quality is bounded by the quality, biases, and coverage of those labels. The paper identifies unsupervised or semi-supervised routing training as a possible alternative. It also reports that the current neural router is slower than strong heuristic routers in sequential inference, while concurrent inference improves speed at the cost of overflow and wirelength. The authors additionally note a possible negative societal impact: automation of placement and routing could reduce employment in the EDA industry.

Coverage note — The full concurrent-versus-classical benchmark table and visual placement illustrations were summarized rather than reproduced in full; their main speed, quality, and geometric findings are retained.

References

1.A. Mirhoseini, A. Goldie, M. Yazgan, J. W. Jiang, E. Songhori, S. Wang, Y.-J. Lee, E. Johnson, O. Pathak, A. Nazi et al., “A graph placement methodology for fast chip design,” Nature, 2021.
2.R. Cheng and J. Yan, “On joint learning for solving placement and routing in chip design,” NeurIPS, 2021.
3.J. Liu, G. Chen, and E. F. Young, “Rest: Constructing rectilinear steiner minimum tree via reinforcement learning,” in DAC, 2021.
4.Y.-L. Chuang, G.-J. Nam, C. J. Alpert, Y.-W. Chang, J. Roy, and N. Viswanathan, “Design hierarchy aware mixed-size placement for routability optimization,” in ICCAD, 2010.
5.G.-J. Nam, M. Yildiz, D. Pan, and P. Madden, “Ispd 2007 global routing contest,” 2007.
6.H. Liao, W. Zhang, X. Dong, B. Poczos, K. Shimada, and L. Burak Kara, “A deep reinforcement learning approach for global routing,” Journal of Mechanical Design, 2020.
7.D. Vashisht, H. Rampal, H. Liao, Y. Lu, D. Shanbhag, E. Fallon, and L. B. Kara, “Placement in integrated circuits using cyclic reinforcement learning and simulated annealing,” arXiv preprint arXiv:2011.07577, 2020.
8.H. Liao, Q. Dong, X. Dong, W. Zhang, W. Zhang, W. Qi, E. Fallon, and L. B. Kara, “Attention routing: track-assignment detailed routing using attention-based reinforcement learning,” in International Design Engineering Technical Conferences and Computers and Information in Engineering Conference. American Society of Mechanical Engineers, 2020.
9.D. Utyamishev and I. Partin-Vaisband, “Late breaking results: A neural network that routes ics,” in DAC. IEEE, 2020.
10.D. P. Kingma and M. Welling, “Auto-encoding variational bayes,” in ICLR, 2014.
11.B. Xu, Y. Lin, X. Tang, S. Li, L. Shen, N. Sun, and D. Z. Pan, “Wellgan: Generative-adversarial-network-guided well generation for analog/mixed-signal circuit layout,” in DAC. IEEE, 2019.
12.I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” in NIPS, 2014.
13.K. Zhu, H. Chen, M. Liu, X. Tang, W. Shi, N. Sun, and D. Z. Pan, “Generative-adversarial-network-guided well-aware placement for analog circuits,” in ASP-DAC. IEEE, 2022.
14.A. Gusmão, R. Póvoa, N. Horta, N. Lourenço, and R. Martins, “Deepplacer: A custom integrated opamp placement tool using deep models,” Applied Soft Computing, 2022.
15.W. Jin, S. Sadiqbatcha, J. Zhang, and S. X.-D. Tan, “Full-chip thermal map estimation for commercial multi-core cpus with generative adversarial learning** this work is supported in part by nsf grants under no. ccf-1816361, in part by nsf grant under no. ccf-2007135 and no. oise-1854276.” in 2020 IEEE/ACM International Conference On Computer Aided Design (ICCAD). IEEE, 2020.
16.Z. Zhou, Z. Zhu, J. Chen, Y. Ma, and A. Ivanov, “Congestion-aware global routing using deep convolutional generative adversarial networks,” in 2019 ACM/IEEE 1st Workshop on Machine Learning for CAD (MLCAD), 2019.
17.D. Utyamishev and I. Partin-Vaisband, “Multiterminal pathfinding in practical vlsi systems with deep neural networks,” Preprint from Research Square, 2022.
18.H. Yang, P. Pathak, F. Gennari, Y. C. Lai, and B. Yu, “Deepattern: Layout pattern generation with transforming convolutional auto-encoder,” in the 56th Annual DAC 2019, 2019.
19.A. Hottung, B. Bhandari, and K. Tierney, “Learning a latent search space for routing problems using variational autoencoders,” in ICLR, 2020.
20.M. D. Moffitt, “Maizerouter: Engineering an effective global router,” TCAD, 2008.
21.M. M. Ozdal and M. D. Wong, “Archer: A history-based global routing algorithm,” TCAD, 2009.
22.V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski et al., “Human-level control through deep reinforcement learning,” Nature, vol. 518, no. 7540, pp. 529–533, 2015.
23.T. N. Kipf and M. Welling, “Semi-supervised classification with graph convolutional networks,” in ICLR, 2017.
24.J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017.
25.M. R. Garey and D. S. Johnson, “The rectilinear steiner tree problem is np-complete,” SIAM Journal on Applied Mathematics, vol. 32, no. 4, pp. 826–834, 1977.
26.J. Johnson, A. Alahi, and L. Fei-Fei, “Perceptual losses for real-time style transfer and super-resolution,” in European conference on computer vision. Springer, 2016, pp. 694–711.
27.T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár, “Focal loss for dense object detection,” in Proceedings of the IEEE international conference on computer vision, 2017.
28.D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros, “Context encoders: Feature learning by inpainting,” in CVPR, 2016.
29.P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” CVPR 2017, pp. 5967–5976, 2017.
30.Y.-J. Chang, Y.-T. Lee, J.-R. Gao, P.-C. Wu, and T.-C. Wang, “Nthu-route 2.0: a robust global router for modern designs,” TCAD, 2010.
31.Y. Xu, Y. Zhang, and C. Chu, “Fastroute 4.0: Global router with efficient via minimization,” in ASP-DAC. IEEE, 2009.
32.G.-J. Nam, C. J. Alpert, P. Villarrubia, B. Winter, and M. Yildiz, “The ispd2005 placement contest and benchmark suite,” in Proceedings of the 2005 international symposium on Physical design, 2005, pp. 216–220.
33.W.-H. Liu, W.-C. Kao, Y.-L. Li, and K.-Y. Chao, “Nctu-gr 2.0: Multithreaded collision-aware global routing with bounded-length maze routing,” TCAD, 2013.
34.C. J. Alpert, “The ispd98 circuit benchmark suite,” in Proceedings of the 1998 international symposium on Physical design, 1998, pp. 80–85.
35.P. D. Kingma and L. J. Ba, “Adam: A method for stochastic optimization,” ICLR, 2015.
36.C.-K. Cheng, A. B. Kahng, I. Kang, and L. Wang, “Replace: Advancing solution quality and routability validation in global placement,” TCAD, 2018.
37.Y. Lin, Z. Jiang, J. Gu, W. Li, S. Dhar, H. Ren, B. Khailany, and D. Z. Pan, “Dreamplace: Deep learning toolkit-enabled gpu acceleration for modern vlsi placement,” TCAD, 2020.
38.X. Yu, X. Zhang, Y. Cao, and M. Xia, “Vaegan: A collaborative filtering framework based on adversarial variational autoencoders.” in IJCAI, 2019, pp. 4206–4212.
39.R. O, F. P, and B. T, “U-net: Convolutional networks for biomedical image segmentation,” in MICAAI, 2015.
40.K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in CVPR, 2016.
41.M. Cho, K. Lu, K. Yuan, and D. Z. Pan, “Boxrouter 2.0: A hybrid and robust global router with layer assignment for routability,” TODAES, 2009.
42.Y. Zhang, Y. Xu, and C. Chu, “Fastroute3. 0: a fast and high quality global router based on virtual capacity,” in ICCAD. IEEE, 2008.
43.R. Kastner, E. Bozorgzadeh, and M. Sarrafzadeh, “Pattern routing: Use and theory for increasing predictability and avoiding coupling,” TCAD, 2002.
44.R. T. Hadsell and P. H. Madden, “Improved global routing through congestion estimation,” in DAC. IEEE, 2003.
45.M. Cho and D. Z. Pan, “Boxrouter: A new global router based on box expansion and progressive ilp,” TCAD, 2007.

Citation

MLA

Cheng, R., et al. “The Policy-gradient Placement and Generative Routing Neural Networks for Chip Design”. Advances in Neural Information Processing Systems, vol. 35, 2022, pp. 26350–62, https://proceedings.neurips.cc/paper_files/paper/2022/file/a8b8c1ad51df1b93d9e3d1fca75debbf-Paper-Conference.pdf.

APA

Cheng, R., Lyu, X., Li, Y., Ye, J., Hao, J., & Yan, J. (2022). The Policy-gradient Placement and Generative Routing Neural Networks for Chip Design. Advances in Neural Information Processing Systems, 35, 26350–26362. https://proceedings.neurips.cc/paper_files/paper/2022/file/a8b8c1ad51df1b93d9e3d1fca75debbf-Paper-Conference.pdf

Chicago

Cheng, R., X. Lyu, Y. Li, J. Ye, J. Hao, and J. Yan. 2022. “The Policy-gradient Placement and Generative Routing Neural Networks for Chip Design”. Advances in Neural Information Processing Systems 35: 26350–62. https://proceedings.neurips.cc/paper_files/paper/2022/file/a8b8c1ad51df1b93d9e3d1fca75debbf-Paper-Conference.pdf.

Harvard

Cheng, R. et al. (2022) “The Policy-gradient Placement and Generative Routing Neural Networks for Chip Design”, Advances in Neural Information Processing Systems. Curran Associates, Inc., pp. 26350–26362. Available at: https://proceedings.neurips.cc/paper_files/paper/2022/file/a8b8c1ad51df1b93d9e3d1fca75debbf-Paper-Conference.pdf.

Vancouver

1. Cheng R, Lyu X, Li Y, Ye J, Hao J, Yan J (2022) The Policy-gradient Placement and Generative Routing Neural Networks for Chip Design. In: Advances in Neural Information Processing Systems. Curran Associates, Inc., pp 26350–26362

BibTeX

@inproceedings{cheng2022the,
  title = {The Policy-gradient Placement and Generative Routing Neural Networks for Chip Design},
  author = {Cheng, Ruoyu and Lyu, Xianglong and Li, Yang and Ye, Junjie and Hao, Jianye and Yan, Junchi},
  year = {2022},
  booktitle = {Advances in Neural Information Processing Systems},
  publisher = {Curran Associates, Inc.},
  volume = {35},
  pages = {26350-26362},
  url = {https://proceedings.neurips.cc/paper_files/paper/2022/file/a8b8c1ad51df1b93d9e3d1fca75debbf-Paper-Conference.pdf}
}

Metadata:DOI registry

Access the Paper

This paper is available from its original source. Click below to access the PDF.

Open PDF

License: Authors

The Policy-gradient Placement and Generative Routing Neural Networks for Chip Design

Abstract

Topics

Keywords

Table of Contents

Knowls

Knowl 1 — PRNet neural placement-and-routing pipeline

Knowl 2 — Mixed-size macro-placement Markov decision process

Knowl 3 — One-net global-routing representation

Knowl 4 — Input-size-adapting routing generator

Knowl 5 — Bi-discriminator conditional routing objective

Knowl 6 — Focal, geometric, and adversarial losses for route synthesis

Knowl 7 — Alternating end-to-end training with adaptive wirelength reward

Knowl 8 — Reinforcement learning for adaptive net-order selection

Knowl 9 — Training and evaluation protocol

Knowl 10 — Mixed-size placement reduces macro overlap

Knowl 11 — Bi-discriminator ResNet routing achieves the strongest benchmark accuracy

Knowl 12 — End-to-end PRNet improves wirelength and congestion

Knowl 13 — ISPD-1998 routing is competitive but slower than heuristic routers

Knowl 14 — Supervised-router labels limit the routing model

References

Citation

BibTeX

Access the Paper