Benchmarking for large-scale placement and beyond

Over the last five years, the large scale integrated circuit placement community achieved great strides in the understanding of placement problems, developed new high-performance algorithms, and achieved impressive empirical results. These advances have been supported by a nontrivial benchmarking infrastructure, and future achievements are set to draw on benchmarking as well. In this paper, we review motivations for benchmarking, especially for commercial electronic design automation, analyze available benchmarks, and point out major pitfalls in benchmarking. Our empirical data offers perhaps the first comprehensive evaluation of several leading large-scale placers on multiple benchmark families. We outline major outstanding problems and discuss the future of placement benchmarking. Furthermore, we attempt to extrapolate our experience to circuit layout tasks beyond placement.