Can AI Agents Design a Working Chip Layout? A New Benchmark Tests the Hard Part

A chip layout is more than a drawing. It is a map of devices and connections across layers of material, built to obey manufacturing rules. Even when a layout is physically legal and matches the intended circuit, its geometry can affect speed, power-related behavior and other electrical properties. The challenge tested by ICLayout-Bench is whether language-model agents can use electronic-design-automation (EDA) software to make these layouts—and refine them based on tool feedback.

The benchmark starts each task with a transistor-level netlist: a description of the circuit’s devices and how they connect. The agent must create a GDSII file, a standard format for representing chip-layout geometry. Tasks also specify a process design kit (PDK), which supplies technology-specific rules and models, along with implementation constraints and electrical tests.

A proposed layout does not earn points simply for looking plausible. An independent evaluator checks whether it obeys physical design rules and matches the circuit’s schematic, then extracts estimated parasitic resistance and capacitance from the geometry. The evaluator simulates the resulting circuit under specified conditions. A layout that fails required physical or functional checks receives a score of zero. Passing layouts are scored on electrical objectives and area, using task-specific targets and weights.

This layered evaluation matters because a layout can fail in different ways: it may break a manufacturing rule, connect devices incorrectly, or produce poor circuit behavior after the effects of wiring are included. The scoring system also prevents strength in one objective from fully compensating for weakness in another: it combines capped credits, including an area credit, through a weighted geometric mean.

The full benchmark contains 165 tasks spanning 13 PDKs and technology nodes from 3 to 180 nanometers. It includes analog, mixed-signal and digital circuits, with both open-source and commercial EDA flows. But the reported model experiments use selected subsets: 21 public and 19 private tasks, chosen to cover circuit types, processes and toolchains while limiting redundancy. The researchers tested 22 configurations across seven models, using the models’ official agent harnesses. Each task had a time budget between one and 12 hours, including EDA operations and checks.

The results show a wide range of performance rather than a single dependable level of capability. On the public subset, average scores ranged from 4.22 to 80.26 out of 100; on the private subset, they ranged from 1.43 to 67.01. These are averages across tasks, with failed attempts and missing submissions counted as zero. The leading configuration differed between the two collections: GPT-6 Astra at medium effort led the public subset, while GPT-6.1 Sol at high effort led the private subset.

More reasoning effort did not consistently produce better layouts. For example, GPT-6 Astra’s public mean score was lower at high effort than at medium effort. That does not establish why the scores changed, or that effort is generally unhelpful; it shows that turning up a model-specific setting was not a reliable route to higher averages in these experiments. Effort labels also differ between providers, so settings with the same name are not necessarily equivalent.

Quality and expense did not line up neatly either. On private tasks, GPT-6 Astra and GPT-6.1 Sol at high effort had nearly equal mean scores—66.92 and 67.01—but reported model costs of $28.19 and $7.19 per task, respectively. Those figures cover model usage under the paper’s pricing and accounting assumptions, not the full cost of running EDA: unpriced computing and software licences are excluded. Token counts also rely on provider reporting and may not be directly comparable.

A five-transistor amplifier example makes the pass-or-fail gate concrete. Among seven high-effort submissions, five passed the complete evaluation and scored between 69.94 and 95.54. One failed physical checks; another passed those checks but failed parasitic extraction. Passing was not the same as matching the best result: two layouts both met the area target, yet their total scores differed because of their electrical credits. These outcomes illustrate this task’s evaluation, not a general success rate for all circuit layouts.

The benchmark’s breadth is useful, but the experiment is narrower than the full inventory. Its selected tasks do not represent every circuit, process or EDA flow, and results may depend on tool access, task constraints and the particular model harnesses used. Electrical behavior is assessed under specified device models, extraction settings and simulation conditions; it is not a direct demonstration that a layout will perform identically in manufactured silicon. The paper also notes limits in what its modeled immunity tests imply about real-world isolation and manufacturing variation.

ICLayout-Bench is presented in an arXiv preprint, which may not have undergone peer review. The paper and accompanying project materials describe the benchmark, but independent replication and broader testing would help establish how well these results generalize. For now, the study provides a structured way to test an important engineering task—and evidence that producing a valid, high-quality layout remains substantially more demanding than generating a circuit description.