RISC-V for AI Inference Is a 'Flat-Out Bad Idea,' Says Quadric

Quadric CMO Steve Roddy calls building an AI accelerator from RISC-V a "flat-out bad idea," arguing the CPU-plus-accelerator partition creates compiler and data-movement problems that scale badly with accelerator size. His verdict anchored a Semiconductor Engineering roundtable on where the open ISA fits in AI silicon, joined by Siemens EDA's Francisca Tan, SignatureIP's Ewald Liess and Breker CEO Dave Kelf.

The core dispute is architectural. Roddy said a 100 TOPS design with 16 RISC-V processors slaved to accelerator blocks forces designers to solve data movement across a network-on-chip rather than the compute itself.

Why a CPU is the wrong starting point?

Roddy's argument is statistical. CPUs excel at chasing pointers and handling dozens of unpredictable threads. AI inference is the opposite workload. "You know for millions of cycles in advance what's happening," he said. "You don't need most of the CPU hardware."

DSPs come closer, but they are vector machines — vector-by-scalar or vector-by-vector. The hard math in AI is matrix-by-matrix. Quadric therefore built a matrix-native processor. Roddy said the only comparable architecture he knows of is Google's data-center TPUs. In his view, everyone else runs partitioned systems and spends billions of dollars compensating for a bad starting point.

Where RISC-V actually wins

Kelf offered the counterweight from customer projects. In four instances, both large and small companies, teams found the complex state machines controlling SIMD and SIMT accelerators "hopeless." They replaced them with RISC-V control processors.

The control role suits RISC-V for three reasons, Kelf said:

• Teams can add vector instructions, or the new matrix multiplication instructions, for limited math.
• Custom load/store instructions cut latency between the host and accelerator partitions.
• A standard compiler still works, which is impossible with ad-hoc control logic.

Kelf cited one SIMT customer whose parallel lanes occasionally diverged. The team used the RISC-V control processor to absorb the divergent math, keeping the accelerator running the main workload. It took significant engineering effort, but the customer now considers the design successful.

Kelf also noted the business case: RISC-V carries no royalty. Arm and other vendors have built similar instructions into their processors, he said, but those are not user-variable. RISC-V is, in his words, unique in its ability to create powerful custom instructions.

What about automated customization?

Roddy, who spent 17 years at Tensilica, acknowledged that automated custom-instruction flows already exist. Tensilica's system generates matching compilers and simulators, with verification handled in a structured environment. The trade-off is a per-unit royalty — what Roddy called a "tribute, a tax to the big guy."

His advice to RISC-V adopters is blunt: commit fully. If a team wants only slight customization, the automated commercial flow is cheaper. If it chooses RISC-V, it should customize heavily to justify the manual tool work.

He also drew a boundary around the market. RISC-V can run the thousands of general control tasks CPUs handle today, more flexibly and at a lower price point than fixed-function cores. Heavyweight inference, in his view, is not one of those tasks.

Can verification keep up with custom instructions?

Tan said Siemens EDA supports debug of custom-instruction tracing, letting users specify which custom instructions to trace while the processor runs. Siemens donated the Efficient Trace specification to RISC-V International. E-Trace is a highly compressed instruction trace format designed for complex systems where trace data volumes grow large. The Siemens infrastructure captures traces from the core and from the wider SoC — interconnect, bus monitors and arbitrary signals.

Liess framed the harder problem at system level. Generated IP usually ships with test benches, traffic generators and performance models, so ISA-level verification often passes. The failure mode appears later. Multiple RISC-V cores with caches create coherency corner cases, and traffic bursts can deadlock the fabric.

"An instruction can execute correctly, but the chip can still fail if the data burst needed for the instruction isn't available," Liess said. SignatureIP addresses this by verifying traffic flow through its pre-verified NoC at system level. On interfaces, Liess said customers freely modify RISC-V cores but keep standard AMBA, CXL and PCI Express links for memory and off-chip access — which is where SignatureIP's IP attaches.

The roundtable consensus points one direction: RISC-V's customization economics favor control, orchestration and observability roles around accelerators, while matrix-native architectures contest the inference math itself — a division of labor likely to harden as accelerator designs grow.