1 Abstract
I present the engineering process to design a many-fidelity dot product core as described in lab 2 for the University of Waterloo course ECE 493.
The course CI-CD computes an Area-Delay product of ~2500 for the design. This is the result of an implementation flow running on Cadence Genus for the ASAP 7nm education process.
2 Specification
The core is intended to be used in the following manner: implement a GEMM engine by using a 2D array of dot product units. Matrix tiles are streamed through the array.
… the Tenstorrent Wormhole Tensix engine implements a GEMM unit that multiplies (8x16) x (16x16) matrices to produce a (8x16) result matrix. This requires a total of 128 sixteen-lane dot product units, executing 2048 multiplies and adds per cycle.
This is public information [1]
The core is a 16-lane dot product unit in SystemVerilog which would be replicated many times to implement a complete GEMM engine. It supports six number formats (UINT8, E5M10, E8M7, E8M10, E5M2, and E4M3). Each number format may be computed at four fidelities, which represent additional partial products computed for greater accuracy of computation. All in all, the core as specified is very flexible for ML workloads which do not require super high precision to tradeoff on throughput in tokens/sec.
2.1 Handshake
The handshake used is a ready-valid handshake such that the design is latency in-sensitive (but is 1cycle latency in my design). The unit is full-throughput, receiving a new input every cycle for LoFi operations.
3 Structure of the problem
The
4 Ready-Valid Handshake Waveform
The following illustrates the ready-valid handshake timing. Data
is transferred on a rising clock edge when both ivalid
and oready are asserted. The producer asserts
ivalid with data; the consumer asserts
oready when it can accept. ovalid goes
high the cycle after a successful transfer, and result
holds the computed output.