Senad S Durakovic, Age 61410 Sheridan Ave APT 222, Palo Alto, CA 94306

2021024, Aug 5, 2021

Apr 22, 2021

17/237752

- Singapore, SG
Senad DURAKOVIC - Palo Alto CA, US
Gopal NALAMALAPU - Santa Clara CA, US

G06F 9/48
G06F 3/06
G06N 20/00
G06F 9/52

A system to support a machine learning (ML) operation comprises an array-based inference engine comprising a plurality of processing tiles each comprising at least one or more of an on-chip memory (OCM) configured to maintain data for local access by components in the processing tile and one or more processing units configured to perform one or more computation tasks on the data in the OCM by executing a set of task instructions. The system also comprises a data streaming engine configured to stream data between a memory and the OCMs and an instruction streaming engine configured to distribute said set of task instructions to the corresponding processing tiles to control their operations and to synchronize said set of task instructions to be executed by each processing tile, respectively, to wait current certain task at each processing tile to finish before starting a new one.

2021005, Feb 25, 2021

Oct 2, 2020

16/948867

- Singapore, SG
Ulf Hanebutte - Gig Harbor WA, US
Senad Durakovic - Palo Alto CA, US
Hamid Reza Ghasemi - Sunnyvale CA, US
Chia-Hsin Chen - Santa Clara CA, US

G06F 9/38
G06N 20/00
G06N 20/10
G06F 17/16
G06F 15/78

An array-based inference engine includes a plurality of processing tiles arranged in a two-dimensional array of a plurality of rows and a plurality of columns. Each processing tile comprises at least one or more of an on-chip memory (OCM) configured to load and maintain data from the input data stream for local access by components in the processing tile and further configured to maintain and output result of the ML operation performed by the processing tile as an output data stream. The array includes a first processing unit (POD) configured to perform a dense and/or regular computation task of the ML operation on the data in the OCM. The array also includes a second processing unit/element (PE) configured to perform a sparse and/or irregular computation task of the ML operation on the data in the OCM and/or from the POD.

2019024, Aug 8, 2019

Dec 19, 2018

16/226508

- Santa Clara CA, US
Ulf HANEBUTTE - Gig Harbor WA, US
Senad DURAKOVIC - Palo Alto CA, US
Hamid Reza GHASEMI - Sunnyvale CA, US
Chia-Hsin CHEN - Santa Clara CA, US

G06F 9/38
G06N 5/04
G06N 20/00

A programmable hardware system for machine learning (ML) includes a core and an inference engine. The core receives commands from a host. The commands are in a first instruction set architecture (ISA) format. The core divides the commands into a first set for performance-critical operations, in the first ISA format, and a second set of performance non-critical operations, in the first ISA format. The core executes the second set to perform the performance non-critical operations of the ML operations and streams the first set to inference engine. The inference engine generates a stream of the first set of commands in a second ISA format based on the first set of commands in the first ISA format. The first set of commands in the second ISA format programs components within the inference engine to execute the ML operations to infer data.

2019024, Aug 8, 2019

Dec 19, 2018

16/226539

- Santa Clara CA, US
Ulf HANEBUTTE - Gig Harbor WA, US
Senad DURAKOVIC - Palo Alto CA, US
Hamid Reza GHASEMI - Sunnyvale CA, US
Chia-Hsin CHEN - Santa Clara CA, US

G06F 15/80
G06N 5/04
G06N 20/00
G06F 15/78
G06F 9/38

An array-based inference engine includes a plurality of processing tiles arranged in a two-dimensional array of a plurality of rows and a plurality of columns. Each processing tile comprises at least one or more of an on-chip memory (OCM) configured to load and maintain data from the input data stream for local access by components in the processing tile and further configured to maintain and output result of the ML operation performed by the processing tile as an output data stream. The array includes a first processing unit (POD) configured to perform a dense and/or regular computation task of the ML operation on the data in the OCM. The array also includes a second processing unit/element (PE) configured to perform a sparse and/or irregular computation task of the ML operation on the data in the OCM and/or from the POD.

2019024, Aug 8, 2019

Dec 19, 2018

16/226559

- Santa Clara CA, US
Ulf HANEBUTTE - Gig Harbor WA, US
Senad DURAKOVIC - Palo Alto CA, US
Hamid Reza GHASEMI - Sunnyvale CA, US
Chia-Hsin CHEN - Santa Clara CA, US
Rishan TAN - Santa Clara CA, US

G06F 17/16
G06F 15/78
G06N 20/00
G06N 5/04

A processing unit of an inference engine for machine learning (ML) includes a first data load steamer, a second data load streamer, an operator component, and a store streamer. The first data load streamer streams a first data stream from an on-chip memory (OCM) to the operator component. The second data load streamer streams a second data stream from the OCM to the operator component. The operator component performs a matrix operation on the first data stream and the second data stream. The store streamer receives a data output stream from the operator component and to store the data output stream in a buffer.

2019024, Aug 8, 2019

Dec 19, 2018

16/226534

- Santa Clara CA, US
Ulf HANEBUTTE - Gig Harbor WA, US
Senad DURAKOVIC - Palo Alto CA, US
Hamid Reza GHASEMI - Sunnyvale CA, US
Chia-Hsin CHEN - Santa Clara CA, US

G06N 5/04
G06N 20/10
G06N 3/063
G06F 9/54
G06F 12/06

A programmable hardware system for machine learning (ML) includes a core and a streaming engine. The core receives a plurality of commands and a plurality of data from a host to be analyzed and inferred via machine learning. The core transmits a first subset of commands of the plurality of commands that is performance-critical operations and associated data thereof of the plurality of data for efficient processing thereof. The first subset of commands and the associated data are passed through via a function call. The streaming engine is coupled to the core and receives the first subset of commands and the associated data from the core. The streaming engine streams a second subset of commands of the first subset of commands and its associated data to an inference engine by executing a single instruction.

2019024, Aug 8, 2019

Dec 19, 2018

16/226564

- Santa Clara CA, US
Ulf HANEBUTTE - Gig Harbor WA, US
Senad DURAKOVIC - Palo Alto CA, US
Hamid Reza GHASEMI - Sunnyvale CA, US
Chia-Hsin CHEN - Santa Clara CA, US

G06N 5/04
G06N 20/00
G06F 17/16

A programmable hardware system for machine learning (ML) includes a core and an inference engine. The core receives commands from a host. The commands are in a first instruction set architecture (ISA) format. The core divides the commands into a first set for performance-critical operations, in the first ISA format, and a second set of performance non-critical operations, in the first ISA format. The core executes the second set to perform the performance non-critical operations of the ML operations and streams the first set to inference engine. The inference engine generates a stream of the first set of commands in a second ISA format based on the first set of commands in the first ISA format. The first set of commands in the second ISA format programs components within the inference engine to execute the ML operations to infer data.

2019024, Aug 8, 2019

Dec 19, 2018

16/226550

- Santa Clara CA, US
Ulf HANEBUTTE - Gig Harbor WA, US
Senad DURAKOVIC - Palo Alto CA, US
Hamid Reza GHASEMI - Sunnyvale CA, US
Chia-Hsin CHEN - Santa Clara CA, US

G06N 20/00
G06N 5/04
G06F 17/16
G06F 12/0862

A processing unit of an inference engine for machine learning (ML) includes a first, a second, and a third register, and a matrix multiplication block. The first register receives a first stream of data associated with a first matrix data that is read only once. The second register receives a second stream of data associated with a second matrix data that is read only once. The matrix multiplication block performs a multiplication operation based on data from the first register and the second register resulting in an output matrix. A row associated with the first matrix is maintained while rows associated with the second matrix is fed to the matrix multiplication block to perform a multiplication operation. The process is repeated for each row of the first matrix. The third register receives the output matrix from the matrix multiplication block and stores the output matrix.