1. [Publications](/publications) 2. Symphony: Orchestrating Sparse and Dense Tensors with Hierarchical Heterogeneous Processing # Symphony: Orchestrating Sparse and Dense Tensors with Hierarchical Heterogeneous Processing ![](/sites/default/files/styles/wide/public/publications/Symphony_Image.png?itok=LVjplzo0) Sparse tensor algorithms are becoming widespread, particularly in the domains of deep learning, graph and data analytics, and scientific computing. Current high-performance broad-domain architectures, such as GPUs, often suffer memory system inefficiencies by moving too much data or moving it too far through the memory hierarchy. To increase performance and efficiency, proposed domain-specific accelerators tailor their architectures to the data needs of a narrow application domain, but as a result cannot be applied to a wide range of algorithms or applications that contain a mix of sparse and dense algorithms. This article proposes Symphony, a hybrid programmable/specialized architecture that focuses on the orchestration of data throughout the memory hierarchy to simultaneously reduce the movement of unnecessary data and data movement distances. Key elements of the Symphony architecture include (1) specialized reconfigurable units aimed not only at roofline floating-point computations but also at supporting data orchestration features, such as address generation, data filtering, and sparse metadata processing; and (2) distribution of computation resources (both programmable and specialized) throughout the on-chip memory hierarchy. We demonstrate that Symphony can match non-programmable ASIC performance on sparse tensor algebra and provide 31× improved runtime and 44× improved energy over a comparably provisioned GPU for these applications. ## Authors [Michael Pellauer](/person/michael-pellauer) [Jason Clemons](/person/jason-clemons) [Vignesh Balaji](/person/vignesh-balaji) [Neal Crago](/person/neal-crago) [Aamer Jaleel](/person/aamer-jaleel) [Donghyuk Lee](/person/donghyuk-lee) [Mike O'Connor](/person/mike-o-connor) [Angshuman Parashar](/person/angshuman-parashar) Sean Treichler (NVIDIA) [Po-An Tsai](/person/po-an-tsai) [Steve Keckler](/person/stephen-keckler) [Joel Emer](/person/joel-emer) ## Publication Date Monday, December 18, 2023 ## Published in [ACM Transactions on Computer Systems](https://dl.acm.org/doi/10.1145/3630007) ## Research Area [Computer Architecture](/research-area/computer-architecture) ## Copyright Copyright by the Association for Computing Machinery, Inc. Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from Publications Dept, ACM Inc., fax +1 (212) 869-0481, or . The definitive version of this paper can be found at ACM's Digital Library .