4.5 Article

The future of computing beyond Moore's Law

Publisher

ROYAL SOC
DOI: 10.1098/rsta.2019.0061

Keywords

high-performance computing; Moore's Law; microelectronics; lithography; computing; post-CMOS

Funding

  1. Office of Advanced Scientific Computing Research in the Department of Energy Office of Science [DE-AC02-05CH11231]

Ask authors/readers for more resources

Moore's Law is a techno-economic model that has enabled the information technology industry to double the performance and functionality of digital electronics roughly every 2 years within a fixed cost, power and area. Advances in silicon lithography have enabled this exponential miniaturization of electronics, but, as transistors reach atomic scale and fabrication costs continue to rise, the classical technological driver that has underpinned Moore's Law for 50 years is failing and is anticipated to flatten by 2025. This article provides an updated view of what a post-exascale system will look like and the challenges ahead, based on our most recent understanding of technology roadmaps. It also discusses the tapering of historical improvements, and how it affects options available to continue scaling of successors to the first exascale machine. Lastly, this article covers the many different opportunities and strategies available to continue computing performance improvements in the absence of historical technology drivers. This article is part of a discussion meeting issue 'Numerical algorithms for high-performance computational science'.

Authors

I am an author on this paper
Click your name to claim this paper and add it to your profile.

Reviews

Primary Rating

4.5
Not enough ratings

Secondary Ratings

Novelty
-
Significance
-
Scientific rigor
-
Rate this paper

Recommended

Article Computer Science, Hardware & Architecture

A Case For Intra-rack Resource Disaggregation in HPC

George Michelogiannakis, Benjamin Klenk, Brandon Cook, Min Yee Teh, Madeleine Glick, Larry Dennison, Keren Bergman, John Shalf

Summary: With the emergence of specialized accelerators, there is a need for more flexible resource allocation in HPC systems. This study analyzes NERSC's Cori system and profiles deep-learning applications to investigate hardware resource disaggregation. The results demonstrate that intra-rack disaggregation can effectively locate required resources and reduce the utilization of memory and NICs.

ACM TRANSACTIONS ON ARCHITECTURE AND CODE OPTIMIZATION (2022)

Article Computer Science, Hardware & Architecture

ASA: Accelerating Sparse Accumulation in Column-wise SpGEMM

Chao Zhang, Maximilian Bremer, C. Y. Chan, John Shalf, Xiaochen Guo

Summary: Sparse linear algebra is crucial in many applications. The column-wise SpGEMM algorithm has good locality but the sparse accumulation (SPA) step remains a performance bottleneck. This work proposes the ASA architecture to overcome the challenges of SPA and achieves significant speedup.

ACM TRANSACTIONS ON ARCHITECTURE AND CODE OPTIMIZATION (2022)

Article Computer Science, Interdisciplinary Applications

Exascale Was Not Inevitable; Neither Is What Comes Next

Erik W. Draeger, Andrew Siegel

Summary: This article discusses the historical progress of supercomputing capabilities and the challenges of achieving exascale systems, as well as providing insights on designing and implementing scientific applications in the exascale and post-exascale eras.

COMPUTING IN SCIENCE & ENGINEERING (2023)

Article Engineering, Electrical & Electronic

Peta-Scale Embedded Photonics Architecture for Distributed Deep Learning Applications

Zhenguo Wu, Liang Yuan Dai, Asher Novick, Madeleine Glick, Ziyi Zhu, Sebastien Rumley, George Michelogiannakis, John Shalf, Keren Bergman

Summary: SiPAC is a Silicon Photonic Accelerated Compute cluster that accelerates distributed deep learning training by utilizing embedded photonics and a novel collective algorithm. It expedites collective communications commonly used in deep learning, effectively solving the communication bottleneck problem.

JOURNAL OF LIGHTWAVE TECHNOLOGY (2023)

Proceedings Paper Computer Science, Hardware & Architecture

Efficient Intra-Rack Resource Disaggregation for HPC Using Co-Packaged DWDM Photonics

George Michelogiannakis, Yehia Arafa, Brandon Cook, Liang Yuan Dai, Abdel-Hameed (Hameed) Badawy, Madeleine Glick, Yuyang Wang, Keren Bergman, John Shalf

Summary: Resource disaggregation is a promising approach to meet the diverse workload requirements of HPC systems. By co-designing modern photonics with HPC racks, flexible intra-rack resource disaggregation can be implemented, effectively meeting the bandwidth and latency requirements of HPC applications.

2023 IEEE INTERNATIONAL CONFERENCE ON CLUSTER COMPUTING, CLUSTER (2023)

Article Computer Science, Interdisciplinary Applications

Preparing for the Future-Rethinking Proxy Applications

Satoshi Matsuoka, Jens Domke, Mohamed Wahib, Aleksandr Drozd, Andrew A. Chien, Raymond Bair, Jeffrey S. Vetter, John Shalf

Summary: The translation discusses the importance of designing proxy applications for co-designing and evaluating current generation supercomputers, and highlights the need to adapt to future heterogeneity.

COMPUTING IN SCIENCE & ENGINEERING (2022)

Proceedings Paper Computer Science, Hardware & Architecture

HyPC-Map: A Hybrid Parallel Community Detection Algorithm Using Information-Theoretic Approach

Md Abdul M. Faysal, Shaikh Arifuzzaman, Cy Chan, Maximilian Bremer, Doru Popovici, John Shalf

Summary: Community detection is crucial in graph analysis, and using an information-theoretic approach like Infomap can deliver better solutions. However, the inherently sequential nature of Infomap algorithm limits its scalability for large networks. This work introduces a hybrid parallel approach, HyPC-Map, which achieves a significant speedup of 25 times without sacrificing solution quality.

2021 IEEE HIGH PERFORMANCE EXTREME COMPUTING CONFERENCE (HPEC) (2021)

Proceedings Paper Computer Science, Software Engineering

Facilitating CoDesign with Automatic Code Similarity Learning

Tan Nguyen, Erich Strohmaier, John Shalf

Summary: Automating the workload characterization process in hardware design is crucial, especially with the increasing complexity of applications and input problems. A new approach based on code similarity learning is proposed, decomposing applications into small kernels mapped to known patterns for reusing behavior on hardware. This approach involves a new code representation and similarity metric, automated detection using compiler and machine learning methods, and distinguishing kernels in benchmarks for suggesting optimal configurations for hardware accelerators.

PROCEEDINGS OF THE SEVENTH ANNUAL WORKSHOP ON THE LLVM COMPILER INFRASTRUCTURE IN HPC (LLVM-HPC2021) (2021)

Proceedings Paper Engineering, Electrical & Electronic

Maximizing The Impact of Emerging Photonic Switches At The System Level

George Michelogiannakis, Min Yeh Teh, Madeleine Glick, John Shalf, Keren Bergman

Summary: Photonic switches have the potential to have a significant impact on future HPC and datacenter networks, but system-wide benefits cannot be achieved without changing other parts of the system architecture. Therefore, co-design between photonic switches and the rest of the system is necessary to adapt the system network and support system-wide trends.

OPTICAL INTERCONNECTS XXI (2021)

Article Computer Science, Hardware & Architecture

Temporal Computing With Superconductors

Georgios Tzimpragos, Jennifer Volk, Dilip Vasudevan, Nestan Tsiskaridze, George Michelogiannakis, Advait Madhavan, John Shalf, Timothy Sherwood

Summary: As we approach the end of CMOS transistor scaling, it is necessary to explore novel computing methods to meet the growing demands. The intersection of superconductor electronics and delay-coded computation could provide a more efficient foundation for computing.

IEEE MICRO (2021)

Article Computer Science, Interdisciplinary Applications

Large-Scale Scientific Computing in the Fight Against COVID-19

John West

Summary: U.S. computing leaders are collaborating with various entities to accelerate research on responses to COVID-19, providing resources such as some of the world's fastest computers. Efforts include understanding how the virus spreads, its biology and structure, as well as developing vaccines and antiviral treatments. The rapid mobilization of national research cyberinfrastructure highlights the importance of ongoing investments in large-scale scientific computing.

COMPUTING IN SCIENCE & ENGINEERING (2021)

Proceedings Paper Computer Science, Information Systems

TAGO: Rethinking Routing Design in High Performance Reconfigurable Networks

Min Yee Teh, Yu-Han Hung, George Michelogiannakis, Shijia Yan, Madeleine Glick, John Shalf, Keren Bergman

PROCEEDINGS OF SC20: THE INTERNATIONAL CONFERENCE FOR HIGH PERFORMANCE COMPUTING, NETWORKING, STORAGE AND ANALYSIS (SC20) (2020)

Proceedings Paper Computer Science, Hardware & Architecture

Evaluating the Numerical Stability of Posit Arithmetic

Nicholas Buoncristiani, Sanjana Shah, David Donofrio, John Shalf

2020 IEEE 34TH INTERNATIONAL PARALLEL AND DISTRIBUTED PROCESSING SYMPOSIUM IPDPS 2020 (2020)

Proceedings Paper Computer Science, Theory & Methods

Asynchronous AMR on Multi-GPUs

Muhammad Nufail Farooqi, Tan Nguyen, Weiqun Zhang, Ann S. Almgren, John Shalf, Didem Unat

HIGH PERFORMANCE COMPUTING: ISC HIGH PERFORMANCE 2019 INTERNATIONAL WORKSHOPS (2020)

No Data Available