Journal
INTERNATIONAL JOURNAL OF HIGH PERFORMANCE COMPUTING APPLICATIONS
Volume 26, Issue 1, Pages 43-53Publisher
SAGE PUBLICATIONS LTD
DOI: 10.1177/1094342011428143
Keywords
Parallel I/O; Earth System Modeling; parallel computing; memory efficient; netCDF
Categories
Funding
- National Science Foundation [NSF01, OCI-0749206, CCF-0937939, OCE-0825754, AGS-0856145]
- Department of Energy, Office of Biological and Environmental Research [DE-FC02-97ER62402, DE-FC02-07ER64340]
- Climate Change Prediction Program [DE-PS02-07ER07-06]
- DOE [DE-FG02-06ER06-04]
- US Department of Energy by Lawrence Livermore National Laboratory [DE-AC52-07NA27344]
- Office of Science of the US Department of Energy [DE-AC02-06CH11357]
- Direct For Computer & Info Scie & Enginr
- Division of Computing and Communication Foundations [0937939] Funding Source: National Science Foundation
- Division Of Ocean Sciences
- Directorate For Geosciences [0825754] Funding Source: National Science Foundation
- Office of Advanced Cyberinfrastructure (OAC)
- Direct For Computer & Info Scie & Enginr [0749206] Funding Source: National Science Foundation
Ask authors/readers for more resources
We describe the design and implementation of an application-level parallel I/O (PIO) library for the reading and writing of distributed arrays to several common scientific data formats. PIO provides the flexibility to control the number of I/O tasks through data rearrangement to an I/O friendly decomposition. This flexibility enables reductions in per task memory usage and improvements in disk I/O performance versus a serial I/O approach. We illustrate the impact various features within PIO have on memory usage and disk I/O bandwidth on a Cray XT5 system.
Authors
I am an author on this paper
Click your name to claim this paper and add it to your profile.
Reviews
Recommended
No Data Available
Share Paper
