Storage Monitoring and Forecasting at the ERDC MSRC

In December 2004, the U.S. Army Engineer Research and Development Center Major Shared Resource Center (ERDC MSRC) observed a sharp rise in data storage rates on the mass storage archival (MSA) system. Projections suggested that file storage space would be exhausted by March 2005. In January 2005, the ERDC MSRC began to monitor the top storage users and request that they reduce their file storage. By late February, consenting users had freed about 50 terabytes of space, a 15-percent reduction in overall storage. That, combined with the projected delivery of new tapes and additional capacity realized once the new Disaster Recovery site is fully populated, brings the Center out of the immediate crisis. However, the question of how to properly manage storage resources remains. With only historical data, the ERDC MSRC cannot make accurate long-term forecasts regarding data growth. To provide the optimal user environment, a mechanism must exist for incorporating project storage requirements into the growth model. At the beginning of Fiscal Year 2006, the ERDC MSRC will initiate a program for monitoring storage utilization by tracking storage usage for each project allocated computing hours by the Department of Defense High Performance Computing Modernization Program. This paper discusses the kinds of data and reporting required for storage management, considerations in adopting an implementation model, and some new user tools and best practices for file maintenance on the MSA system.