Does Your Storage Solution Keep Up with AI Workloads? Use MLPerf Storage to Find Out

In today’s world of machine learning and artificial intelligence, there are many varied solutions and models available to tackle and solve various business problems. However, to benchmark underlying systems and hardware that run these advanced algorithms is difficult. Although there are some benchmarks available from various organizations to test hardware resources, like CPUs and GPUs, there are few benchmarks that are able to characterize storage performance requirements, especially as they relate to these compute resources.

The requirements for keeping modern GPUs constantly utilized are challenging. Traditional training requires high capacity to hold large datasets that can include text, high resolution images and videos. In addition, regular checkpointing is also required to ensure that model training is not interrupted in case of failures in hardware and software. Failures in model training without checkpointing require models to be trained from the beginning, which can be expensive for both time and compute resources. Checkpointing also allows for model experimentation in testing different versions of models to determine which weight parameters are the most successful at a given task. Storage subsystems must be able to maintain high read and write throughputs to service many GPU threads concurrently, where each thread is serviced quickly and maintains low latency from the storage backend.

MLCommons^® for machine learning benchmarking seeks to fill that gap by providing an industry-standard benchmark called MLPerf^® Storage. This benchmark specifically tests storage subsystems for their readiness to not only take on the demands of training models using diverse datasets, but also to support the ability to benchmark storage for other aspects of the training process such as checkpointing. The MLPerf Storage v2.0 benchmark was recently released to model a storage subsystem’s capabilities of performing training on three different training datasets (3D U-Net, ResNet-50, CosmoFlow), as well as performing checkpointing on various models with  differing total number of parameters, varying anywhere from 8 billion to 1 trillion.

KIOXIA addresses customer needs through our technology leadership in flash memory and SSDs, supported by a broad product portfolio. KIOXIA SSDs are built using BiCS FLASH^™ 3D flash memory, delivering optimized solutions for the most demanding ML/AI workloads.

“At MLCommons, we are excited to see companies like KIOXIA certify their MLPerf Storage benchmark results and demonstrate their commitment to the AI industry,” said David Kanter, Head of MLPerf, Board member, MLCommons Association. “The KIOXIA CM7 Series NVMe SSD showed great performance under a variety of benchmark conditions and workloads, and we look forward to see more of their high-performance SSDs benchmark validated.”

KIOXIA CM7-R Series Enterprise E3.S NVMe™ SSDs are read intensive SSDs optimized to support a broad range of applications including ML/AI. Built on PCIe^® 5.0 and NVMe 2.0 technologies, the KIOXIA CM7 Series E3.S NVMe SSDs offer high-capacities up to 15.36 TB and deliver excellent performance of up to 14,000 MB/s (for sequential read) and up to 6,750 MB/s (for sequential write).

 One 7.68 TB KIOXIA CM7-R Series E3.S NVMe SSD was utilized and demonstrated the following results:

• Serviced four (4) H100 GPUs concurrently showcasing read bandwidths on average of 11.1025 GiB/s for the 3D U-Net dataset training benchmark
• Serviced sixty (60) H100 GPUs concurrently showcasing read bandwidths on average of 10.4597 GiB/s for the ResNet-50 dataset training benchmark.
• Serviced fourteen (14) H100 GPUs concurrently showcasing read bandwidths on average of 9.48793 GiB/s for the CosmoFlow dataset training benchmark.
• Loading checkpoints with an average throughput of 6.6066 GiB/s and saving checkpoints with an average throughput of 7.61969 GiB/s for the Llama^™ 3 8B model.

 
Result verified by MLCommons Association 

The full MLPerf Storage v2.0 benchmark guidelines from MLCommons used for the machine learning benchmarking, and for our conducted tests, are available here. 

Notes:

All other company names, product names and service names may be trademarks of their respective companies.

MLCommons and MLPerf, are registered trademarks of MLCommons Association.

Llama is a trademark of Meta.Read and write speed may vary depending on various factors such as host devices, software (drivers, OS etc.), and read/write conditions.

Performance is preliminary and subject to change without notice.

Approved images may differ from the actual product and services.

Definition of capacity: KIOXIA Corporation defines a kilobyte (KB) as 1,000 bytes, a megabyte (MB) as 1,000,000 bytes, a gigabyte (GB) as 1,000,000,000 bytes,a terabyte (TB) as 1,000,000,000,000 bytes, and a kibibyte (KiB) is 1,024 bytes. A computer operating system, however, reports storage capacity using powers of 2 for the definition of 1GB = 2^30 bytes = 1,073,741,824 bytes and 1TB = 2^40 bytes = 1,099,511,627,776 bytes and therefore shows less storage capacity. Available storage capacity (including examples of various media files) will vary based on file size, formatting, settings, software and operating system, and/or pre-installed software applications, or media content. Actual formatted capacity may vary.

NVMe is a registered or unregistered trademark of NVM Express, Inc. in the United States and other countries.

PCIe is a registered trademark of PCI-SIG.