From Kioxia’s vantage point, AI is reshaping nearly every aspect of infrastructure design. The scale and synchronization of modern AI workloads introduce demands that differ fundamentally from traditional enterprise computing, and these demands are increasingly visible in how storage, networking, power, and cooling systems are architected.
AI Training Drives New Storage Demands
One of the most immediate impacts of AI is the scale of data required to train modern models. Training workloads rely on vast datasets that must be processed efficiently by GPUs, driving demand for large-capacity storage devices. In addition, requirements around data retention signal that storage needs extend well beyond raw training input.
At the same time, the relative value of infrastructure components is shifting. Storage acquisition once dominated data center capital planning, but GPUs have become the most valuable resource in AI systems. This shift places new pressure on storage bandwidth and network infrastructure to keep GPUs operating at high utilization.
Reliability also becomes more challenging at scale. Large GPU clusters are prone to frequent hardware failures, which makes regular checkpointing essential. Training jobs can run for several months using tens of thousands of GPUs, and without frequent checkpoints, failures would result in significant lost progress. Checkpoint restore is a synchronous operation across the cluster, making latency critical. A single slow node can idle the entire cluster. As an example, Llama™ 3 was trained on approximately 16,000 GPUs over five months and experienced hardware failures on average every three hours.
Power, Cooling, and Efficiency Take Center Stage
GPUs consume far more power than traditional compute servers, and dense GPU deployments introduce substantial cooling challenges. Liquid cooling, once largely confined to supercomputing environments, is now becoming a requirement for the latest GPU systems.
As SSD performance continues to increase, attention is also turning toward advanced cooling methods for storage, including cold-plate cooling solutions. These developments reflect a broader challenge: there are practical limits to how much power can be delivered to large data centers. In response, hyperscale cloud service providers are exploring alternatives such as private nuclear power generation.
In environments where total available power is constrained, efficiency becomes a zero-sum game. Improving power efficiency in storage devices directly benefits GPU deployment. Because SSDs can scale capacity alongside performance, deploying fewer, higher-capacity devices can reduce power consumption and free up budget that can be reallocated to GPUs.
Retrieval Augmented Generation Expands the Data Footprint
Another important trend is the growing adoption of retrieval augmented generation (RAG). This approach allows publicly available large language models to be used in conjunction with private datasets and newly created data. By supplying relevant context at inference time, RAG reduces hallucinations and delivers more accurate and timely results.
Implementing RAG requires creating embeddings and storing them in vector databases. These additional data structures significantly increase overall storage requirements, often multiplying the size of the original datasets. As with all databases, low latency and fast access are beneficial, particularly at scale. Power efficiency remains a recurring concern, as improved efficiency allows more GPUs to be supported within constrained power budgets.
Operational Effects of Large-Scale AI Workloads
From an operational standpoint, large AI deployments behave very differently from traditional data center workloads. Conventional environments host a mix of applications with diverse compute and storage patterns, resulting in a blended average utilization.
By contrast, AI workloads operate in large, tightly synchronized GPU clusters, often numbering in the tens or hundreds of thousands. These clusters generate massive, coordinated bursts of activity that place significant strain on power distribution, networking, and storage systems. At synchronization points, the entire cluster waits for the slowest node to complete its task, making rapid detection and resolution of issues critical. Monitoring and fleet management therefore become even more important than in prior generations of data centers.
Cost and Infrastructure Implications
While the full operational cost impact may not be visible from a supplier perspective, the emphasis on power efficiency and advanced cooling is a clear indicator that AI workloads are significantly affecting operating costs. Power delivery and cooling infrastructure are becoming central considerations in data center economics.
The Risk to Legacy Data Centers
Older data centers face increasing risk of obsolescence. Reference rack designs for modern AI systems now require power densities that would have been considered unrealistic just a few years ago. Combined with the shift toward liquid cooling, these requirements often necessitate major facility upgrades that may not be feasible in legacy environments.
Looking Forward
AI is transforming data centers in fundamental ways. As workloads grow in scale, complexity, and data intensity, demand for advanced storage solutions will continue to rise. Kioxia’s innovations in flash memory, SSDs, and software will play a key role in addressing these challenges and supporting the continued evolution of AI infrastructure.
Llama is a trademark of Meta Platforms, Inc.
All other company names, product names and service names may be trademarks of their respective companies.