As discussed in prior posts, KumoScale™ software implements a fast, networked block storage service that optimizes performance, utilization, and scalability for on-premise cloud infrastructure. To more efficiently manage disaggregated flash storage at data center scale, the KumoScale software suite now includes a distributed, resilient provisioner service. This KumoScale Provisioner performs intelligent and dynamic mapping of user-specified storage volumes to physical SSDs over the data center network, utilizing the NVMe over Fabrics™ standard.
Not just deployed in standalone storage systems or shared storage for a fixed cluster of compute resources, the KumoScale Provisioner is a highly scalable gateway to an entire fleet of 10s to 1000s of KumoScale storage nodes. This service builds and tracks a data center-wide view of all allocated KumoScale storage resources, client connections, and SSD device state. It can simultaneously provide unified storage access to multiple compute clusters across the data center, even when they are managed by different provisioning tools or orchestration frameworks. And – it’s flexibility allows it to work with containerized as well as bare metal cloud-native applications.
So, how does it work, exactly? The provisioner service gathers intelligence first from a “storage class” specification from the application layer – a set of requirements for the storage volume including capacity, performance, resilience, and network access protocol (TCP or RDMA). This storage class concept was borrowed from the Kubernetes® platform. While Kubernetes is a framework for intelligently assigning containerized applications to compute resources, similarly the KumoScale Provisioner is a storage optimization service for intelligently assigning data volumes to the population of network attached storage devices. Provisioning requests are processed so KumoScale software can select the best storage node and SSD mapping to deliver the required quality of service. The volume is created and then exposed via standard REST APIs back to the application, which appear just like a local drive. The goal is to optimize workload placement across all KumoScale software nodes based on specific customer-defined policies while balancing performance, resource utilization and resiliency.
Consistent with the KumoScale theme of integrating with existing customer tools, the provisioner service uses standard APIs to allow adaptation to virtually any data center provisioning tool or orchestration framework used to manage cloud infrastructure. In a Kubernetes environment, customer pods are connected to KumoScale storage by the Container Storage Interface (CSI). A KumoScale CSI driver is built-in to the Kubernetes platform on each compute node so KumoScale software is a fully supported service for provisioning storage to containerized applications. For “bare-metal” environments, the KumoScale Provisioner is deployed on a private Kubernetes cluster typically consisting of three nodes for control plane resiliency. Ansible® playbooks or other popular automation tools may be used for KumoScale deployment automation and storage provisioning.
The provisioner service is yet another enhancement to the innovative KumoScale solution, which enables flash as a service at data center scale. This results in the dynamic allocation of exactly the right amount and type of storage for each application workload, and ensures the best performance and resource utilization of data center infrastructures.
Notes:
NVM Express, NVMe and NVMe-oF are trademarks of NVM Express, Inc.
KUBERNETES® is a registered trademark of the Linux Foundation in the United States and other countries, and is used pursuant to a license from the Linux Foundation
Disclaimer
The views and opinions expressed in this blog are those of the author(s) and do not necessarily reflect those of KIOXIA America, Inc.