Picture a world where flash storage stores data in the exact format used by applications. This is the promise of key-value flash media, a concept embraced by major drive manufacturers, researchers, startups, and the NVMe Key-Value command set.
While there is limited availability of productized key-value storage solutions, there has been significant research in this area. The development of a key-value command set in NVMe is a notable advancement.
In 2019, Samsung introduced a prototype key-value drive, which led to the creation of Stellus, a spin-out company with plans for a key-value storage array. However, the current status of this project is unknown. More recently, startup QiStor has expressed intentions to commercialize storage software and FPGA chips for key-value storage, identifying a potentially lucrative market opportunity.
The key idea behind key-value storage is that storing data in this format results in superior efficiency, speed, energy efficiency, and durability compared to traditional I/O processes.
Currently, applications and hosts must convert storage I/O into logical block addressing (LBA) at the drive level to locate data during read/write operations. This method dates back to the early days of spinning disk hard drives and has been recognized for its inefficiencies by key-value storage developers.
Key-value storage eliminates the need for translation through the host file system and LBA addressing. It allows applications to communicate directly with the storage media, managing data placement and retrieval internally without involving the OS or file system.
Key-value storage is a prevalent method used in various applications, including JSON format, the etcd datastore in Kubernetes, and NoSQL databases. In this model, keys represent variable names, while values represent their corresponding values. Keys and values can vary in length and data type, and can even be nested within each other.
