Asset Details
Do you wish to reserve the book?
SPANoF: A Scalable and Performant Architecture for NVMeoF-Based Storage Disaggregation with Limited Network Resources
Hey, we have placed the reservation for you!
By the way, why not check out events that you can attend while you pick your title.
Oops! Something went wrong.
Looks like we were not able to place the reservation. Kindly try again later.
Are you sure you want to remove the book from the shelf?
SPANoF: A Scalable and Performant Architecture for NVMeoF-Based Storage Disaggregation with Limited Network Resources
Do you wish to request the book?
SPANoF: A Scalable and Performant Architecture for NVMeoF-Based Storage Disaggregation with Limited Network Resources
Please be aware that the book you have requested cannot be checked out. If you would like to checkout this book, you can reserve another copy
We have requested the book for you!
Your request is successful and it will be processed during the Library working hours. Please check the status of your request in My Requests.
Oops! Something went wrong.
Looks like we were not able to place your request. Kindly try again later.
Journal Article
SPANoF: A Scalable and Performant Architecture for NVMeoF-Based Storage Disaggregation with Limited Network Resources
2023
Overview
NVMe-over-Fabrics (NVMeoF) is expected to have high-performance and be highly scalable for disaggregating NVMe SSDs to High-Speed Network (HSN)-attached storage servers, thus the aggregated NVMe SSDs in storage servers can be elastically allocated to remote host servers for better utilization. However, due to the well-known connection scalability issue of RDMA NICs (RNICs), RDMA-enabled HSN can only provide a limited scale of performant Queue Pairs (QPs) for NVMeoF I/O queues to transfer capsule and data between the storage server and remote host servers. However, in current NVMeoF implementations, multiplexing multiple NVMeoF I/O queues onto a single RNIC QP is not supported yet. In this paper, we investigate how NVMeoF capsule and data transfers are performed efficiently over HSN with a limited number of RNIC QPs, and propose SPANoF, a Scalable and Performant Architecture for NVMe-over-Fabrics. SPANoF dissolves the intrinsic one-to-one mapping relationship between NVMeoF I/O queues and RNIC QPs, allocates a dedicated send-list for each NVMeoF I/O queue rather than for each RNIC QP, transfers NVMeoF capsules and data in send-lists with a QP-centric manner to remove lock-contention overhead, and polls for transfer completion notifications to remove interrupt-caused context switch overhead. We implemented SPANoF in the Linux kernel and evaluated it by the FIO benchmarks. Our experimental results demonstrate that SPANoF can avoid the performance collapses for commercial RNICs with a limited number of performant QPs and avoid the system crash for domain-specific RNICs with only limited-scale available QPs. Compared with the native NVMeoF implementation in Linux kernel, SPANoF can saturate an RNIC of the storage server with only three RNIC QPs of the remote host server. Compared with lock-based QP-sharing mechanisms, SPANoF improves bandwidth by up to 1.55× under 64 KB sequential write requests, improves throughput by up to 4.18× and reduces the average latency by 28.31% under 4 KB random read requests.
Publisher
MDPI AG
