mirror of
https://git.FreeBSD.org/src.git
synced 2024-10-19 02:29:40 +00:00
8469b5aac0
Switch FIFO queues (SYNC/TRIM) and active queue of vdev queue from time-sorted AVL-trees to simple lists. AVL-trees are too expensive for such a simple task. To change I/O priority without searching through the trees, add io_queue_state field to struct zio. To not check number of queued I/Os for each priority add vq_cqueued bitmap to struct vdev_queue. Update it when adding/removing I/Os. Make vq_cactive a separate array instead of struct vdev_queue_class member. Together those allow to avoid lots of cache misses when looking for work in vdev_queue_class_to_issue(). Introduce deadline of ~0.5s for LBA-sorted queues. Before this I saw some I/Os waiting in a queue for up to 8 seconds and possibly more due to starvation. With this change I no longer see it. I had to slightly more complicate the comparison function, but since it uses all the same cache lines the difference is minimal. For a sequential I/Os the new code in vdev_queue_io_to_issue() actually often uses more simple avl_first(), falling back to avl_find() and avl_nearest() only when needed. Arrange members in struct zio to access only one cache line when searching through vdev queues. While there, remove io_alloc_node, reusing the io_queue_node instead. Those two are never used same time. Remove zfs_vdev_aggregate_trim parameter. It was disabled for 4 years since implemented, while still wasted time maintaining the offset-sorted tree of TRIM requests. Just remove the tree. Remove locking from txg_all_lists_empty(). It is racy by design, while 2 pair of locks/unlocks take noticeable time under the vdev queue lock. With these changes in my tests with volblocksize=4KB I measure vdev queue lock spin time reduction by 50% on read and 75% on write. Reviewed-by: Brian Behlendorf <behlendorf1@llnl.gov> Signed-off-by: Alexander Motin <mav@FreeBSD.org> Sponsored by: iXsystems, Inc. Closes #14925 |
||
|---|---|---|
| .. | ||
| os | ||
| sys | ||
| .gitignore | ||
| cityhash.h | ||
| libnvpair.h | ||
| libuutil_common.h | ||
| libuutil_impl.h | ||
| libuutil.h | ||
| libzfs_core.h | ||
| libzfs.h | ||
| libzfsbootenv.h | ||
| libzutil.h | ||
| Makefile.am | ||
| thread_pool.h | ||
| zfeature_common.h | ||
| zfs_comutil.h | ||
| zfs_deleg.h | ||
| zfs_fletcher.h | ||
| zfs_namecheck.h | ||
| zfs_prop.h | ||