1
0
mirror of https://git.FreeBSD.org/src.git synced 2024-12-18 10:35:55 +00:00
freebsd/sys/geom
Steven Hartland c28078e903 Improve ZFS N-way mirror read performance by using load and locality
information.

The existing algorithm selects a preferred leaf vdev based on offset of the zio
request modulo the number of members in the mirror. It assumes the devices are
of equal performance and that spreading the requests randomly over both drives
will be sufficient to saturate them. In practice this results in the leaf vdevs
being under utilized.

The new algorithm takes into the following additional factors:
* Load of the vdevs (number outstanding I/O requests)
* The locality of last queued I/O vs the new I/O request.

Within the locality calculation additional knowledge about the underlying vdev
is considered such as; is the device backing the vdev a rotating media device.

This results in performance increases across the board as well as significant
increases for predominantly streaming loads and for configurations which don't
have evenly performing devices.

The following are results from a setup with 3 Way Mirror with 2 x HD's and
1 x SSD from a basic test running multiple parrallel dd's.

With pre-fetch disabled (vfs.zfs.prefetch_disable=1):

== Stripe Balanced (default) ==
Read 15360MB using bs: 1048576, readers: 3, took 161 seconds @ 95 MB/s
== Load Balanced (zfslinux) ==
Read 15360MB using bs: 1048576, readers: 3, took 297 seconds @ 51 MB/s
== Load Balanced (locality freebsd) ==
Read 15360MB using bs: 1048576, readers: 3, took 54 seconds @ 284 MB/s

With pre-fetch enabled (vfs.zfs.prefetch_disable=0):

== Stripe Balanced (default) ==
Read 15360MB using bs: 1048576, readers: 3, took 91 seconds @ 168 MB/s
== Load Balanced (zfslinux) ==
Read 15360MB using bs: 1048576, readers: 3, took 108 seconds @ 142 MB/s
== Load Balanced (locality freebsd) ==
Read 15360MB using bs: 1048576, readers: 3, took 48 seconds @ 320 MB/s

In addition to the performance changes the code was also restructured, with
the help of Justin Gibbs, to provide a more logical flow which also ensures
vdevs loads are only calculated from the set of valid candidates.

The following additional sysctls where added to allow the administrator
to tune the behaviour of the load algorithm:
* vfs.zfs.vdev.mirror.rotating_inc
* vfs.zfs.vdev.mirror.rotating_seek_inc
* vfs.zfs.vdev.mirror.rotating_seek_offset
* vfs.zfs.vdev.mirror.non_rotating_inc
* vfs.zfs.vdev.mirror.non_rotating_seek_inc

These changes where based on work started by the zfsonlinux developers:
https://github.com/zfsonlinux/zfs/pull/1487

Reviewed by:	gibbs, mav, will
MFC after:	2 weeks
Sponsored by:	Multiplay
2013-10-23 09:54:58 +00:00
..
bde - Don't pass geom and provider names as format strings. 2012-11-20 12:32:18 +00:00
cache - Don't pass geom and provider names as format strings. 2012-11-20 12:32:18 +00:00
concat Merge GEOM direct dispatch changes from the projects/camlock branch. 2013-10-22 08:22:19 +00:00
eli Make ELI destruction (including orphanization) less aggressive, making it 2013-09-02 10:44:54 +00:00
gate Merge GEOM direct dispatch changes from the projects/camlock branch. 2013-10-22 08:22:19 +00:00
journal When panicing due to the gjournal overflow, print the geom metadata 2013-07-10 10:11:43 +00:00
label Fix build with gcc by spelling unused format string as "unused" instead of NULL. 2013-10-19 08:20:00 +00:00
linux_lvm
mirror Merge GEOM direct dispatch changes from the projects/camlock branch. 2013-10-22 08:22:19 +00:00
mountver - Don't pass geom and provider names as format strings. 2012-11-20 12:32:18 +00:00
multipath Merge GEOM direct dispatch changes from the projects/camlock branch. 2013-10-22 08:22:19 +00:00
nop gnop: make sure that newly allocated memory for softc is zeroed 2013-10-23 01:34:18 +00:00
part Merge GEOM direct dispatch changes from the projects/camlock branch. 2013-10-22 08:22:19 +00:00
raid Merge GEOM direct dispatch changes from the projects/camlock branch. 2013-10-22 08:22:19 +00:00
raid3 Allow to insert new component to geom_raid3 without specifying number. 2013-01-15 10:06:35 +00:00
sched - Don't pass geom and provider names as format strings. 2012-11-20 12:32:18 +00:00
shsec
stripe Merge GEOM direct dispatch changes from the projects/camlock branch. 2013-10-22 08:22:19 +00:00
uncompress Remove unneeded G_PF_CANDELETE flag. 2012-08-28 19:28:31 +00:00
uzip Remove unneeded G_PF_CANDELETE flag. 2012-08-28 19:28:31 +00:00
vinum
virstor In virstor_ctl_stop(), check for a valid softc before trying to update 2012-08-03 20:24:16 +00:00
zero Merge GEOM direct dispatch changes from the projects/camlock branch. 2013-10-22 08:22:19 +00:00
geom_aes.c - Don't pass geom and provider names as format strings. 2012-11-20 12:32:18 +00:00
geom_bsd_enc.c
geom_bsd.c
geom_ccd.c
geom_ctl.c
geom_ctl.h
geom_dev.c Merge GEOM direct dispatch changes from the projects/camlock branch. 2013-10-22 08:22:19 +00:00
geom_disk.c Improve ZFS N-way mirror read performance by using load and locality 2013-10-23 09:54:58 +00:00
geom_disk.h Improve ZFS N-way mirror read performance by using load and locality 2013-10-23 09:54:58 +00:00
geom_dump.c Remove an extra semicolon from the DOT language output. 2013-05-21 18:40:54 +00:00
geom_event.c Make g_wither_washer() to not loop by itself, but only when there was some 2013-03-24 03:15:20 +00:00
geom_flashmap.c MFP4 @222836 2013-05-30 01:19:02 +00:00
geom_fox.c
geom_int.h Merge GEOM direct dispatch changes from the projects/camlock branch. 2013-10-22 08:22:19 +00:00
geom_io.c Merge GEOM direct dispatch changes from the projects/camlock branch. 2013-10-22 08:22:19 +00:00
geom_kern.c Merge GEOM direct dispatch changes from the projects/camlock branch. 2013-10-22 08:22:19 +00:00
geom_map.c Use %j to match intmax_t. 2012-07-01 05:22:13 +00:00
geom_mbr_enc.c
geom_mbr.c - Don't pass geom and provider names as format strings. 2012-11-20 12:32:18 +00:00
geom_pc98_enc.c
geom_pc98.c Change <sys/diskpc98.h> to not redefine the same symbols that are 2013-08-07 00:00:48 +00:00
geom_redboot.c
geom_slice.c Merge GEOM direct dispatch changes from the projects/camlock branch. 2013-10-22 08:22:19 +00:00
geom_slice.h - Don't pass geom and provider names as format strings. 2012-11-20 12:32:18 +00:00
geom_subr.c Improve ZFS N-way mirror read performance by using load and locality 2013-10-23 09:54:58 +00:00
geom_sunlabel_enc.c
geom_sunlabel.c
geom_vfs.c Merge GEOM direct dispatch changes from the projects/camlock branch. 2013-10-22 08:22:19 +00:00
geom_vfs.h
geom_vol_ffs.c
geom.h Improve ZFS N-way mirror read performance by using load and locality 2013-10-23 09:54:58 +00:00
notes