1
0
mirror of https://git.FreeBSD.org/src.git synced 2024-12-24 11:29:10 +00:00
Commit Graph

708 Commits

Author SHA1 Message Date
Mark Johnston
7e00348e76 Fix a couple of bugs around the handling of structs and unions of size zero.
These would cause ctfconvert(1) to return an error when attempting to
resolve valid C types.

Reviewed by:	Robert Mustacchi <rm@joyent.com>
MFC after:	2 weeks
Sponsored by:	EMC / Isilon Storage Division
2014-11-16 04:07:53 +00:00
Glen Barber
b10cc05cf5 Fix an mdoc(7) macro that is not an option in the provided
description.

Bump Dd.

As CDDL License dictates, update the Copyright accordingly.

Sponsored by:	The FreeBSD Foundation
2014-11-12 08:36:42 +00:00
Xin LI
8bcd603968 MFV r274273:
ZFS large block support.

Please note that booting from datasets that have recordsize greater
than 128KB is not supported (but it's Okay to enable the feature on
the pool).  This *may* remain unchanged because of memory constraint.

Limited safety belt is provided for mounted root filesystem but use
caution is advised.

Illumos issue:
    5027 zfs large block support

MFC after:	1 month
2014-11-10 08:20:21 +00:00
Xin LI
42350b6bde MFV r274272 and diff reduction with upstream.
Illumos issue:
    5244 zio pipeline callers should explicitly invoke next stage

Tested with:	ztest plus ZFS over GELI configuration
MFC after:	1 month
2014-11-09 07:37:00 +00:00
Xin LI
68ce9bc32d Apply upstream 13597:3eac1e8e0f4c (git: illumos-gate@aa846ad9):
Initialize tqent_flags in the userland taskq implementation.  Without
this the assertion of tq->tq_freelist != NULL may fail in taskq_destroy.

The problem is that tqent_flags is never initialized in the userland
implementation while the kernel one does initialize it.  Without proper
initialization, the flag may have its lowest bit set, making it treated
as TQENT_FLAG_PREALLOC and never removing taskq_ent_t from tq_freelist.

MFC after:	2 weeks
2014-11-09 01:42:28 +00:00
Xin LI
81f1255e58 MFV r274271:
Improve zdb -b performance:

 - Reduce gethrtime() call to 1/100th of blkptr's;
 - Skip manipulating the size-ordered tree;
 - Issue more (10, previously 3) async reads;
 - Use lighter weight testing in traverse_visitbp();

Illumos issue:
    5243 zdb -b could be much faster

MFC after:	2 weeks
2014-11-08 07:30:40 +00:00
Mark Johnston
fd8b318ab6 Correct the calculation of tcps_rto in the struct tcpcb -> tcpsinfo_t
translator.

Submitted by:	Grenville Armitage <garmitage@swin.edu.au>
MFC after:	1 week
2014-10-21 05:19:08 +00:00
George V. Neville-Neil
bd537a28cc Update the TCP structure used by DTrace to show the smoothed RTT.
This will allow similar functionality to SIFTR to be built with DTrace.

Submitted by:	Grenville Armitage
MFC after:	2 weeks
2014-10-19 23:13:16 +00:00
Mark Johnston
028d58547d Remove a build artifact of the USDT tests.
MFC after:	3 days
2014-10-15 03:15:46 +00:00
Mark Johnston
9fb83be0d9 Document the CPU+Pri_Class column rather than CPU+PRI, as the latter isn't
used or implemented on FreeBSD.

Reported by:	kmacy
Reviewed by:	kmacy, rpaulo
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2014-10-09 17:45:58 +00:00
Xin LI
15f3c56e3d MFV r272802:
- Limit ARC for zdb at 256MB.  zdb do not typically revisit data
   in the ARC.
 - Increase default max_inflight from 200 to 1000 (can be overriden
   by -I) so we can queue more I/Os when doing scrubbing.
 - Print status while loading meataslabs for leak detection.

Illumos issues:

    5169 zdb should limit its ARC size
    5170 zdb -c should create more scrub i/os by default
    5171 zdb should print status while loading metaslabs for leak detection

MFC after:	2 weeks
2014-10-09 06:02:53 +00:00
Mark Johnston
84e874bd6c Treat D keywords as identifiers in certain postfix expressions. This allows
one to, for example, access the "provider" field of a struct g_consumer,
even though "provider" is a D keyword.

PR:		169657
MFC after:	2 months
Discussed with:	Bryan Cantrill
Sponsored by:	EMC / Isilon Storage Division
2014-10-06 21:52:40 +00:00
Xin LI
9ee8d90af5 MFV r272588:
Handle old format deadlist.

Illumos issue:
    5178 zdb -vvvvv on old-format pool fails in dump_deadlist()

MFC after:	2 weeks
2014-10-06 07:09:47 +00:00
Xin LI
8fb26f5aef MFV r272585:
Split the godfather zio into CPU number's to reduce lock
contention.

Illumos issue:
    5176 lock contention on godfather zio

MFC after:	2 weeks
2014-10-06 07:03:17 +00:00
Xin LI
7c2f66bb3b MFV r272493:
Show individual disk capacity when doing zpool list -v.

Illumos issue:
    5147 zpool list -v should show individual disk capacity

MFC after:	1 week
2014-10-04 07:56:50 +00:00
Mark Johnston
41da933c87 Hook up support for userland CTF support in DTrace. This required some
modifications to libproc to support fetching the CTF info for a given file.

With this change, dtrace(1) is able to resolve type info for function and
USDT probe arguments, and function return values. In particular, the args[n]
syntax should now work for referencing arguments of userland probes,
provided that the requisite CTF info is available.

The uctf tests pass if the test programs are compiled with CTF info. The
current infrastructure around the DTrace test suite doesn't support this
yet.

Differential Revision:	https://reviews.freebsd.org/D891
MFC after:		1 month
Relnotes:		yes
Sponsored by:		EMC / Isilon Storage Division
2014-10-03 23:20:37 +00:00
Xin LI
c36047bdb7 Add dependencies to various libraries to libzfs and libzpool.
Submitted by:	sef
2014-10-03 20:36:09 +00:00
Mark Johnston
486de25d46 Have dtrace(1) handle SIGPIPE by cleaning up and exiting. Additionally,
install signal handlers when running in list mode (-l), and acknowledge
interrupts by cleaning up and exiting. This ensures that a command like

$ dtrace -l -P 'pid$target' -p <target PID> | less

won't cause the ptrace(2)d target process to be killed if less(1) exits
before all dtrace output is consumed.

Reported by:		Anton Yuzhaninov <citrin+bsd@citrin.ru>
Differential Revision:	https://reviews.freebsd.org/D880
Reviewed by:		rpaulo
MFC after:		1 month
Sponsored by:		EMC / Isilon Storage Division
2014-10-02 22:33:35 +00:00
Xin LI
456a4e16a6 Revert r272189, the committed version was an old version and breaks build.
Pointy hat to:	delphij
2014-09-27 09:39:19 +00:00
Xin LI
0b03875cc2 Add libuutil to dependency list.
Noticed by:	sef
MFC after:	3 days
2014-09-26 21:46:32 +00:00
Mark Johnston
946497ec23 Fix the description of the -h option.
Reported by:	swills
MFC after:	3 days
2014-09-26 07:01:27 +00:00
Rui Paulo
881417d74f Build plockstat if WITH_PLOCKSTAT is defined. 2014-09-23 18:22:40 +00:00
Mark Johnston
527e468fc9 Remove an incorrect close(2) call that was added in r271413.
Reported by:	Coverity
CID:		1238923
X-MFC-With:	r271413
2014-09-22 21:09:30 +00:00
Mark Johnston
f0de463732 Remove some variables that are no longer used as of r271413.
Reported by:	Coverity
CID:		1238924
X-MFC-With:	r271413
2014-09-22 21:02:38 +00:00
Steven Hartland
137a165dd3 Output boot code warning when zpool upgrade -a is used to add features.
In the case where new features where enabled by a zpool upgrade -a the
boot code warning wasn't output.

Submitted by:	Jan Kokemueller
MFC after:	3 days
2014-09-21 19:31:19 +00:00
Will Andrews
c63a110d6d zfs_setprop_error(): Handle errno value E2BIG.
This errno value is emitted by dsl_props_set_check() in
sys/cddl/contrib/opensolaris/uts/common/fs/zfs/dsl_prop.c, and
is used to mean that the property value is too long.  For the record,
the maximum length is ZAP_MAXVALUELEN, which is 8*1024 bytes.

Instead of claiming an unknown error (and abort()ing), provide
something more specific to the scenario involved.  As far as I
can tell, E2BIG is not emitted for any other scenario.

MFC after:	1 week
Sponsored by:	Spectra Logic
Affects:	All ZFS versions starting 27 Feb 2009 (illumos ccba0801)
		This change modified the value returned by
		dsl_props_set_check(), so that it can distinguish between
		a name that's too long and a value that's too long, but
		libzfs was not updated accordingly.
MFSpectraBSD:	r1051499 on 2014/03/28 11:07:59
2014-09-18 14:09:42 +00:00
Will Andrews
abf6a03921 Fix an assert to tolerate spare parents with more than 2 children.
This can occur if a spare is being spared, which would yield three
children: the original pool drive, the previous spare, and the spare
that is replacing it.

MFC after:	1 week
Sponsored by:	Spectra Logic
Affects:	All ZFS versions starting 7 Jun 2006 (illumos 94de1d4c)
MFSpectraBSD:	r668345 on 2013/06/04 17:10:43
2014-09-18 14:02:25 +00:00
Mark Johnston
dab6e14240 Implement a workaround to allow this test program to be compiled with clang.
It seems that if a pragma is used to define a weak alias for a local
function, the pragma must appear after the function is defined.

PR:		193056
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2014-09-17 04:02:56 +00:00
Mark Johnston
583b13f588 Fix a number of typos and programming errors in the userland CTF tests. It
seems that they would only pass by chance on illumos; on FreeBSD, they still
fail since userland CTF is not yet supported.

MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2014-09-17 03:40:49 +00:00
Mark Johnston
6574b8ed19 Fix some incorrect endianness checks.
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2014-09-17 00:54:00 +00:00
Mark Johnston
a5ddd965e0 Fix elfdump(1) usage in the userland CTF tests.
PR:		193110
MFC after:	1 week
Sponsored by:	EMC / Isilon Storage Division
2014-09-16 22:54:15 +00:00
Xin LI
8fb2117122 MFV r271516:
Enable debug printf's when ZFS_DEBUG or debug= is set.

Illumos issue:

    5134 if ZFS_DEBUG or debug= is set, libzpool should enable debug prints

MFC after:	2 weeks
2014-09-13 17:30:46 +00:00
Xin LI
e11a25e8d0 MFV r271511:
Use fnvlist_* to make code more readable.

Illumos issue:
    5135 zpool_find_import_cached() can use fnvlist_*

MFC after:	2 weeks
2014-09-13 16:41:39 +00:00
Mark Johnston
8caf8a8d0e Use the linker to perform relocations in the SUNW_dof section rather than
doing them in drti during startup. This fixes a number of problems with
using USDT probes in stripped executables and shared libraries, and with
USDT probes in static functions.

Reviewed by:	rpaulo
MFC after:	1 month
Sponsored by:	EMC / Isilon Storage Division
Phabric: 	D751
2014-09-11 01:04:56 +00:00
Xin LI
36d45b79de MFV r271225:
Iterate through all the children instead of returning error when we hit
the first error.  This makes the error message give more information
rather than just the first device that causes problem.

Illumos issue:
    5118 When verifying or creating a storage pool, error messages only
	 show one device

MFC after:	2 weeks
2014-09-07 13:22:14 +00:00
Xin LI
8b1bab5161 Fix typo.
Submitted by:	Dmitry Morozovsky <marck rinet ru>
MFC after:	3 days
2014-09-07 11:57:08 +00:00
Xin LI
fa4484104c MFV r270197:
Illumos issue:
    5066 remove support for non-ANSI compilation
    5068 Remove SCCSID() macro from <macros.h>

MFC after:	2 weeks
2014-08-22 22:13:36 +00:00
Xin LI
4ddb46f694 Make DTrace stuff compile with C99 standard. 2014-08-22 20:04:51 +00:00
Xin LI
eb86794485 Fix powerpc build:
Chase r270227 and compile lockstat with C99 standard.

Suggested by:	bde
2014-08-22 01:23:38 +00:00
Xin LI
600080d8a5 Include two headers to provide prototype for modfind(2) and kldload(2).
MFC after:	2 weeks
2014-08-21 22:53:14 +00:00
Simon J. Gerraty
ee7b0571c2 Merge head from 7/28 2014-08-19 06:50:54 +00:00
Rui Paulo
40cc45e7c8 Remove the BROKEN_LIBELF section.
This problem was fixed by Kai Wang in 2011.

MFC after:	3 days
2014-08-10 06:56:43 +00:00
Sean Bruno
20c74f8b4a Quiesce a printf warning from clang, %ul -> %lu
Phabric:	https://phabric.freebsd.org/D472
Reviewed by:	mahrens delphij
2014-08-08 18:00:23 +00:00
Mark Johnston
47890cb590 Preserve the errno value of an ioctl before calling free(3). Previously,
errno was very occasionally being clobbered, resulting in a bogus error from
dt_consume() and thus an error from dtrace(1).

MFC after:	2 weeks
2014-08-04 18:52:26 +00:00
Xin LI
de5edb1245 MFV r269426:
Double test device size for ztest(1).

Illumos issue:
    5039 ztest should default to larger device sizes
    Author: Matthew Ahrens <mahrens@delphix.com>

MFC after:	2 weeks
2014-08-02 07:47:52 +00:00
George V. Neville-Neil
e1c97af45e Update shellsnoop to work on FreeBSD.
Contributed by: skreuzer
2014-07-31 23:19:01 +00:00
Xin LI
a3cbca537e MFV r269223:
Change dn->dn_dbufs from linked list to AVL tree.

Illumos issues:
  4873 zvol unmap calls can take a very long time for larger datasets

MFC after:	2 weeks
2014-07-29 08:42:22 +00:00
George V. Neville-Neil
9971b08e4c Update the header printing to match the rest of the output.
Pointed out by: mdf
2014-07-26 20:42:54 +00:00
George V. Neville-Neil
0100ace8a9 Since we cannot yet display the name print the file descriptor so that
it can be looked up manually using procstat(1).
2014-07-26 20:11:36 +00:00
George V. Neville-Neil
d9a499ce10 Remove debugging options from the rwsnoop script. 2014-07-26 19:25:52 +00:00
George V. Neville-Neil
69d22158da Port the rwsnoop DTrace Toolkit script to FreeBSD.
Remove dependency on the Korn Shell.
Remove Zones in favor of Jails.
Remove support (for now) for filename printing.
2014-07-26 19:21:53 +00:00
Xin LI
7e37b1e609 MFV r269010:
Import Illumos changes to address the following Illumos issues:
  4976 zfs should only avoid writing to a failing non-redundant
       top-level vdev
  4978 ztest fails in get_metaslab_refcount()
  4979 extend free space histogram to device and pool
  4980 metaslabs should have a fragmentation metric
  4981 remove fragmented ops vector from block allocator
  4982 space_map object should proactively upgrade when feature
       is enabled
  4984 device selection should use fragmentation metric

MFC after:	2 weeks
2014-07-26 10:20:48 +00:00
Xin LI
e36b1c7478 Diff reduction against Illumos.
MFC after:	2 weeks
2014-07-25 22:58:55 +00:00
Xin LI
7882b61f60 MFV r268848:
Instead of asserting all zio's be properly aligned, only assert
on the logical ones.

Cap uberblocks at 8k, otherwise with ashift=17, there would be
only one uberblock.

This fixes a problem that zdb would trip assert on pools with
ashift >= 0xe (8k).

While there, also change the code so it only attempt to condense
space map unless the uncondensed size consumes greater than
zfs_metaslab_condense_block_threshold blocks.

Illumos issue:
  4958 zdb trips assert on pools with ashift >= 0xe

MFC after:	2 weeks
2014-07-18 20:41:40 +00:00
Xin LI
7079d5877c MFV r268714:
Improve extreme rewind import.

When doing an "extreme rewind" import ("zpool import -XF"), we attempt
to verify all data in the pool, essentially scrubbing the entire pool.
The problem is that spa_load_verify_cb() issues an unbounded number of
concurrent scrub i/os.  This can lead to all of memory being used for
these zio's, wedging the system. Like normal scrub, we need to put a
cap on the number of outstanding i/os, and have the traverse thread
block when we reach this cap.

For this purpose the cap can be very large (10,000) to optimize the
elevator algorithm.  Three kernel tunables have been added:

	vfs.zfs.spa_load_verify_maxinflight
	vfs.zfs.spa_load_verify_metadata
	vfs.zfs.spa_load_verify_data

The latter two tunables controls whether metadata and/or user data
when doing extreme rewind.

Make 'zpool import -T' imply scrub.

Make zpool import -T <txg> accept hexadecimal values for the txg when
prefixed with 0x.

Skip txg's for which there is no uberblock when doing extreme rewind.

Skip reading all user data twice by skipping prefetches when doing
extreme rewinds as we do not access via the ARC.

Illumos issues:
  4970 need controls on i/o issued by zpool import -XF
  4971 zpool import -T should accept hex values
  4972 zpool import -T implies extreme rewind, and thus a scrub
  4973 spa_load_retry retries the same txg
  4974 spa_load_verify() reads all data twice

MFC after:	2 weeks
2014-07-15 22:44:04 +00:00
Xin LI
6066807a42 Bump mdoc date after r268621.
X-MFC-With:	r268621
2014-07-14 17:54:36 +00:00
Steven Hartland
d143a6f700 Don't report non-native block-size pools under zpool status -x
zpool status -x is used to identify pools that are exhibiting
errors or are otherwise unavailable, therefore non-native
block-size pools shouldn't be reported.

Also update man page to clarify other additional conditions
which won't cause a pool to be displayed under zpool status -x.

Sponsored by:	Multiplay
2014-07-14 14:33:03 +00:00
Xin LI
1b174fa1eb MFV r268455:
Use reserved space for ZFS administrative commands.

We reserve 1/2^spa_slop_shift = 1/32 or 3.125% of pool space (or 32MB at
least) for system use.  Most ZPL operations, e.g. write(2), creat(2), will
fail with ENOSPC if we fall below this.

Certain operations, e.g. file removal and most administrative actions,
still permitted until half of the slop space is used.  This would allow
users to use these operations to free up space in the pool when pool is
close to full but half of slop space is still free.

A very restricted set of operations that frees up space or change quota
are always permitted, regardless of the amount of free space.

MFC after:	 2 weeks
2014-07-09 23:14:59 +00:00
Xin LI
b1396c9f98 MFV r268454:
Refresh zpool list for each interval in order to produce fresh
output.

Illumos issue: 4966 zpool list iterator does not update output

MFC after:	 2 weeks
2014-07-09 21:07:20 +00:00
Xin LI
ad9b19c1e8 MFV r268453:
Diff reduction against Illumos.

MFC after:	 2 weeks
2014-07-09 20:57:42 +00:00
Marcel Moolenaar
e7d939bda2 Remove ia64.
This includes:
o   All directories named *ia64*
o   All files named *ia64*
o   All ia64-specific code guarded by __ia64__
o   All ia64-specific makefile logic
o   Mention of ia64 in comments and documentation

This excludes:
o   Everything under contrib/
o   Everything under crypto/
o   sys/xen/interface
o   sys/sys/elf_common.h

Discussed at: BSDcan
2014-07-07 00:27:09 +00:00
Xin LI
9cc8a15b2e MFV r268121:
4924 LZ4 Compression for metadata

illumos/illumos-gate@b8289d24d8

MFC after:	2 weeks
2014-07-01 22:31:09 +00:00
Xin LI
aa882b9048 MFV r268119:
4914 zfs on-disk bookmark structure should be named *_phys_t

illumos/illumos-gate@7802d7bf98

MFC after:	2 weeks
2014-07-01 21:51:30 +00:00
Xin LI
55f6421982 - Fix handling of "new" style of ioctl in compatiblity mode [1];
- Reorganize code and reduce diff from upstream;
 - Improve forward compatibility shims for previous kernel;

Reported by:	sbruno [1]
X-MFC-With:	r268075
2014-07-01 20:57:39 +00:00
Xin LI
be78a8db97 MFV r267570:
4756 metaslab_group_preload() could deadlock

illumos/illumos-gate@30beaff42d

MFC after:	2 weeks
2014-07-01 08:36:56 +00:00
Xin LI
6bab9dd0e4 MFV r267568:
4891 want zdb option to dump all metadata

illumos/illumos-gate@df15e419cb

MFC after:	2 weeks
2014-07-01 08:20:34 +00:00
Xin LI
71eaf0fda7 MFV r267566:
4390 i/o errors when deleting filesystem/zvol can lead to space map corruption

MFC after:	2 weeks
2014-07-01 07:29:42 +00:00
Xin LI
29441ba3fa MFV r267565:
4757 ZFS embedded-data block pointers ("zero block compression")
4913 zfs release should not be subject to space checks

MFC after:	2 weeks
2014-07-01 06:43:15 +00:00
Rui Paulo
a43f0be9fe MFV illumos
4471 DTrace count() with histogram
4472 DTrace full width distribution histograms
4473 DTrace frequency trails

MFC after:	2 weeks
2014-06-26 23:24:59 +00:00
Rui Paulo
8e648814b0 MFV illumos
4474 DTrace Userland CTF Support
4475 DTrace userland Keyword
4476 DTrace tests should be better citizens
4479 pid provider types
4480 dof emulation is missing checks

MFC after:	2 weeks
2014-06-26 23:21:11 +00:00
Rui Paulo
09779aae33 Add stubs for CTF functions which are not yet implemented.
MFC after:	2 weeks
2014-06-26 22:38:06 +00:00
Rui Paulo
b1f9167f94 MFV illumos
4477 DTrace should speak JSON

MFC after:	2 weeks
2014-06-26 21:45:49 +00:00
Rui Paulo
0c2b601953 MFV illumos r266986:
2915 DTrace in a zone should see "cpu", "curpsinfo", et al
2916 DTrace in a zone should be able to access fds[]
2917 DTrace in a zone should have limited provider access

MFC after:	2 weeks
2014-06-26 19:38:16 +00:00
Rui Paulo
dd9b2abed8 Revert r267898. 2014-06-26 17:34:42 +00:00
Rui Paulo
d8e37c5f72 Bring the following change from the illumos-joyent repository:
commit 78e24ab6803bbe11ba37642624e1498ede5b239d
Author: Bryan Cantrill <bryan@joyent.com>
Date:   Thu Oct 31 01:20:54 2013

    OS-1688 DTrace count() with histogram
    OS-2360 DTrace full width distribution histograms
    OS-2361 DTrace frequency trails

MFC after:	2 weeks
2014-06-26 07:06:43 +00:00
Pedro F. Giffuni
fcb8a88918 MFV r258381:
4251 libdtrace leaks open file handles

Illumos commit:		93ed8d0d4b068b95d0bb50d57bb854df462a8485
			(partial)
Reference:
https://www.illumos.org/issues/4251

Discussed with:	Robert Mustacchi
Obtained from:	Illumos
MFC after:	1 week
2014-06-25 17:27:15 +00:00
Joel Dahl
df2d82e003 mdoc: remove superfluous paragraph macros. 2014-06-23 18:40:21 +00:00
Xin LI
51040796d0 MFV r249332 (illumos-gate 14005:55fc53126003)
Illumos ZFS issues:
  3654 zdb should print number of ganged blocks

MFC after:	2 weeks
2014-06-17 08:11:45 +00:00
Pedro F. Giffuni
1bf222cc83 MFV r266988:
Merge from r258379 missed the tests.

4248 dtrace(1M) should never create DOF with empty probes section
4249 Only probes from the first DTrace object file will be included

Illumos Revision:	54a20ab41aadcb81c53e72fc65886e964e9add59

MFC after:	5 days
2014-06-15 16:54:26 +00:00
Xin LI
2bdf7f79bc MFV r266766:
Add a new zfs property, "redundant_metadata" which can have values "all" or
"most".  The default will be "all", which is the current behavior.  When set
to all, ZFS stores an extra copy of all metadata.  If a single on-disk block
is corrupt, at worst a single block of user data (which is recordsize bytes
long) can be lost.

Setting to "most" will cause us to only store 1 copy of level-1 indirect
blocks of user data files.  This can improve performance of random writes,
because less metadata has to be written.  In practice,  at worst about
100 blocks (of recordsize bytes each) of user data can be lost if a single
on-disk block is corrupt.

The exact behavior of which metadata blocks are stored redundantly may change
in future releases.

Illumos issue: 3835 zfs need not store 2 copies of all metadata

MFC after:	2 weeks
2014-05-27 19:46:11 +00:00
Xin LI
f3ce69748f Explicitly link libzfs against libavl as it is done in OpenSolaris
(4543:12bb2876a62e).  Without this, some third party applications
may break because the lack of AVL related symbols.

FreeBSD base system are not affected because the FreeBSD ZFS command
line tools were all linked against libavl and thus hide the underlying
issue.

PR:		java/183081
Tested by:	jkim
MFC after:	3 days
2014-05-22 00:01:31 +00:00
Mark Johnston
a69a8c422a Fix tst.ZeroModuleProbes.d.ksh, which was incorrectly modified in r178534.
Since "BEGIN" is not the name of a module, the test would just hang.

MFC after:	3 days
2014-05-19 20:11:55 +00:00
Simon J. Gerraty
fae50821ae Updated dependencies 2014-05-16 14:09:51 +00:00
Mark Johnston
2f28ceeb34 Bind ip/tcp/udp provider translators and symbols to the same versions as in
illumos, rather than using "1.0" everywhere.

Some of the translators use D functions that are not present in version
1.0 (e.g. inet_ntoa()) which can result in libdtrace crashing when running
scripts that restrict themselves to version 1.0
(e.g. with "-x version=1.0").

MFC after:	1 week
2014-05-14 19:02:00 +00:00
Alexander Motin
7585c68486 Comment out some pointless device open/close around reading device IDs.
FreeBSD ZFS port unlike OpenSolaris does not use device IDs, and does not
implement respective devid_*() fuctions.  It is pointless to open devices
just to close them back immediately.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2014-05-10 15:21:37 +00:00
Simon J. Gerraty
76b28ad6ab Updated dependencies 2014-05-10 05:16:28 +00:00
Simon J. Gerraty
cc3f4b9965 Merge from head 2014-05-08 23:54:15 +00:00
Alexander Motin
afb8674c79 Import adapted OpenSolaris' thread pool API implementation.
The thread pool is used by libzfs to implement parallel disk scanning.
Without this change our dummy wrapper made `zpool import ZZZ` command to
scan all disks sequentially from the single thread when searching for pools.
This change makes it use two threads per CPU, same as in OpenSolaris.

On system with 200 HDDs this change reduces ZFS pool import time from 35
to 22 seconds.
2014-05-08 16:59:36 +00:00
Mark Johnston
f0736f4cec Re-apply r248644. This fixes an annoying problem which caused dtrace -c to
fail to attach to stripped binaries. With the _r_debug_postinit symbol,
dtrace(1) can now set a breakpoint in the victim process after it has
registered its DOF table(s) with the kernel. r_debug_state cannot be used
for this purpose since it is called before DOF is made available, in which
case dtrace(1) cannot create USDT probes before the program begins
execution.

MFC after:	2 weeks
2014-05-08 03:43:18 +00:00
Warner Losh
c6063d0da8 Use src.opts.mk in preference to bsd.own.mk except where we need stuff
from the latter.
2014-05-06 04:22:01 +00:00
Mark Johnston
604457091d Remove a duplicate definition.
MFC after:	3 days
2014-05-04 03:37:39 +00:00
Simon J. Gerraty
9d2ab4a62d Merge head 2014-04-27 08:13:43 +00:00
Warner Losh
98407b8bc2 Spell NO_PROFILE= as MK_PROFILE=no. 2014-04-25 19:25:26 +00:00
Steven Hartland
fdc8132f2b Silence compiler warning due to missing return in idmap_id_to_numeric_domain_rid 2014-04-24 01:20:43 +00:00
Steven Hartland
d9dab6e243 Eliminated optarg global being used outside of the function which called getopt
MFC after:	2 weeks
2014-04-24 01:12:52 +00:00
Xin LI
f8587167e4 MFV r264829:
3897 zfs filesystem and snapshot limits

MFC after:	2 weeks
2014-04-23 20:29:46 +00:00
Julio Merino
38f0b757fd Add placeholder Kyuafiles for various top-level hierarchies.
This change adds tests/ directories in the source tree to create various
subdirectories in /usr/tests/ and to install placeholder Kyuafiles for
them.

the relevant hierarchies are: cddl, etc, games, gnu and secure.

The reason for this is to simplify the addition of new test programs for
utilities or libraries under any of these directories.  Doing so on a
case by case basis is unnecessary and is quite an obscure process.
2014-04-21 21:39:25 +00:00
Xin LI
613074ec08 MFV r264666:
4374 dn_free_ranges should use range_tree_t

illumos/illumos-gate@bf16b11e8d

MFC after:	2 weeks
2014-04-18 21:15:12 +00:00
Mark Johnston
7df83e08ba Replace a few Solarisisms with their corresponding FreeBSDisms to make a few
printf tests pass.
2014-04-15 02:32:00 +00:00
Mark Johnston
28af9281f7 Use the correct format specifiers for wide characters and strings of wide
characters.

MFC after:	1 week
2014-04-15 02:28:08 +00:00
Xin LI
8621ec029b Take into account when zpool history block grows exceeding 128KB in zpool(8)
and zdb(8) by growing the buffer on demand with a cap of 1GB (specified in
spa_history_create_obj()).

PR:		bin/186574
Submitted by:	Andrew Childs <lorne cons org nz> (with changes)
MFC after:	2 weeks
2014-04-14 18:38:14 +00:00
Warner Losh
3bdf775801 NO_MAN= has been deprecated in favor of MAN= for some time, go ahead
and finish the job. ncurses is now the only Makefile in the tree that
uses it since it wasn't a simple mechanical change, and will be
addressed in a future commit.
2014-04-13 05:21:56 +00:00
Alexander Motin
537650f54d Add property and sysctl to control how ZVOLs are exposed to OS.
New ZFS property volmode and sysctl vfs.zfs.vol.mode allow switching ZVOL
between three modes:
 geom -- existing fully functional behavior (default);
 dev -- exposing volumes only as raw disk device file in devfs;
 none -- not exposing volumes outside ZFS.

The "dev" mode is less functional (can't be partitioned, mounted, etc),
but it is faster, and in some scenarios with untrusted consumers safer.
It can be useful for NAS, VM block storages, etc.
The "none" mode may be convenient for backup servers, etc. that don't
need direct data access.

Due to the way ZVOL is integrated with main ZFS code, those property
and sysctl are checked only during pool import and volume creation.

MFC after:	1 month
Sponsored by:	iXsystems, Inc.
2014-04-05 13:01:44 +00:00
Pedro F. Giffuni
23e4da439c MFV r258379;
4248 dtrace(1M) should never create DOF with empty probes section
4249 Only probes from the first DTrace object file will be included

Illumos Revision:	4a20ab41aadcb81c53e72fc65886e964e9add59

Reference:
https://www.illumos.org/issues/4248
https://www.illumos.org/issues/4249

Obtained from:	Illumos
MFC after:	1 month
2014-04-02 15:32:44 +00:00
Xin LI
49a6711083 MFV r263887:
3993 zpool(1M) and zfs(1M) should support -p for "list" and "get"
4700 "zpool get" doesn't support -H or -o options

MFC after:	2 weeks
2014-03-28 23:12:00 +00:00
Xin LI
2df503a63a MFV 263436-263438:
3947 zpool(1M) references nonexistent zfs-features(5)
  4540 zpool(1M) man page doesn't describe "readonly" property
  3948 zfs sync=default is not accepted
  4611 zfs(1M) still mentions 'send -r' in synopsis
  4415 zpool(1M) man page missing "import -m" description
  4570 Document dedupditto pool property
  4572 Dedup-related documentation additions for zpool and zdb.
  1371 Add -D option description to zpool(1M) manpage
  4571 Add documentation for -T and interval to "zpool list"

MFC after:	2 weeks
2014-03-21 01:32:25 +00:00
Xin LI
cd3ea9e055 Remove unused option -r from zpool.
Submitted by:	Richard Yao <ryao gentoo org>
MFC after:	2 weeks
2014-03-19 23:04:52 +00:00
Alan Somers
2d7c8761fc cddl/contrib/opensolaris/lib/libuutil/common/uu_avl.c
Fix a memory leak in uu_avl_pool_create: pthread_mutex_init without
	a corresponding pthread_mutex_destroy.  It shows up, among other
	places, when doing "zfs list".

MFC after:	3 weeks
Sponsored by:	Spectra Logic Corporation
2014-03-07 23:01:35 +00:00
John-Mark Gurney
1189b87f96 mark that libctf depends upon libz so that if you dlopen libctf, you
don't get:
Undefined symbol "zError"
2014-03-05 23:37:25 +00:00
Mark Johnston
44c25a3d82 When our linker merges .SUNW_dof sections from multiple files, it simply
concatenates the DOF tables into one section. Previously, the USDT init
code in drti.o would only look at the first table in the DOF section; with
this change, it iterates over all the tables, passing each DOF table to
the kernel.

PR:		186821
Submitted by:	Fedor Indutny <fedor@indutny.com>
MFC after:	1 month
2014-03-01 23:09:07 +00:00
Mark Johnston
ae9f1a185c 4478 dtrace_dof_maxsize is far too small
illumos/illumos-gate@d339a29bb4

PR:		187027
MFC after:	1 week
2014-02-28 02:04:41 +00:00
Xin LI
9af4c716d0 MFV r262570:
4626 libzfs memleak in zpool_in_use()

illumos/illumos-gate@fb13f48f1d

MFC after:	2 weeks
2014-02-27 21:50:46 +00:00
Mark Johnston
0339a1c2b4 Move some files that are identical on i386 and amd64 to an x86 subdirectory
rather than keeping duplicate copies.

Discussed with:	avg
MFC after:	1 week
2014-02-27 01:04:35 +00:00
Mark Johnston
33db01542c 1452 DTrace buffer autoscaling should be less violent
illumos/illumos-gate@6fb4854bed

This fixes the tst.resize1.d and tst.resize2.d DTrace tests, which have
been failing since r261122 since they were causing dtrace(1) to attempt to
allocate and use large amounts of memory, and get killed by the OOM killer
as a result.

MFC after:	1 month
2014-02-22 05:18:55 +00:00
Mark Felder
eba76eb552 Fix formatting.
"Manpages should start a new sentence on a new line. This makes it easier
for translators to track changes." -jhb

Approved by:	jhb
MFC after:	3 days
Sponsored by:	SupraNet Communications, Inc
2014-02-17 13:23:49 +00:00
Andriy Gapon
ae2ea19ddc zfs.8: fix garbled options in a sample zfs send -R command line
MFC after:	5 days
2014-02-14 15:21:21 +00:00
Andriy Gapon
b5ef255191 zpool.8: fix typo in option description of labelclear command
MFC after:	5 days
2014-02-14 15:20:49 +00:00
Mark Felder
2d084599da Add caveat to zpool manpage indicating that we do not automatically activate
hot spares. This should be MFC'd to all STABLE branches.

Upon the availability of zfsd, the zpool manpage on relevant branches should
be updated to remove this caveat and document hot spare's reliance on zfsd.

Approved by:	avg
MFC after:	1 week
Sponsored by:	SupraNet Communications
2014-02-11 15:38:29 +00:00
Kai Wang
d8b88105c2 Only declare `bysz' variable under little endian archs. 2014-01-29 09:58:05 +00:00
Kai Wang
a36ae249b8 MFH@261151. 2014-01-25 14:02:02 +00:00
Kai Wang
a52b8956ff Simplify DWARF version check.
Submitted by:	emaste
2014-01-25 11:27:09 +00:00
Andriy Gapon
9a5be1a0d8 dtrace: remove unexplained 16MB limitation from dt_alloc/dt_zalloc
The limitation was introduced in r178556 without any note or comment.
It seems pretty artificial and now it leads to problems like the following:
  $ dtrace -x bufsize=17m -n ...
  dtrace: processing aborted: Memory allocation failure
OpenSolaris and illumos never had this limitation.

Sponsored by:	HybridCluster
2014-01-24 15:04:02 +00:00
Kai Wang
529a53abe2 Let ctfconvert accept DWARF version 3 and 4. 2014-01-22 13:43:54 +00:00
Kai Wang
945a2095f6 MFH@260917. 2014-01-20 19:38:44 +00:00
Kai Wang
d32f014939 Clang 3.4 will sometimes emit DIE for struct/union member before
emitting the DIE for the type of that member. ctfconvert can not
handle this properly and will calculate a wrong member bit offset.
Same struct/union type from different .o file will be treated as
different types when their member bit offsets are different, and
gets added/merged multiple times. This will in turn cause many other
structs/pointers/typedefs that refer to the duplicated struct/union
gets added/merged multiple times and eventually causes numerous
duplicated CTF types in the kernel.debug file.

The simple workaround here is to make use of DW_AT_byte_size attribute
of the member DIE to calculate the bits occupied by the member's type,
without actually resolving the type.
2014-01-20 01:35:14 +00:00
Kai Wang
e3fa6e64ee * Make die_mem_offset() be able to handle DW_AT_data_member_location
attributes generated by Clang 3.4.
* Document how different compilers generate DW_AT_data_member_location
  attributes differently.
* Document the quirks about DW_FORM_data[48].
2014-01-19 13:48:02 +00:00
Andriy Gapon
df217ba9ba zdb -R: do not treat numeric parameters to a flag as more flags
Reviewed by:	Matthew Ahrens <mahrens@delphix.com>
MFC after:	1 week
2014-01-17 10:18:45 +00:00
Kai Wang
aedbdd2c5c We should not set the unnamed DIE's name to "__anon__" since that will
bring back a known issue with DTrace regarding type name
comparison. Instead, we can set the name to an empty string.

Pointed out by:	     avg
2014-01-17 08:44:12 +00:00
Kai Wang
1391789ee2 If function die_name() finds a DIE without a name, set its name to
"__anon__". This hack is used to workaround a issue that compilers
like GCC could generate DW_TAG_base_type DIE without a name.

Note that we didn't need this before because the old libdwarf
internally set all the unnamed DIE's name to "__anon__".
2014-01-16 22:28:33 +00:00
Kai Wang
58fc1c99e4 Convert ctfconvert to use the new libdwarf API. 2014-01-16 21:56:05 +00:00
Andriy Gapon
c38e50a638 zinject must use ioctl(2) compatibility wrapper
MFC after:	8 days
Sponsored by:	HybridCluster
2014-01-16 12:21:21 +00:00
Xin LI
ab0b9f6b30 MFV r260154 + 260182:
4369 implement zfs bookmarks
4368 zfs send filesystems from readonly pools

Illumos/illumos-gate@78f1710053

MFC after:	2 weeks
2014-01-02 07:34:36 +00:00
Xin LI
7353c6e991 MFV r260152:
4208 Typo in zfs_main.c: "posxiuser"

illumos/illumos-gate@f38cb554a5

Note: this is a stripped down version of Illumos change.

MFC after:	2 weeks
2014-01-01 01:23:40 +00:00
Xin LI
f4c8ba8370 MFV r259170:
4370 avoid transmitting holes during zfs send

4371 DMU code clean up

illumos/illumos-gate@43466aae47

NOTE: Make sure the boot code is updated if a zpool upgrade is
done on boot zpool.

MFC after:	2 weeks
2014-01-01 00:45:28 +00:00
Xin LI
f86b34932a MFV r258972:
4373 add block contents print to zstreamdump

illumos/illumos-gate@994fb6b8a9

MFC after:	2 weeks
2013-12-31 21:37:24 +00:00
Xin LI
db2aff5f8b MFV r242733:
3306 zdb should be able to issue reads in parallel
3321 'zpool reopen' command should be documented in the man page
and help message

illumos/illumos-gate@31d7e8fa33

FreeBSD porting notes: the kernel part of this changeset depends
on Solaris buf(9S) interfaces and are not really applicable for
our use.  vdev_disk.c is patched as-is to reduce diverge from
upstream, but vdev_file.c is left intact.

MFC after:	2 weeks
2013-12-31 19:39:15 +00:00
Mark Johnston
2b374230b1 When clearing relocations to __dtrace* symbols, handle both SHT_REL and
SHT_RELA sections properly instead of assuming that the relocation section
is of type SHT_REL.

Submitted by:	Prashanth Kumar <pra_udupi@yahoo.co.in> (original version)
MFC after:	1 month
2013-12-29 19:27:32 +00:00
Xin LI
4e5c0539bb MFV r258384:
2583 Add -p (parsable) option to zfs list

illumos/illumos-gate@43d68d68c1

MFC after:	2 weeks
2013-12-25 00:39:04 +00:00
Xin LI
9b94cdc122 Fix incorrect markup introduced in r259813.
Pointy hat to:	delphij
X-MFC-after:	r259813
2013-12-24 07:27:55 +00:00
Xin LI
1aaa945f67 MFV r258374:
4171 clean up spa_feature_*() interfaces

4172 implement extensible_dataset feature for use by other zpool
features

illumos/illumos-gate@2acef22db7

MFC after:	2 weeks
2013-12-24 07:14:25 +00:00
Xin LI
ec097c1634 MFV r258373:
4168 ztest assertion failure in dbuf_undirty

4169 verbatim import causes zdb to segfa
4170 zhack leaves pool in ACTIVE state

illumos/illumos-gate@7fdd916c47

MFC after:	2 weeks
2013-12-24 06:56:17 +00:00
Alexander Motin
f192c4873d Don't even try to read vdev labels from devices smaller then SPA_MINDEVSIZE
(64MB).  Even if we would find one somehow, ZFS kernel code rejects such
devices.  It is funny to look on attempts to read 4 256K vdev labels from
1.44MB floppy, though it is not very practical and quite slow.
2013-12-10 12:36:44 +00:00
Xin LI
48af354c07 Don't panic when we get ZPOOL_STATUS_NON_NATIVE_ASHIFT
while listing importable pools.

MFC after:	3 days
2013-12-09 18:52:21 +00:00
Joel Dahl
2727e97436 mdoc: remove EOL whitespace. 2013-12-06 21:22:33 +00:00
Mark Johnston
bc968a581f Enable some previously-disabled DTrace tests for umod, ufunc and usym. They
expect the installed ksh binary to be named "ksh", which is not the case
when it's installed on FreeBSD via the shells/ksh93 port. Allow for it to be
"ksh93" as well so that the tests can actually pass.
2013-12-04 01:40:39 +00:00
Mark Johnston
e9be89ce40 The uaddr, ufunc, umod and usym functions all seem to work as expected on
FreeBSD, so stop hiding them behind a "#if defined(sun)".

Reported by:	Prashanth Kumar <pra_udupi@yahoo.co.in>
2013-12-04 01:35:04 +00:00
Mark Johnston
33d8425001 Use mkstemp(3) to create the temporary file used in the FreeBSD-specific
portions of dtrace_program_link().
2013-12-03 03:40:47 +00:00
Andriy Gapon
456a87bb3b MFV r258371,r258372: 4101 metaslab_debug should allow for fine-grained control
4101 metaslab_debug should allow for fine-grained control
4102 space_maps should store more information about themselves
4103 space map object blocksize should be increased
4104 ::spa_space no longer works
4105 removing a mirrored log device results in a leaked object
4106 asynchronously load metaslab

illumos/illumos-gate@0713e232b7

Note that some tunables have been removed and some new tunables have
been added.  Of particular note, FreeBSD-only knob
vfs.zfs.space_map_last_hope is removed as it was a nop for some time now
(after one of the previous merges from upstream).

MFC after:	11 days
Sponsored by:	HybridCluster [merge]
2013-11-28 19:37:22 +00:00
Andriy Gapon
2a4704ab01 MFV r255255: 4045 zfs write throttle & i/o scheduler performance work
illumos/illumos-gate@69962b5647

Please note the following changes:
- zio_ioctl has lost its priority parameter and now TRIM is executed
  with 'now' priority
- some knobs are gone and some new knobs are added; not all of them are
  exposed as tunables / sysctls yet

MFC after:	10 days
Sponsored by:	HybridCluster [merge]
2013-11-26 09:57:14 +00:00
Andriy Gapon
34140e78ab 734 taskq_dispatch_prealloc() desired
943 zio_interrupt ends up calling taskq_dispatch with TQ_SLEEP
illumos/illumos-gate@5aeb94743e

Essentially FreeBSD taskqueues already operate in a mode that
was added to Illumos with taskq_dispatch_ent change.
We even exposed the superior FreeBSD interface as taskq_dispatch_safe.
Now we just rename taskq_dispatch_safe to taskq_dispatch_ent and
struct struct ostask to taskq_ent_t, so that code differences will be
minimal.

After this change sys/cddl/compat/opensolaris/sys/taskq.h header is no
longer needed.

Note that this commit is not an MFV because the upstream change was not
individually committed to the vendor area.

MFC after:	8 days
2013-11-26 09:26:18 +00:00