1
0
mirror of https://git.FreeBSD.org/src.git synced 2024-12-14 10:09:48 +00:00
Commit Graph

184 Commits

Author SHA1 Message Date
Mariusz Zaborski
277f38abff zfs: add an option to the bootloader to rewind the ZFS checkpoint
The checkpoints are another way of keeping the state of ZFS.
During the rewind, the pool has to be exported.
This makes checkpoints unusable when using ZFS as root.
Add the option to rewind the ZFS checkpoint at the boot time.
If checkpoint exists, a new option for rewinding a checkpoint will appear in
the bootloader menu.
We fully support boot environments.
If the rewind option is selected, the boot loader will show a list of
boot environments that existed before the checkpoint.

Reviewed by:	tsoome, allanjude, kevans (ok with high-level overview)
Differential Revision:	https://reviews.freebsd.org/D24920
2020-08-18 19:48:04 +00:00
Toomas Soome
de776da323 loader: implement GELI writes
Bug: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247482

This patch is based on initial work from allanjude.

PR:		247482
Obtained from:	https://reviews.freebsd.org/D10236
Differential Revision:	https://reviews.freebsd.org/D25605
2020-07-11 06:51:42 +00:00
Toomas Soome
1a4b982e1e loader: setting vdev size based on label asize is not working
Because we need to read asize from vdev_tree. We also need to consider
different vdev type difference.

Reviewed by:	allanjude
Sponsored by:	Netflix, Klara Inc.
Differential Revision:	https://reviews.freebsd.org/D25586
2020-07-09 13:19:00 +00:00
Toomas Soome
0d1a620681 loader: geli_dev_ioctl does return huge mediasize
The DIOCGMEDIASIZE is calculated md->md_sectorsize * md->md_provsize, and
for boot disk, the md_sectorsize is 4k. However, the md_provsize is already
in units of bytes.

Sponsored by:	Netflix, Klara Inc.
2020-07-07 12:24:40 +00:00
Toomas Soome
771f100ceb loader: zfs reader does not need BOOT2 bits
After switching zfsloader to use full libsa, we do not need
spa_get_primary() and spa_get_primary_vdev() any more.

Sponsored by:	Netflix, Klara Inc.
2020-07-02 07:15:48 +00:00
Toomas Soome
045f497cbc loader: potential memory leak and check return values
Need to free nvlist before return from vdev_from_nvlist().

Sponsored by:	Netflix, Klara Inc.
2020-07-02 07:03:15 +00:00
Toomas Soome
b93b14dc75 loader: can not read zfs pool with slog removed
The vdev_init() does check for "known" vdev types, the [log] device removal will create "hole"
device, but vdev_init() does not allow it.

Obtained from: illumos
MFC after:	1 week
2020-06-26 21:21:35 +00:00
Toomas Soome
3830659e99 loader: create single zfs nextboot implementation
We should have nextboot feature implemented in libsa zfs code.
To get there, I have created zfs_nextboot() implementation based on
two sources, our current simple textual string based approach with added
structured boot label PAD structure from OpenZFS.

Secondly, all nvlist details are moved to separate source file and
restructured a bit. This is done to provide base support to add nvlist
add/update feature in followup updates.

And finally, the zfsboot/gptzfsboot disk access functions are swapped to use
libi386 and libsa.

Sponsored by:	Netflix, Klara Inc.
Differential Revision:	https://reviews.freebsd.org/D25324
2020-06-20 06:23:31 +00:00
Kirk McKusick
34816cb9ae Move the pointers stored in the superblock into a separate
fs_summary_info structure. This change was originally done
by the CheriBSD project as they need larger pointers that
do not fit in the existing superblock.

This cleanup of the superblock eases the task of the commit
that immediately follows this one.

Suggested by: brooks
Reviewed by:  kib
PR:           246983
Sponsored by: Netflix
2020-06-19 01:02:53 +00:00
Toomas Soome
aba2397e74 loader: r362262 did miss the pathlen check
While we are checking the "/dev/" prefix, we can skip the paths shorter than
this prefix.

Sponsored by:	Netflix, Klara Inc.
2020-06-17 10:56:58 +00:00
Toomas Soome
c98740b5e2 loader: strings in nvlist are counted strings, not c-strings
We need to fetch both string size and data, then handle the data.

Reviewed by:	allanjude
Sponsored by:	Netflix, Klara Inc.
2020-06-17 10:41:01 +00:00
John Baldwin
4f98ffdd1d Fix libstand build breakage after r361298.
- Use enc_xform_aes_xts.setkey() directly instead of duplicating the code
  now that it no longer calls malloc().
- Rather than bringing back all of xform_userland.h, add a conditional
  #include of <stand.h> to xform_enc.h.
- Update calls to encrypt/decrypt callbacks in enc_xform_aes_xts for
  separate input/output pointers.

Pointy hat to:	jhb
2020-05-20 22:25:41 +00:00
Eric van Gyzen
fac6dee9eb Remove tests for obsolete compilers in the build system
Assume gcc is at least 6.4, the oldest xtoolchain in the ports tree.
Assume clang is at least 6, which was in 11.2-RELEASE.  Drop conditions
for older compilers.

Reviewed by:	imp (earlier version), emaste, jhb
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D24802
2020-05-12 15:22:40 +00:00
Toomas Soome
4a2d7cee02 loader: vdev_read() can corrupt memory
When reading less than sector size but from sector boundary,
the vdev_read() will read full sector into the provided buffer
and therefore corrupting memory past buffer end.

MFC after:	2 days
2020-05-09 06:25:20 +00:00
Adrian Chadd
6c88ef1c81 [libsa] Fix typecast of pointer for st_dev
This code was trying to use a pointer value for st_dev, which is definitely
not a pointer.  Instead, cast to uintptr_t so it becomes a non-pointer value
before casting it.

Tested: mips-gcc cross compile, mips32 build
2020-04-16 23:29:49 +00:00
Simon J. Gerraty
723f904176 Improve interaction of vectx and tftp
On slow platforms, it helps to spread the hashing load
over time so that tftp does not timeout.

Also, some .4th files are too big to fit in cache of pkgfs,
so increase cache size and ensure fully populated.

Reviewed by:	stevek
MFC after:	1 week
Differential Revision: https://reviews.freebsd.org/D24287
2020-04-07 16:56:34 +00:00
Toomas Soome
5292d0a7f3 loader: use #elif defined() in stand.h
Small cleanup.

Reported by:	imp
2020-03-29 06:48:59 +00:00
Toomas Soome
6c01b7102d loader: typo did slip into 359408
somehow r was lost from 'char'.
2020-03-28 21:55:42 +00:00
Toomas Soome
215597f05f loader: strdup name strings from dataset walker
The removal of zfs scratch buffer did miss the fact the dataset
lookup was picking up the names from zap list.
2020-03-28 21:50:27 +00:00
Toomas Soome
c907ec9e16 loader: add knob to build with user malloc
This option is intended to aid development, to allow building with user malloc.
The use case would be to build userboot & test with libc (or other) malloc and
use extra malloc debug features.
2020-03-28 21:47:44 +00:00
Simon J. Gerraty
53f151f906 Fix pkgfs stat so it satisfies libsecureboot
We need a valid st_dev, st_ino and st_mtime
to correctly track which files have been verified
and to update our notion of time.

ve_utc_set(): ignore utc if it would jump our current time
by more than VE_UTC_MAX_JUMP (20 years).

Allow testing of install command via userboot.
Need to fix its stat implementation too.

bhyveload also needs stat fixed - due to change to userboot.h

Call ve_error_get() from vectx_close() when hash is wrong.

Track the names of files we have hashed into pcr

For the purposes of measured boot, it is important
to be able to reproduce the hash reflected in
loader.ve.pcr
so loader.ve.hashed provides a list of names in the order they
were added.

Reviewed by:	imp
MFC after:	1 week
Sponsored by:	Juniper Networks
Differential Revision:	https://reviews.freebsd.org//D24027
2020-03-25 19:12:19 +00:00
Toomas Soome
27bae6150a loader: remove libsa/crc32.c and use version from zlib
we have crc32(const void *, size_t) in libsa. Unfortunately zlib has
crc32(long, const unigned char *, unsigned) and we have conflict.

Since we do build libsa with zlib, we can use zlib version instead.

Reviewed by:	allanjude
Differential Revision:	https://reviews.freebsd.org/D24068
2020-03-19 21:05:11 +00:00
Toomas Soome
c1c4c81fd7 loader: replace zfs_alloc/zfs_free with malloc/free
Use common memory management.
2020-02-26 18:12:12 +00:00
Warner Losh
a64f0b83e5 Move smbios.c to libsa.
smbios used to be an i386 only kinda weird quirk to the x86
architecture. But UEFI picked it up, dusted it off and now it's many
other locations. Make it base technology by moving it to libsa and
fixing up the compliation. The code has issues with unaligned access
still, but that will be addressed in a followup commit.

Differential Revision: https://reviews.freebsd.org/D23660
2020-02-20 00:46:22 +00:00
Warner Losh
ed2a65769a Create ptov() function.
Create a ptov() function. It's basically the same as the btx PTOV
macro, but works everywhere. smbios needs this to translate addresses,
but the translation differs between BIOS booting and EFI booting. Make
it a function so one smbios.o can be used everywhere. Provide
definitions for it in the two loaders affected.

Differential Revision: https://reviews.freebsd.org/D23660
2020-02-20 00:46:16 +00:00
Toomas Soome
380977d557 followup on r357497: clean obsolete comment and use shift instead of multiplication
Based on illumos review feedback: leftover comment, but also
have consistent block size calculation and add missing else leg to if
statement.
2020-02-05 13:08:24 +00:00
Toomas Soome
4d297e7035 loader: rewrite zfs reader zap code to use malloc
First step on removing zfs_alloc.

Reviewed by:	delphij
Differential Revision:	https://reviews.freebsd.org/D23433
2020-02-04 07:37:55 +00:00
Warner Losh
eb24e1491f Remove sparc64 support from the boot loader.
Remove all the sparc64 specific bits, both files and ifdefs.
2020-02-03 17:34:57 +00:00
Toomas Soome
653c3383c7 loader: update zfsimpl.c from illumos review suggestions
Add extra comments and update error handling.

MFH:		1 week
2020-01-06 19:35:22 +00:00
Dimitry Andric
0b57cec536 Move all sources from the llvm project into contrib/llvm-project.
This uses the new layout of the upstream repository, which was recently
migrated to GitHub, and converted into a "monorepo".  That is, most of
the earlier separate sub-projects with their own branches and tags were
consolidated into one top-level directory, and are now branched and
tagged together.

Updating the vendor area to match this layout is next.
2019-12-20 19:53:05 +00:00
Toomas Soome
3c2db0ef43 loader: rewrite zfs vdev initialization
In some cases the pool discovery will get stuck in infinite loop while setting
up the vdev children.

To fix, we split the vdev setup into two parts, first we create vdevs based on
configuration we do get from pool label, then, we process pool config from MOS
and update the pool config if needed.

Testing done: confirm previously hung loader is not hung any more.

MFC after:	1 week
2019-12-15 21:52:40 +00:00
Toomas Soome
2e6bb6553b loader: zfsimpl.c cstyle cleanup
No functional changes intended.

MFC after:	1 week
2019-12-15 14:09:49 +00:00
Andriy Gapon
c527e92004 zfs boot: fix a crash in a rarely taken path in fzap_lookup
Instead of passing NULL to fzap_name_equal and crashing, just return
ENOENT.  This happened when higher bits of a hash of the searched key
(its hash prefix) matched a hash prefix of some key in the ZAP, but the
full hash value of the searched key did not match any key in the ZAP.

I observerved this problem when loader tried to look up
"features_for_read" in a particular old pool that predates pool
features.

MFC after:	2 weeks
Sponsored by:	Panzura
2019-12-13 22:04:13 +00:00
Toomas Soome
8ac66965f5 loader: cd9660_open() warn: is 'buf' large enough for 'struct iso_primary_descriptor'?
We do allocate amount of memory (void * or char *), and then assign this
buffer to struct iso_primary_descriptor *vd. Make sure we do
allocate enough bytes.

In fact we do allocate enough, but it is good idea to make sure this really
is so.

MFC after:	1 week
2019-12-13 12:36:16 +00:00
Warner Losh
f86e60008b Regularize my copyright notice
o Remove All Rights Reserved from my notices
o imp@FreeBSD.org everywhere
o regularize punctiation, eliminate date ranges
o Make sure that it's clear that I don't claim All Rights reserved by listing
  All Rights Reserved on same line as other copyright holders (but not
  me). Other such holders are also listed last where it's clear.
2019-12-04 16:56:11 +00:00
Andriy Gapon
489912da7b MFV r354382,r354385: 10601 10757 Pool allocation classes
illumos/illumos-gate@663207adb1
663207adb1

10601 Pool allocation classes
https://www.illumos.org/issues/10601
  illumos port of ZoL Pool allocation classes. Includes at least these two
  commits:
  441709695 Pool allocation classes misplacing small file blocks
  cc99f275a Pool allocation classes

10757 Add -gLp to zpool subcommands for alt vdev names
https://www.illumos.org/issues/10757
  Port from ZoL of
  d2f3e292d Add -gLp to zpool subcommands for alt vdev names
  Note that a subsequent ZoL commit changed -p to -P
  a77f29f93 Change full path subcommand flag from -p to -P

Portions contributed by: Jerry Jelinek <jerry.jelinek@joyent.com>
Portions contributed by: Håkan Johansson <f96hajo@chalmers.se>
Portions contributed by: Richard Yao <ryao@gentoo.org>
Portions contributed by: Chunwei Chen <david.chen@nutanix.com>
Portions contributed by: loli10K <ezomori.nozomu@gmail.com>
Author: Don Brady <don.brady@delphix.com>

11541 allocation_classes feature must be enabled to add log device

illumos/illumos-gate@c1064fd7ce
c1064fd7ce

https://www.illumos.org/issues/11541
  After the allocation_classes feature was integrated, one can no longer add a
  log device to a pool unless that feature is enabled. There is an explicit check
  for this, but it is unnecessary in the case of log devices, so we should handle
  this better instead of forcing the feature to be enabled.

Author: Jerry Jelinek <jerry.jelinek@joyent.com>

FreeBSD notes.
I faithfully added the new -g, -L, -P flags, but only -g does something:
vdev GUIDs are displayed instead of device names.  -L, resolve symlinks,
and -P, display full disk paths, do nothing at the moment.
The use of special vdevs is backward compatible for read-only access, so
root pools should be bootable, but exercise caution.

MFC after:	4 weeks
2019-11-21 08:20:05 +00:00
Ravi Pokala
7951fbc23d Logging improvements to loader::nfs
Include the server IP address when logging nfs_open(), add a few missing
"\n"s, and correct a typo.

Reviewed by:	kevans
MFC after:	2 weeks
Sponsored by:	Panasas
Differential Revision:	https://reviews.freebsd.org/D22346
2019-11-13 03:56:51 +00:00
Toomas Soome
c082b391b0 reverting r354594
In our case the structure is more complex and simple static initializer
will upset compiler diagnostics - using memset is still better than building
more complext initializer.
2019-11-12 10:02:39 +00:00
Toomas Soome
e5d9d07948 loader: use struct initializer in vdev_probe().
Hopefully it is a bit more clear this way.
2019-11-10 15:07:36 +00:00
Toomas Soome
3cac32d147 loader: memory leak in vdev_label_read_config()
We need to free the allocated buffer for label.
2019-11-10 15:03:59 +00:00
Toomas Soome
ae3f74e763 loader: clean up the noise around log device
We are ignoring log device but we also do want to have clean code.
2019-11-05 18:35:13 +00:00
Toomas Soome
9a3ea7594b loader: populate nvl with data even when label_txg is 0
We actually do not use that data, at least not now, but we want to
avoid possible surprises.
2019-11-05 18:07:30 +00:00
Toomas Soome
f331eb7d34 loader: allow booting from pool with log device
Apparently people are using log with boot pool, so we can not just
error out there. Still we will need to investigate how to handle
log properly.
2019-11-03 21:36:29 +00:00
Toomas Soome
79a4bf8975 loader: factor out label and uberblock load from vdev_probe, add MMP checks
Clean up the label read.
2019-11-03 21:19:52 +00:00
Toomas Soome
21da9f14f6 loader: zfs.c is missing malloc checks, fix it
malloc() can return NULL, we need to check the return value.
2019-11-03 14:36:16 +00:00
Toomas Soome
0c0a882c7a loader: we do not support booting from pool with log device
If pool has log device, stop there and tell about it.
2019-11-03 13:25:47 +00:00
Toomas Soome
f4ed004573 loader: should check malloc in zfs_dev_open
malloc can return NULL.
2019-11-03 13:03:47 +00:00
Toomas Soome
abca0bd501 loader: calculate physical vdev psize from asize
Since physical device asize is calculated from psize and the asize is stored
in pool label, we can use asize to set the value of psize, which is used to
calculate the location of the pool labels.

MFC after:	1 week
2019-11-03 11:09:06 +00:00
Toomas Soome
e499793e76 Remove duplicate lz4 implementations
Port illumos change: https://www.illumos.org/issues/11667

Move lz4.c out of zfs tree to opensolaris/common/lz4, adjust it to be
usable from kernel/stand/userland builds, so we can use just one single
source. Add lz4.h to declare lz4_compress() and lz4_decompress().

MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D22037
2019-11-02 12:28:04 +00:00
Toomas Soome
98e28b71b2 loader: asprinf does crash arm64 due to missing NULL pointer check
PCHAR macro needs to check if d is NULL.

MFC after:	3 days
2019-11-01 06:54:07 +00:00