This is not completely necessary today, but this change is being made in a
conservative manner to avoid accidental breakage in the future, if this ever
was a unicode string.
PR: 237403
MFC after: 1 week
In python 3, the default encoding was switched from ascii character sets to
unicode character sets in order to support internationalization by default.
Some interfaces, like ioctls and packets, however, specify data in terms of
non-unicode encodings formats, either in host endian (`fcntl.ioctl`) or
network endian (`dpkt`) byte order/format.
This change alters assumptions made by previous code where it was all
data objects were assumed to be basestrings, when they should have been
treated as byte arrays. In order to achieve this the following are done:
* str objects with encodings needing to be encoded as ascii byte arrays are
done so via `.encode("ascii")`. In order for this to work on python 3 in a
type agnostic way (as it anecdotally varied depending on the caller), call
`.encode("ascii")` only on str objects with python 3 to cast them to ascii
byte arrays in a helper function name `str_to_ascii(..)`.
* `dpkt.Packet` objects needing to be passed in to `fcntl.ioctl(..)` are done
so by casting them to byte arrays via `bytes()`, which calls
`dpkt.Packet__str__` under the covers and does the necessary str to byte array
conversion needed for the `dpkt` APIs and `struct` module.
In order to accomodate this change, apply the necessary typecasting for the
byte array literal in order to search `fop.name` for nul bytes.
This resolves all remaining python 2.x and python 3.x compatibility issues on
amd64. More work needs to be done for the tests to function with i386, in
general (this is a legacy issue).
PR: 237403
MFC after: 1 week
Tested with: python 2.7.16 (amd64), python 3.6.8 (amd64)
Even though some python styles suggest there should be multiple newlines between
methods/classes, for consistency with the surrounding code, it's best to be
consistent by having merely one newline between each functional block.
MFC after: 1 week
Make `KAT(CCM)?Parser` into a context suite-capable object by implementing
`__enter__` and `__exit__` methods which manage opening up the file descriptors
and closing them on context exit. This implementation was decided over adding
destructor logic to a `__del__` method, as there are a number of issues around
object lifetimes when dealing with threading cleanup, atexit handlers, and a
number of other less obvious edgecases. Plus, the architected solution is more
pythonic and clean.
Complete the iterator implementation by implementing a `__next__` method for
both classes which handles iterating over the data using a generator pattern,
and by changing `__iter__` to return the object instead of the data which it
would iterate over. Alias the `__next__` method to `next` when working with
python 2.x in order to maintain functional compatibility between the two major
versions.
As part of this work and to ensure readability, push the initialization of the
parser objects up one layer and pass it down to a helper function. This could
have been done via a decorator, but I was trying to keep it simple for other
developers to make it easier to modify in the future.
This fixes ResourceWarnings with python 3.
PR: 237403
MFC after: 1 week
Tested with: python 2.7.16 (amd64), python 3.6.8 (amd64)
In version 3.2+, `array.array(..).tostring()` was renamed to
`array.array(..).tobytes()`. Conditionally call `array.array(..).tobytes()` if
the python version is 3.2+.
PR: 237403
MFC after: 1 week
This was enumerated with exhaustive search for sys/eventhandler.h includes,
cross-referenced against EVENTHANDLER_* usage with the comm(1) utility. Manual
checking was performed to avoid redundant includes in some drivers where a
common os_bsd.h (for example) included sys/eventhandler.h indirectly, but it is
possible some of these are redundant with driver-specific headers in ways I
didn't notice.
(These CUs did not show up as missing eventhandler.h in tinderbox.)
X-MFC-With: r347984
This change properly terminates the formatting string quote modification done
in r348005, which is triggered when `ELF_VERBOSE` is defined.
MFC with: r348005
Reported by: ci (amd64, gcc)
Replace uses of `foo.encode("hex")` with `binascii.hexlify(foo)` for forwards
compatibility between python 2.x and python 3.
PR: 237403
MFC after: 1 week
These are obviously missing from the .c files, but don't show up in any
tinderbox configuration (due to latent header pollution of some kind). It
seems some configurations don't have this pollution, and the includes are
obviously missing, so go ahead and add them.
Reported by: Peter Jeremy <peter AT rulingia.com>
X-MFC-With: r347984
all-ones then carving out blocks of zeroes where specified values go, init
it to all-zeroes, put in ones where values need to be masked, then use it
as value &= ~sc_led_modes_mask. In addition to being more idiomatic, this
means everything related to FDT data is initialized to zero along with the
rest of the softc, and that allows removing some #ifdef FDT sections and
wrapping the whole muge_set_leds() function in a single ifdef block.
This also deletes the early-out from muge_set_leds() when an eeprom exists.
Even if there is an eeprom with led config in it, the fdt data (if present)
should override that, because the user is in control of the fdt data.
OTP registers (because the user is in control of the fdt data). Remove the
early returns from the code that tries to find a good mac address, so that
the execution always flows through the routine to get an address from FDT
data last, when on FDT-enabled systems.
the device instance, and to get the MAC address for the device instance.
The ad-hoc code this replaces could find the wrong instance if multiple
devices were present.
The .init and .fini epilogues from crtn should be placed immediately
after any instructions in .init and .fini sections from the linked
objects. Using 16-byte alignment for the epilogues on MIPS was a bug,
but it did not cause any issue with GNU ld as GNU ld (2.17.50) fills the
padding with NOPs.
Current versions of LLD fill any padding between different object files
with trap instructions. Inserting trap padding prior to the .init/.fini
epilogue is undesriable as the resulting binary will crash at runtime.
The .init and .fini sections in object files linked between crti and
crtn must already be a multiple of the instruction size and so no
alignment directive is required in crtn. Indeed, other architectures
(except sparc64) do not specify alignment in their crtn implementations.
Reported by: arichardson
Reviewed by: andrew
Event: Waterloo Hackathon 2019
Differential Revision: https://reviews.freebsd.org/D18291
Also use LED mode settings from the FDT to set the PHY.
From v3 of the patch submitted in the PR.
I moved the sc_led_modes and sc_led_modes_mask default setting outside
of the #ifdef FDT case.
PR: 237325
Submitted by: Ralf <iz-rpi03@hs-karlsruhe.de>
Reviewed by: ian
MFC after: 2 weeks
MFC with: r348001
Event: Waterloo Hackathon 2019
Differential Revision: https://reviews.freebsd.org/D20325
Also apply some style(9) and remove the message about EEPROM configuration
(if there's an EEPROM the hardware handles LED configuration itself).
PR: 237325
Reviewed by: ian
MFC after: 2 weeks
Submitted by: Ralf <iz-rpi03@hs-karlsruhe.de>
The existing SPI support only worked for directly attached flash chips.
it didn't implement clock programming or chipselect. It also supports
transfers with unbalanced tx/rx command sizes.
Submitted by: <yamori813@yahoo.co.jp>
Differential Revision: https://reviews.freebsd.org/D20101
Python 3 no longer doesn't support encoding/decoding hexadecimal numbers using
the `str.format` method. The backwards compatible new method (using the
binascii module/methods) is a comparable means of converting to/from
hexadecimal format.
In short, the functional change is the following:
* `foo.decode('hex')` -> `binascii.unhexlify(foo)`
* `foo.encode('hex')` -> `binascii.hexlify(foo)`
While here, move the dpkt import in `cryptodev.py` down per PEP8, so it comes
after the standard library provided imports.
PR: 237403
MFC after: 1 week
When using LLVM+clang+lld on PowerPC64, ld.bfd is also needed,
to link 32-bit binaries correctly, as lld support for 32-bit is
still minimal.
This change enables it to be built and installed when lld is used.
Reviewed by: jhibbits
Differential Revision: https://reviews.freebsd.org/D20259
Sync libarchive with vendor.
Relevant vendor changes:
Issue #795: XAR - do not try to add xattrs without an allocated name
PR #812: non-recursive option for extract and list
PR #958: support reading metadata from compressed files
PR #999: add --exclude-vcs option to bsdtar
Issue #1062: treat empty archives with a GNU volume header as valid
PR #1074: Handle ZIP files with trailing 0s in the extra fields
(Android APK archives)
PR #1109: Ignore padding in Zip extra field data (Android APK archives)
PR #1167: fix problems related to unreadable directories
Issue #1168: fix handling of strtol() and strtoul()
PR #1172: RAR5 - fix invalid window buffer read in E8E9 filter
PR #1174: ZIP reader - fix of MSZIP signature parsing
PR #1175: gzip filter - fix reading files larger than 4GB from memory
PR #1177: gzip filter - fix memory leak with repeated header reads
PR #1180: ZIP reader - add support for Info-ZIP Unicode Path Extra Field
PR #1181: RAR5 - fix merge_block() recursion
(OSS-Fuzz 12999, 13029, 13144, 13478, 13490)
PR #1183: fix memory leak when decompressing ZIP files with LZMA
PR #1184: fix RAR5 OSS-Fuzz issues 12466, 14490, 14491, 12817
OSS-Fuzz 12466: RAR5 - fix buffer overflow when parsing huffman tables
OSS-Fuzz 14490, 14491: RAR5 - fix bad shift-left operations
OSS-Fuzz 12817: RAR5 - handle a case with truncated huffman tables
PR #1186: RAR5 - fix invalid type used for dictionary size mask
(OSS-Fuzz 14537)
PR #1187: RAR5 - fix integer overflow (OSS-Fuzz 14555)
PR #1190: RAR5 - RAR5 don't try to unpack entries marked as directories
(OSS-Fuzz 14574)
PR #1196: RAR5 - fix a potential SIGSEGV on 32-bit builds
OSS-Fuzz 2582: RAR - fix use after free if there is an invalid entry
OSS-Fuzz 14331: RAR5 - fix maximum owner name length
OSS-Fuzz 13965: RAR5 - use unsigned int for volume number + range check
Additional RAR5 reader changes:
- support symlinks, hardlinks, file owner, file group, versioned files
- change ARCHIVE_FORMAT_RAR_V5 to 0x100000
- set correct mode for readonly directories
- support readonly, hidden and system Windows file attributes
MFC after: 2 weeks
Relevant vendor changes:
Issue #795: XAR - do not try to add xattrs without an allocated name
PR #812: non-recursive option for extract and list
PR #958: support reading metadata from compressed files
PR #999: add --exclude-vcs option to bsdtar
Issue #1062: treat empty archives with a GNU volume header as valid
PR #1074: Handle ZIP files with trailing 0s in the extra fields
(Android APK archives)
PR #1109: Ignore padding in Zip extra field data (Android APK archives)
PR #1167: fix problems related to unreadable directories
Issue #1168: fix handling of strtol() and strtoul()
PR #1172: RAR5 - fix invalid window buffer read in E8E9 filter
PR #1174: ZIP reader - fix of MSZIP signature parsing
PR #1175: gzip filter - fix reading files larger than 4GB from memory
PR #1177: gzip filter - fix memory leak with repeated header reads
PR #1180: ZIP reader - add support for Info-ZIP Unicode Path Extra Field
PR #1181: RAR5 - fix merge_block() recursion
(OSS-Fuzz 12999, 13029, 13144, 13478, 13490)
PR #1183: fix memory leak when decompressing ZIP files with LZMA
PR #1184: fix RAR5 OSS-Fuzz issues 12466, 14490, 14491, 12817
OSS-Fuzz 12466: RAR5 - fix buffer overflow when parsing huffman tables
OSS-Fuzz 14490, 14491: RAR5 - fix bad shift-left operations
OSS-Fuzz 12817: RAR5 - handle a case with truncated huffman tables
PR #1186: RAR5 - fix invalid type used for dictionary size mask
(OSS-Fuzz 14537)
PR #1187: RAR5 - fix integer overflow (OSS-Fuzz 14555)
PR #1190: RAR5 - RAR5 don't try to unpack entries marked as directories
(OSS-Fuzz 14574)
PR #1196: RAR5 - fix a potential SIGSEGV on 32-bit builds
OSS-Fuzz 2582: RAR - fix use after free if there is an invalid entry
OSS-Fuzz 14331: RAR5 - fix maximum owner name length
OSS-Fuzz 13965: RAR5 - use unsigned int for volume number + range check
Additional RAR5 reader changes:
- support symlinks, hardlinks, file owner, file group, versioned files
- change ARCHIVE_FORMAT_RAR_V5 to 0x100000
- set correct mode for readonly directories
- support readonly, hidden and system Windows file attributes
NOTE: a version bump of libarchive will happen in the following days
Summary:
loader has grown to the point it's overflowing the existing 307200 bytes
allocated to it in the HFS boot partition on the isos.
Bump the space reservation up to 500k.
Submitted by: Brandon Bergren
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D20301
Summary:
PowerPC kernels are fully position independent, just like kernel modules.
The same fixups that are done for modules therefore need to be done to the
kernel, else symbol resolution in, e.g., DTrace, cannot resolve the kernel
symbols, so only addresses in the kernel are printed, while kernel module
symbols are printed.
Test Plan:
Run lockstat on powerpc64. Note symbols are resolved for kernel and
modules.
Reviewed By: markj
Differential Revision: https://reviews.freebsd.org/D20316
This allows replacing "sys/eventfilter.h" includes with "sys/_eventfilter.h"
in other header files (e.g., sys/{bus,conf,cpu}.h) and reduces header
pollution substantially.
EVENTHANDLER_DECLARE and EVENTHANDLER_LIST_DECLAREs were moved out of .c
files into appropriate headers (e.g., sys/proc.h, powernv/opal.h).
As a side effect of reduced header pollution, many .c files and headers no
longer contain needed definitions. The remainder of the patch addresses
adding appropriate includes to fix those files.
LOCK_DEBUG and LOCK_FILE_LINE_ARG are moved to sys/_lock.h, as required by
sys/mutex.h since r326106 (but silently protected by header pollution prior
to this change).
No functional change (intended). Of course, any out of tree modules that
relied on header pollution for sys/eventhandler.h, sys/lock.h, or
sys/mutex.h inclusion need to be fixed. __FreeBSD_version has been bumped.
Currently such routes are added with a link-level IFA, which is
plain wrong. Only after the insertion they get fixed by the special
link_rtrequest() ifa handler. This behaviour complicates routing code
and makes ifa selection more complex.
Streamline this process by explicitly moving link_rtrequest() logic
to the pre-insertion rt_getifa_fib() ifa selector. Avoid calling all
this logic in the loopback route case by explicitly specifying
proper rt_ifa inside the ifa_maintain_loopback_route().§
MFC after: 2 weeks
Differential Revision: https://reviews.freebsd.org/D20076
During boot we only want to measure things which *must*
be verified - this should provide more deterministic ordering.
Reviewed by: stevek
MFC after: 1 week
Sponsored by: Juniper Networks
Differential Revision: https://reviews.freebsd.org/D20297
does not ship a -lomp symlink. Also update OptionalObsoleteFiles for
this, and add 32-bit variants while here.
Submitted by: jbeich
PR: 237975
MFC after: 3 days
FDT data is sometimes used to configure usb devices which are hardwired into
an embedded system. Because the devices are instantiated by the usb
enumeration process rather than by ofwbus iterating through the fdt data, it
is somewhat difficult for a usb driver to locate fdt data that belongs to
it. In the past, various ad-hoc methods have been used, which can lead to
errors such applying configuration that should apply only to a hardwired
device onto a similar device attached by the user at runtime. For example,
if the user adds an ethernet device that uses the same driver as the builtin
ethernet, both devices might end up with the same MAC address.
These changes add a new usb_fdt_get_node() helper function that a driver can
use to locate FDT data that belongs to a single unique instance of the
device. This function locates the proper FDT data using the mechanism
detailed in the standard "usb-device.txt" binding document [1].
There is also a new usb_fdt_get_mac_addr() function, used to retrieve the
mac address for a given device instance from the fdt data. It uses
usb_fdt_get_node() to locate the right node in the FDT data, and attempts to
obtain the mac-address or local-mac-address property (in that order, the
same as linux does it).
The existing if_smsc driver is modified to use the new functions, both as an
example and for testing the new functions. Rpi and rpi2 boards use this
driver and provide the mac address via the fdt data.
[1] https://github.com/torvalds/linux/blob/master/Documentation/devicetree/bindings/usb/usb-device.txt
Differential Revision: https://reviews.freebsd.org/D20262
The source file was moved to base earlier and also improved upon,
but never compiled in. This patch will:
- Make a module in sys/modules
- Make lindebugfs depend on linuxkpi (for seq_file)
- Check if read/write functions are set before calling, DRM drivers
don't always set both of them.
Reviewed by: hps
Approved by: imp (mentor), hps
MFC after: 1 week
socket is non-blocking and connect() is not finished yet.
Initial patch developed by Steven Hartland in 2008 and adopted by me.
PR: 129169
Reported by: smh@
MFC after: 2 weeks
Submitted by: Patrick Mooney <pmooney@pfmooney.com>
Reviewed by: kib
Tested by: Patrick on SmartOS with Linux and Windows guests
Obtained from: Joyent
MFC after: 3 days
Differential Revision: https://reviews.freebsd.org/D20296
