Inheriting $PATH during the build phase can cause the build to fail when
compiling on a different system due to missing build tools or incompatible
versions somewhere in $PATH. This has cause build failures for us before
due to the jenkins slaves still running FreeBSD 10.
Listing the tools we depend on explicitly instead of just using whatever
happens to be in $PATH allows us to check that we don't accidentally add a
new build dependency.
All tools that do no need to be bootstrapped will now be symlinked to
${WORLDTMP}/legacy/bin and during the build phase $PATH will only contain
${WORLDTMP}. There is also a new variable "BOOTSTRAP_ALL_TOOLS" which can
be set to force compiling almost all bootstrap tools instead of symlinking
them. This will not bootstrap tools such as cp,mv, etc. since they may be
used during the build and for those we should really only be using POSIX
compatible options.
Furthermore, this change is required in order to be able to build on
non-FreeBSD hosts. While the same binaries may exist on Linux/MacOS they
often accept different flags or produce incompatible output.
Approved By: brooks (mentor)
Differential Revision: https://reviews.freebsd.org/D16815
riscv64sf has been broken due to duplicate symbols for months and
degrades the quality of universe builds. Remove it until this is
resolved leaving a comment to it is not re-added.
PR: 232085
Reviewed by: emaste
Approved by: re (gjb, kib)
Sponsored by: DARPA, AFRL
This fixes errors from the MK_CLANG_BOOTSTRAP/MK_LLD_BOOTSTRAP lookups
to not force using XCC/XLD but to rather just build them as normal by
allowing their own bootstrap logic to work.
MFC after: 3 weeks
X-MFC-with: r335711 r335769
Sponsored by: Dell EMC
This fixes a warning for each RISCV target during universe by passing in
the required CROSS_TOOLCHAIN setting which will in turn set
CROSS_BINUTILS_PREFIX correctly. It also ensures that a tinderbox build
uses the correct compiler for riscv. Previously it was using the shared
clang compiler instead of riscv64-gcc.
Reviewed by: bdrewery
Sponsored by: DARPA / AFRL
Differential Revision: https://reviews.freebsd.org/D16049
Need to handle LLD_BOOTSTRAP separately (for archs like i386).
This would be much better off with an off-by-default option like
SHARED_TOOLCHAIN that universe force-enabled. Then a normal buildworld
would store the toolchain there if enabled and otherwise in WORLDTMP
with only the 1 arch selected.
MFC after: 3 weeks
Sponsored by: Dell EMC
This is a bit noisy now but it was silent before leading to
wondering if it was doing anything.
MFC after: 1 week
Suggested by: rpokala
Sponsored by: Dell EMC
This works similar to WITH_SYSTEM_COMPILER added in r300354. It only
supports lld via WITH_LLD_BOOTSTRAP.
When both SYSTEM_COMPILER and SYSTEM_LINKER logic passes then libclang
will not build in cross-tools. If either check fails though then
libclang is built.
The .info is reworked to notify when libclang will be built since if
either clang or lld needs to be rebuilt, but not the other, the
notification can lead to confusion on why "clang is building".
-fuse-ld= is not used with this method so some combinations of compiler
and linker are expected to fail.
A new 'make test-system-linker' target is added to see the logic results.
Makefile.inc1:
CROSS_BINUTILS_PREFIX support had to be moved higher up so that XLD
could be set and MK_LLD_BOOTSTRAP disabled before checking SYSTEM_LINKER
logic as done with SYSTEM_COMPILER. This also required moving where
bsd.linker.mk was read since XLD needs to be set before parsing it. This
creates a situation where src.opts.mk can not test LINKER_FEATURES or
add LLD_BOOTSTAP to BROKEN_OPTIONS.
Reviewed by: emaste (earlier version)
Sponsored by: Dell EMC
Differential Revision: https://reviews.freebsd.org/D15894
This is mostly to allow using MAKEOBJDIRPREFIX in src-env.conf on stable where
src.sys.obj.mk is not going to be MFC'd. It is still valid on head but
effectively a NOP due to MAKEOBJDIRPREFIX being handled differently in
src.sys.obj.mk.
Reported by: eadler
MFC after: 1 week
Sponsored by: Dell EMC
bsd.compiler.mk. It's so fmake from older 9.x systems still
works (still a supported build config, and having the note here
will let us know when we can cull it more easily).
Also pull in a related change from include to sinclude from
arichardson@'s cross building work, as well as it's companion in
Makefile.inc1 with a note about why we do the odd thing there.
Submitted by: archardson
Differential Revision: https://reviews.freebsd.org/D14241
Linux /usr/bin/find doesn't understand the -mtime -0s flag.
Instead create a temporary file and compare that file's mtime to
sys/sys/param.h to check whether the clock is correct.
Reviewed By: jhb, imp
Approved By: jhb (mentor)
Differential Revision: https://reviews.freebsd.org/D14157
The supported targets are riscv64 and riscv64sf. Use the former when
building with a bare TARGET=riscv and it is the more common one.
Sponsored by: Netflix
There's no reason to create object directories for targets like 'installworld'
or 'distributeworld', and the others in this list. Specifying MK_AUTO_OBJ as a
make argument allows circumventing this if needed for some reason.
This fixes mergemaster creating a full object tree due to doing a 'make
installconfig' tree walk.
Reported by: Mark Millard
Sponsored by: Dell EMC Isilon
- This also adds in a _cleanobj step as needed.
- This redirects TARGET/TARGET_ARCH to NXB_TARGET/NXB_TARGET_ARCH in
Makefile.inc1 as the main build needs to be for MACHINE rather
than TARGET.
First build the toolchain and then use that as an external toolchain
to build the needed directories and NXB_TARGET-toolchain, all as
MACHINE files though via TARGET_TRIPLE=MACHINE_TRIPLE.
The NXBDIRS is evaluated in the 'everything' submake as it needs to be
based on TARGET's src.opts.mk values, such as MK_GCC=yes when building
on a MK_CLANG=yes MACHINE. This can likely be changed to a specific
_native-xtools-everything target later and the funky late evaluation
of SUBDIR_OVERRIDE removed.
X-MFC-With: r325001
Pointyhat to: bdrewery
MFC after: 2 months
Sponsored by: Dell EMC Isilon
- Don't discard SRCCONF value since it may incorrectly have MAKEOBJDIRPREFIX
in it.
- Add note about src.conf not being a suitable place for MAKEOBJDIRPREFIX.
Sponsored by: Dell EMC Isilon
Without this the user has to mess with 'make -f Makefile.inc1 ...' to figure
out where the files are installed in the OBJDIR and then they need to copy them
to where they really wanted them. Using DESTDIR may be problematic after
r325001 as well.
The files will be installed to DESTDIR/NXTP where NXTP defaults to /nxb-bin.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
The top of Makefile.inc1 requires TARGET/TARGET_ARCH be defined. Just
building 'make xdev' would already set them, so this error was never
triggered. Moving it to Makefile fixes the problem.
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
Make armv7 as a new MACHINE_ARCH.
Copy all the places we do armv6 and add armv7 as basically an
alias. clang appears to generate code for armv7 by default. armv7 hard
float isn't supported by the the in-tree gcc, so it hasn't been
updated to have a new default.
Support armv7 as a new valid MACHINE_ARCH (and by extension
TARGET_ARCH).
Add armv7 to the universe build.
Differential Revision: https://reviews.freebsd.org/D12010
Consolidate all the regular expressions to convert from MACHINE_ARCH
to MACHINE_CPUARCH into a variable and use that variable in preference
to the almost identical copies in the tree (which should have been
identical).
Differential Revision: https://reviews.freebsd.org/D11986
Previously we excluded riscv from make universe / tinderbox if the
required xtoolchain package was not installed. Make that logic generic
so that we can loop over multiple architectures, in preparation to test
patches to have other architectures rely on external toolchain.
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D11652
It appears that the same arches that lack GENERIC kernel configs also lack
LINT. But enough different arches get built to ensure a kernel change
should build everywhere (32 and 64 bit, clang and old gcc, little and big
endian).
just the GENERIC kernels for each arch (including variations such as
GENERIC-NODEBUG, GENERIC64, etc).
This helps with quickly doing a test build for all[*] arches without
building dozens of variant kernels for the arches that have lots of
hardware/board/system variations.
[*] Not all arches have a generic kernel (but they probably should for
test-building purposes, even if it can't boot on any real hardware).
Normally META_MODE ignores host files for "meta mode" decisions on whether a
file should be rebuilt or not. This is because a simple installworld can
update timestamps and cause the next build to rebuild all host tools, when the
previous ones may not have any changes in the source tree. These tools are
normally still ABI compatible. They are only rebuilt if NO_META_IGNORE_HOST is
set from the workaround/hack in r301467.
One of the major problems with this is when a host tool has objects spread
across many revisions that have mixed-ABI. For example, if struct stat were to
change on the host, some objects for a tool may have different ideas of that
struct's definition. If just 1 source file were modified and rebuilt and
linked into the tool, then that toll will have mixed-ABI objects and crash.
This exact thing happened with the ino64 commit in r301467 followed by a
trivial update to libbfd in r318750. The resulting binary would crash in
buildworld.
Sponsored by: Dell EMC Isilon
Right after cross-tools, a compiler-metadata.mk file is created that
stores all of the bsd.compiler.mk metadata. It is then read in
with a fail-safe during installworld time.
The file is explicitly removed when invoking cross-tools to ensure that
a stale file is not left around from odd manual 'make _cross-tools' ->
'make installworld' invocations.
This fixes several issues:
- With WITH_SYSTEM_COMPILER (default yes on head and no on releng/11.0):
If you build on a system where the bootstrap compiler does not
build due to the host compiler matching the in-tree one, but then
installworld on another system where that logic fails (a
bootstrap compiler is needed), the installworld immediately fails
with:
sh: cc: not found
Note that fixing this logic may then hit a case where a rebuild is
attempted in installworld. Normally cc would be ran with
'CFLAGS+=ERROR-tried-to-rebuild-during-make-install' to cause an
error such as:
cc: error: no such file or directory: 'ERROR-tried-to-rebuild-during-make-install'
However, now it will just fail with the 'cc: not found' error.
Inspection of the compile line will show
'ERROR-tried-to-rebuild-during-make-install'; It's not useful to
set CC to anything other than 'cc' during install as it is more
helpful to see the attempted compile rather than some other bogus
error.
- This now avoids running bsd.compiler.mk (cc executions) even more
during installworld. There are compiler-dependent SUBDIR in the
tree which required having a compiler during install.
There is at least 1 case where CC is still executed in the install,
such as from a LOOKUP!= in secure/lib/libcrypto/Makefile.inc checking
for 'vzeroall' support. This is not significant for installworld
as the lookup has a fallback (and hides its error) and only modifies CFLAGS,
thus it's not worth fixing.
PR: 212877
MFC after: 2 weeks
Sponsored by: Dell EMC Isilon
As of r316629 FreeBSD/arm64 uses the in-tree LLD linker by default, and
does not require an external an aarch64-binutils port or package.
Sponsored by: The FreeBSD Foundation
Hardfloat is now default (use riscv64sf as TARGET_ARCH
for softfloat).
Sponsored by: DARPA, AFRL
Differential Revision: https://reviews.freebsd.org/D8529
Summary:
The Freescale e500v2 PowerPC core does not use a standard FPU.
Instead, it uses a Signal Processing Engine (SPE)--a DSP-style vector processor
unit, which doubles as a FPU. The PowerPC SPE ABI is incompatible with the
stock powerpc ABI, so a new MACHINE_ARCH was created to deal with this.
Additionaly, the SPE opcodes overlap with Altivec, so these are mutually
exclusive. Taking advantage of this fact, a new file, powerpc/booke/spe.c, was
created with the same function set as in powerpc/powerpc/altivec.c, so it
becomes effectively a drop-in replacement. setjmp/longjmp were modified to save
the upper 32-bits of the now-64-bit GPRs (upper 32-bits are only accessible by
the SPE).
Note: This does _not_ support the SPE in the e500v1, as the e500v1 SPE does not
support double-precision floating point.
Also, without a new MACHINE_ARCH it would be impossible to provide binary
packages which utilize the SPE.
Additionally, no work has been done to support ports, work is needed for this.
This also means no newer gcc can yet be used. However, gcc's powerpc support
has been refactored which would make adding a powerpcspe-freebsd target very
easy.
Test Plan:
This was lightly tested on a RouterBoard RB800 and an AmigaOne A1222
(P1022-based) board, compiled against the new ABI. Base system utilities
(/bin/sh, /bin/ls, etc) still function appropriately, the system is able to boot
multiuser.
Reviewed By: bdrewery, imp
Relnotes: yes
Differential Revision: https://reviews.freebsd.org/D5683
1. Always calculate what the expected values are.
2. Add 'make test-system-compiler' to show all of the computed values
vs the wanted values.
3. Extend the .info line to buildkernel/kernel-toolchain/toolchain/_cross-tools.
4. Consolidate all of the logic to one condition.
MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division
This also adds missing s/aarch64/arm64 to the sys.mk version and also
adds back armv6hf for universe since it was added to the sys.mk version
in r300438.
MFC after: 3 days
Sponsored by: EMC / Isilon Storage Division
Differential Revision: https://reviews.freebsd.org/D7159
installkernel is technically META_MODE safe but doesn't need an explicit
approval to use it since it's all disabled via .PHONY.
world uses 'make -B installworld' which already will disable META_MODE
via the -B (.MAKE.MODE=compat) usage.
Approved by: re (implicit)
Sponsored by: EMC / Isilon Storage Division
This was a flaw in my change in r287903 but also in the original change
in r282156 since it used empty(${TARGETS}) rather than empty(TARGETS).
Reported by: lidl
Approved by: re (gjb)
Sponsored by: EMC / Isilon Storage Division
This broke in r301887 with the meta mode whitelist. 'make showconfig'
still needs WITH_META_MODE support.
Approved by: re (implicit)
Sponsored by: EMC / Isilon Storage Division
This was broken in r301888.
fmake does not look in share/mk by default and thus does not yet
have MK_META_MODE set with default.
Pointyhat to: bdrewery
Approved by: re (implicit)
Sponsored by: EMC / Isilon Storage Division
- Move the sys.mk filemon requirement to bsd.init.mk as a warning.
This is intended only to show when building directly in a subdirectory
without filemon loaded.
- Move the error into Makefile and only apply it when building
from the META_TGT_WHITELIST target list.
-DNO_FILEMON can be used to suppress both the warning and the error but
makes WITH_META_MODE less useful. It will only compare build commands
in this mode rather than track all dependencies.
This fixes installing from a jail which doesn't need filemon in this
phase [1].
Reported by: Nikolai Lifanov <lifanov@mail.lifanov.com> [1]
Approved by: re (implicit)
Sponsored by: EMC / Isilon Storage Division
META_TGT_WHITELIST is added to define which build targets are safe for
meta mode. See comments for more details.
This fixes 'make delete-old-libs' to properly show the interactive
prompt.
Approved by: re (implicit)
Sponsored by: EMC / Isilon Storage Division
META_MODE may create cookies during staging of files to WORLDTMP that would
also prevent installation of the files to the final DESTDIR, since the cookie
already exists. This is not yet the case but will be soon. Prevent other
similar issues by disabling META_MODE for any top-level install targets.
Sponsored by: EMC / Isilon Storage Division
for the armv6 ABI switch. This also make WITH_LIBSOFT functional on
the arm platform. As a transition thing, this seems to work even
without switching the ABI (we basically build the same libraries
twice when MK_LIBSOFT=yes until the ABI cut over next
month). MK_LIBSOFT remains default no.
The inclusion of .MAKE.DEPENDFILE (.depend) has special logic in make
to ignore stale/missing dependencies. bmake 20160220 added a '.dinclude'
directive that uses the special logic for .depend when including the file.
This fixes a build error when a file is moved or deleted that exists in a
.depend.OBJ file. This happened in r292782 when sha512c.c "moved" and an
incremental build of lib/libmd would fail with:
make: don't know how to make /usr/src/lib/libcrypt/../libmd/sha512c.c. Stop
Now this will just be seen as a stale dependency and cause a rebuild:
make: /usr/obj/usr/src/lib/libmd/.depend.sha512c.o, 13: ignoring stale .depend for /usr/src/lib/libcrypt/../libmd/sha512c.c
--- sha512c.o ---
...
This rebuild will only be done once since the .depend.sha512c.o will
be updated on the build with the -MF flags.
This also removes -MP being passed for the .depend.OBJ generation (which
would create fake targets for system headers) since the logic is no
longer needed to protect from missing files.
Sponsored by: EMC / Isilon Storage Division
architectures we do not provide upstream pkg(8) packages.
This is not tied to anything as-is, and likely will break
your system if used (based on experience with testing with
powerpc).
There is an overwhelming amount of evil happening here,
so until the issues are fixed, it will not be tied into the
'packages' target.
Sponsored by: The FreeBSD Foundation
