1
0
mirror of https://git.FreeBSD.org/ports.git synced 2025-01-03 06:04:53 +00:00
Mirror of the FreeBSD ports git repo https://git.FreeBSD.org/ports.git .
Go to file
Kubilay Kocak e4c2b30ce8 lang/python{27,35,36,37,38}: Add closefrom(2) support
A single close(fd) syscall is cheap, but when MAXFDS (maximum file
descriptor number) is high, the loop calling close(fd) on each file
descriptor can take several milliseconds.

The default value of subprocess.Popen "close_fds" parameter changed to True
in Python 3. Compared to Python 2, close_fds=True can make Popen 10x
slower: see bpo-37790 [1]

The present workaround on FreeBSD to improve performance is to load and
mount the fdescfs kernel module, but this is not enabled by default.

This change adds minimum viable (and upstreamable) closefrom(2) syscall
support to Python's subprocess and posix modules, improving performance
significantly for loads that involve working with many processes, such as
diffoscope, ansible, and many others.

For additional optimizations, upstream recently (3.8) landed posix_spawn(2)
support [3] and has stated that they will adopt close_range(2) after Linux
merges it [4]. Linux/FreeBSD developers are already collaborating on
ensuring compatible implementations, with FreeBSD's implementation pending
in D21627. [5]

Thank you emaste, cem, kevans for providing analysis, input,
clarifications, comms/upstream support and patches.

[1] https://bugs.python.org/issue37790
[2] https://bugs.python.org/issue38061
[3] https://bugs.python.org/issue35537
[4] https://lwn.net/Articles/789023/
[5] https://reviews.freebsd.org/D21627

Additional References:

https://bugs.python.org/issue8052
https://bugs.python.org/issue11284
https://bugs.python.org/issue13788
https://bugs.python.org/issue1663329
https://www.python.org/dev/peps/pep-0446/

PR:		242274, 221700
Submitted by:	kevans (emaste, cem)
Approved by:	koobs (python (maintainer), santa)
2019-11-29 10:55:00 +00:00
accessibility accessibility/redshift: force rebuild after r518170 2019-11-22 22:53:43 +00:00
arabic
archivers archivers/zopfli: update to 1.0.3 2019-11-28 18:26:58 +00:00
astro - Update to r4379 2019-11-28 18:13:15 +00:00
audio - Switch to USES=localbase 2019-11-29 09:40:29 +00:00
base Update binutils to 2.33.1 2019-11-08 08:47:24 +00:00
benchmarks Clean up after java/openjdk6 and java/openjdk6-jre removal 2019-11-26 21:46:12 +00:00
biology biology/unanimity: Remove erroneous BROKEN 2019-11-28 05:46:04 +00:00
cad cad/kicad: update to 5.1.5 2019-11-28 13:02:49 +00:00
chinese Fix build with src r354909 applied. 2019-11-25 23:25:52 +00:00
comms Clean up after java/openjdk6 and java/openjdk6-jre removal 2019-11-26 21:46:12 +00:00
converters Chase lang/sbcl upgrade with PORTREVISION bump on *-sbcl ports 2019-11-27 16:21:18 +00:00
databases Update to 6.7.3 2019-11-29 00:23:37 +00:00
deskutils Clean up after java/openjdk6 and java/openjdk6-jre removal 2019-11-26 21:46:12 +00:00
devel Update to 3.11.0 2019-11-29 10:40:45 +00:00
dns dns/amass: Update 3.3.2 -> 3.4.0 2019-11-29 07:25:56 +00:00
editors editors/o: Update 2.10.1 -> 2.10.2 2019-11-29 08:04:23 +00:00
emulators emulators/yuzu: update to s20191127 2019-11-28 00:47:56 +00:00
finance Clean up after java/openjdk6 and java/openjdk6-jre removal 2019-11-26 21:46:12 +00:00
french Update to 1.6.0 2019-11-29 01:44:09 +00:00
ftp Clean up after java/openjdk6 and java/openjdk6-jre removal 2019-11-26 21:46:12 +00:00
games update games/tome4 to 1.6.4 2019-11-28 03:22:36 +00:00
german Update to 2019.11.17 2019-11-18 18:13:33 +00:00
graphics graphics/dmtx-utils: update to 0.7.6 2019-11-29 02:53:24 +00:00
hebrew
hungarian Mark a few ports BROKEN, unfetchable 2019-11-06 12:48:32 +00:00
irc irc/weechat: Chase previous commit 2019-11-26 07:14:17 +00:00
japanese Fix build with src r354909 applied. 2019-11-25 23:25:52 +00:00
java * Teach javavmwrapper about "13" as a version specifier 2019-11-27 00:56:24 +00:00
Keywords
korean Add missing USES={gnome,php,sdl} to the rest 2019-11-08 14:47:00 +00:00
lang lang/python{27,35,36,37,38}: Add closefrom(2) support 2019-11-29 10:55:00 +00:00
mail mail/imapfilter: Update version 2.6.14=>2.6.16 2019-11-29 08:54:56 +00:00
math - Update to 1.2.4.2 2019-11-28 15:03:53 +00:00
misc New port: misc/opennn: Open neural networks library 2019-11-29 01:58:05 +00:00
Mk Clean up after java/openjdk6 and java/openjdk6-jre removal 2019-11-26 21:46:12 +00:00
multimedia multimedia/mlt: remove SWFDEC from default OPTIONS 2019-11-28 17:26:02 +00:00
net net/inlets: Update 2.6.1 -> 2.6.3 2019-11-29 07:29:09 +00:00
net-im net-im/py-matrix-synapse: Update to 1.6.1 which also fixes two vulnerabitilies 2019-11-28 17:05:12 +00:00
net-mgmt net-mgmt/rubygem-riemann-client: fix ipv6 2019-11-25 13:52:28 +00:00
net-p2p Port cleanup. 2019-11-29 09:07:24 +00:00
news news/tin: update to 2.4.3 2019-11-29 02:42:31 +00:00
polish Update to 20191117 2019-11-18 18:13:38 +00:00
ports-mgmt Upgrade ports-mgmt/pkg-provides to v0.6.0 2019-11-23 09:29:23 +00:00
portuguese Clean up after java/openjdk6 and java/openjdk6-jre removal 2019-11-26 21:46:12 +00:00
print Clean up after java/openjdk6 and java/openjdk6-jre removal 2019-11-26 21:46:12 +00:00
russian Update to 5.2.4 2019-11-12 08:00:51 +00:00
science science/tinker: Update 8.7.1 -> 8.7.2 2019-11-29 07:44:51 +00:00
security Document net-im/py-matrix-synapse vulnerabilities 2019-11-28 15:44:53 +00:00
shells shells/ammonite: Update to 1.8.2 2019-11-24 16:50:55 +00:00
sysutils Update to 3.1.1 2019-11-29 00:25:02 +00:00
Templates
textproc - Update to 0.70.1 2019-11-28 21:49:26 +00:00
Tools
ukrainian
vietnamese Add USES=xorg USES=gl, ports categories v 2019-11-07 17:19:51 +00:00
www www/colly: Update 1.2.0-32 -> 2.0.0 2019-11-29 07:33:35 +00:00
x11 Chase lang/sbcl upgrade with PORTREVISION bump on *-sbcl ports 2019-11-27 16:21:18 +00:00
x11-clocks KDE Applications: update to 19.08.3 2019-11-09 08:13:40 +00:00
x11-drivers Add USES=xorg USES=gl, ports categories x 2019-11-07 18:36:55 +00:00
x11-fm - Update thunar to 1.8.11 2019-11-16 16:42:12 +00:00
x11-fonts x11-fonts/jmk-x11-fonts: update MASTER_SITES and unbreak. 2019-11-27 11:24:07 +00:00
x11-servers Add USES=xorg USES=gl, ports categories x 2019-11-07 18:36:55 +00:00
x11-themes x11-themes/adwaita-qt: update to 1.1.1 2019-11-23 20:58:53 +00:00
x11-toolkits Clean up after java/openjdk6 and java/openjdk6-jre removal 2019-11-26 21:46:12 +00:00
x11-wm x11-wm/stumpwm: Upgrade to version 19.11 2019-11-28 09:11:49 +00:00
.arcconfig
.gitattributes
.gitauthors
.gitignore
.gitmessage
CHANGES
CONTRIBUTING.md
COPYRIGHT
GIDs Update to 0.6.2. 2019-11-28 10:56:59 +00:00
LEGAL net/zerotier: update to 1.4.6, note new LICENSE 2019-11-25 13:37:36 +00:00
Makefile Retire the palm category (1999-2019) 2019-11-04 11:57:46 +00:00
MOVED After the release of GCC 7.5 the GCC 7 release branch has been closed 2019-11-28 07:01:37 +00:00
README
UIDs Update to 0.6.2. 2019-11-28 10:56:59 +00:00
UPDATING net/py-urllib3: Update to 1.25.6 2019-11-26 18:37:58 +00:00

This is the FreeBSD Ports Collection.  For an easy to use
WEB-based interface to it, please see:

	https://www.FreeBSD.org/ports

For general information on the Ports Collection, please see the
FreeBSD Handbook ports section which is available from:

	https://www.FreeBSD.org/doc/en_US.ISO8859-1/books/handbook/ports.html
		for the latest official version
	or:
	The ports(7) manual page (man ports).

These will explain how to use ports and packages.

If you would like to search for a port, you can do so easily by
saying (in /usr/ports):

	make search name="<name>"
	or:
	make search key="<keyword>"

which will generate a list of all ports matching <name> or <keyword>.
make search also supports wildcards, such as:

	make search name="gtk*"

For information about contributing to FreeBSD ports, please see the Porter's
Handbook, available at:

	https://www.FreeBSD.org/doc/en_US.ISO8859-1/books/porters-handbook/

NOTE:  This tree will GROW significantly in size during normal usage!
The distribution tar files can and do accumulate in /usr/ports/distfiles,
and the individual ports will also use up lots of space in their work
subdirectories unless you remember to "make clean" after you're done
building a given port.  /usr/ports/distfiles can also be periodically
cleaned without ill-effect.