e92b8e535a (origin/emacs-27) Remove raw carriage return characters fr...
5747a59a88 Recalculate default font when switching font backend
a2dd8c4234 * lisp/tab-line.el: Fix tab-line-format and tab-line-forma...
a7b8291b6c * etc/NEWS: Make the `--eval` example slightly more precise
f8254aad14 * lisp/image/image-converter.el: Support more ImageMagick ...
5beb269505 Support Unicode 13.0
3a671ad7ed Fix regression in wisent-total-conflicts
# Conflicts:
# etc/NEWS
* admin/unidata/BidiBrackets.txt:
* admin/unidata/BidiMirroring.txt:
* admin/unidata/Blocks.txt:
* admin/unidata/NormalizationTest.txt:
* admin/unidata/SpecialCasing.txt:
* admin/unidata/UnicodeData.txt:
* admin/unidata/copyright.html:
* test/manual/BidiCharacterTest.txt: Updated files imported from
Unicode v13.0.
* admin/unidata/blocks.awk: Add "Symbols for Legacy Computing" to
known aliases.
* lisp/international/fontset.el (script-representative-chars)
(setup-default-fontset): Add new scripts.
* lisp/international/characters.el: Set syntax for Symbols for
Legacy Computing characters. Update setting of char-width-table.
* lisp/international/mule-cmds.el (ucs-names): Update ranges of
characters for which we want names in 'ucs-names'.
* test/lisp/international/ucs-normalize-tests.el
(ucs-normalize-tests--failing-lines-part1)
(ucs-normalize-tests--failing-lines-part2): Update according to
'ucs-normalize-check-failing-lines'.
(unidata-prop): Use defstruct to define the 6 accessor functions.
(unidata-gen-table-character, unidata-gen-table, unidata-gen-table-name)
(unidata-check): Move common code out of `if`.
(unidata-word-list-diff, unidata-split-decomposition):
Move common code out of `if`; use `push`.
* admin/unidata/Makefile.in (extraclean): Don't delete macuvs.h,
because it's a checked-in file, and make targets altering
checked-in files is confusing.
* admin/unidata/Makefile.in (maintainer-clean):
Instead of removing macuvs.h here ...
(extraclean): ... Remove it here.
* admin/unidata/uvs.el (uvs-print-table-ivd):
Add to comment explaining why macuvs.h is in Git.
* src/macuvs.h: Regenerate and re-add to Git.
It can be generated automatically and easily during a normal
bootstrap, so there’s no need to keep it in the repository.
* admin/unidata/Makefile.in (maintainer-clean):
Behave like extraclean and remove macuvs.h etc.
* src/macuvs.h: Remove.
* src/Makefile.in (extraclean): Remove TAGS and config.in.
* Makefile.in (extraclean): Remove info, configure and
emacsver.texi in extraclean.
* admin/unidata/Makefile.in (extraclean): Make it depend on
distclean to remove .elc files.
* leim/Makefile.in (extraclean): Depend on bootstrap-clean to
remove generated .el files.
* lib/Makefile.in (extraclean): Added target to remove sys
directory and run distclean.
* lisp/Makefile.in (extraclean): Also remove loaddefs*.el~ files.
* admin/unidata/copyright.html:
* admin/unidata/UnicodeData.txt:
* admin/unidata/SpecialCasing.txt:
* admin/unidata/NormalizationTest.txt:
* admin/unidata/Blocks.txt:
* admin/unidata/BidiMirroring.txt:
* admin/unidata/BidiBrackets.txt: New versions from Unicode 12.0.
* admin/unidata/unidata-gen.el (unidata-gen-file):
* admin/unidata/blocks.awk (name2alias): Adapt to changes in
new data files.
* admin/notes/unicode: Update and improve instructions for
importing a new Unicode Standard.
* lisp/international/characters.el (char-width-table): Update
lists of characters according to Unicode 12.0.
* lisp/international/fontset.el (script-representative-chars):
Add characters from new scripts to 'script-representative-chars'.
(otf-script-alist): Update according to data on the MS site.
* lisp/international/mule-cmds.el (ucs-names): Update unused
ranges of codepoints according to Unicode 12.0.
* test/lisp/international/ucs-normalize-tests.el
(ucs-normalize-tests--failing-lines-part1)
(ucs-normalize-tests--failing-lines-part2): Update for the new
NormalizationTest.txt file.
* test/manual/BidiCharacterTest.txt: Update with the new
version from Unicode 12.0.
I audited use of lsh in the Lisp source code, and fixed the
glitches that I found. While I was at it, I replaced uses of lsh
with ash when either will do. Replacement is OK when either
argument is known to be nonnegative, or when only the low-order
bits of the result matter, and is a (minor) win since ash is a bit
more solid than lsh nowadays, and is a bit faster.
* lisp/calc/calc-ext.el (math-check-fixnum):
Prefer most-positive-fixnum to (lsh -1 -1).
* lisp/vc/vc-hg.el (vc-hg-state-fast): When testing fixnum width,
prefer (zerop (ash most-positive-fixnum -32)) to (zerop (lsh -1
32)) (Bug#32485#11).
* lisp/emacs-lisp/bytecomp.el (byte-compile-lapcode):
Tighten sanity-check for bytecode overflow, by checking that the
result of (ash pc -8) is nonnegative. Formerly this check was not
needed since lsh was used and the number overflowed differently.
* lisp/net/dns.el (dns-write): Fix some obvious sign typos in
shift counts. Evidently this part of the code has never been
exercised.
* lisp/progmodes/hideif.el (hif-shiftleft, hif-shiftright):
* lisp/term/common-win.el (x-setup-function-keys):
Simplify.
* admin/unidata/unidata-gen.el, admin/unidata/uvs.el:
* doc/lispref/keymaps.texi, doc/lispref/syntax.texi:
* doc/misc/calc.texi, doc/misc/cl.texi, etc/NEWS.19:
* lisp/arc-mode.el, lisp/calc/calc-bin.el, lisp/calc/calc-comb.el:
* lisp/calc/calc-ext.el, lisp/calc/calc-math.el:
* lisp/cedet/semantic/wisent/comp.el, lisp/composite.el:
* lisp/disp-table.el, lisp/dos-fns.el, lisp/edmacro.el:
* lisp/emacs-lisp/bindat.el, lisp/emacs-lisp/byte-opt.el:
* lisp/emacs-lisp/bytecomp.el, lisp/emacs-lisp/cl-extra.el:
* lisp/erc/erc-dcc.el, lisp/facemenu.el, lisp/gnus/message.el:
* lisp/gnus/nndoc.el, lisp/gnus/nnmaildir.el, lisp/image.el:
* lisp/international/ccl.el, lisp/international/fontset.el:
* lisp/international/mule-cmds.el, lisp/international/mule.el:
* lisp/json.el, lisp/mail/binhex.el, lisp/mail/rmail.el:
* lisp/mail/uudecode.el, lisp/md4.el, lisp/net/dns.el:
* lisp/net/ntlm.el, lisp/net/sasl.el, lisp/net/socks.el:
* lisp/net/tramp.el, lisp/obsolete/levents.el:
* lisp/obsolete/pgg-parse.el, lisp/org/org.el:
* lisp/org/ox-publish.el, lisp/progmodes/cc-defs.el:
* lisp/progmodes/ebnf2ps.el, lisp/progmodes/hideif.el:
* lisp/ps-bdf.el, lisp/ps-print.el, lisp/simple.el:
* lisp/tar-mode.el, lisp/term/common-win.el:
* lisp/term/tty-colors.el, lisp/term/xterm.el, lisp/vc/vc-git.el:
* lisp/vc/vc-hg.el, lisp/x-dnd.el, test/src/data-tests.el:
Prefer ash to lsh when either will do.
* admin/unidata/IVD_Sequences.txt: New version from
http://www.unicode.org/ivd/, the 2017-12-12 version of the Unicode
Ideographic Variation Database.
* src/macuvs.h: Regenerated.
Most of this change is to boilerplate commentary such as license URLs.
This change was prompted by ftp://ftp.gnu.org's going-away party,
planned for November. Change these FTP URLs to https://ftp.gnu.org
instead. Make similar changes for URLs to other organizations moving
away from FTP. Also, change HTTP to HTTPS for URLs to gnu.org and
fsf.org when this works, as this will further help defend against
man-in-the-middle attacks (for this part I omitted the MS-DOS and
MS-Windows sources and the test tarballs to keep the workload down).
HTTPS is not fully working to lists.gnu.org so I left those URLs alone
for now.
* admin/unidata/unidata-gen.el (unidata-check): Do not test special
casing mapping of characters since that mapping is not constructed from
the unidata.txt file.
Also, check for integer decoder and cons char earlier so that less
unnecessary processing is performed.
* admin/unidata/unidata-gen.el (unidata-gen-file):
Get Copyright line from copyright.html.
Put information in file header, not separate README.
(unidata-gen-charprop): Mention the source location.
* lisp/international/README: Remove file.
This is better for parallel builds, eg it eliminates race
conditions from having one process write multiple files.
* admin/unidata/Makefile.in (lparen, unifiles): New variables.
Parse unidata-gen.el, not charprop.el, to get the list of uni- files.
(all): Explicitly list the output lisp files.
(PHONY_EXTRAS): Remove.
(${unidir}/charprop.el): Change rule to just be for this file.
(${unifiles}): New rule to write each unicode lisp file.
(extraclean): Simplify.
* admin/unidata/unidata-gen.el (unidata-gen-charprop):
Quieten in batch mode.
(unidata-gen-files): Remove, no longer used.
* lisp/loadup.el: Update command-line parser.
* admin/unidata/unidata-gen.el (unidata-gen-file):
Make it work as a stand-alone function in batch mode.
(unidata-gen-files): Pass extra arguments to unidata-gen-file.
* admin/unidata/unidata-gen.el (unidata-file-alist):
Rename from unidata-prop-alist. All users changed.
Use file name rather than property name as the key.
(unidata-prop-prop): New function.
(unidata-prop-index, unidata-prop-generator, unidata-prop-docstring)
(unidata-prop-describer, unidata-prop-default, unidata-prop-val-list):
Change to parse the argument rather than unidata-prop-alist.
(unidata-gen-table-character, unidata-gen-table)
(unidata-gen-table-symbol, unidata-gen-table-integer)
(unidata-gen-table-numeric, unidata-gen-table-word-list)
(unidata-gen-table-name, unidata-gen-table-decomposition)
(unidata-gen-table-special-casing): Pass index as an argument.
(unidata-check): Adapt to unidata-file-alist.
Pass index to generator functions.
(unidata-gen-files): Adapt to unidata-file-alist.
Write each output file once only. Overwrite rather than delete.
Implement unconditional special casing rules defined in Unicode standard.
Among other things, they deal with cases when a single code point is
replaced by multiple ones because single character does not exist (e.g.
‘fi’ ligature turning into ‘FL’) or is not commonly used (e.g. ß turning
into SS).
* admin/unidata/SpecialCasing.txt: New data file pulled from Unicode
standard distribution.
* admin/unidata/README: Mention SpecialCasing.txt.
* admin/unidata/unidata-get.el (unidata-gen-table-special-casing,
unidata-gen-table-special-casing--do-load): New functions generating
‘special-uppercase’, ‘special-lowercase’ and ‘special-titlecase’
character Unicode properties built from the SpecialCasing.txt Unicode
data file.
* src/casefiddle.c (struct casing_str_buf): New structure for
representing short strings used to handle one-to-many character
mappings.
(case_character_imlp): New function which can handle one-to-many
character mappings.
(case_character, case_single_character): Wrappers for the above
functions. The former may map one character to multiple (or no)
code points while the latter does what the former used to do (i.e.
handles one-to-one mappings only).
(do_casify_natnum, do_casify_unibyte_string,
do_casify_unibyte_region): Use case_single_character.
(do_casify_multibyte_string, do_casify_multibyte_region): Support new
features of case_character.
* (do_casify_region): Updated to reflact do_casify_multibyte_string
changes.
(casify_word): Handle situation when one character-length of a word
can change affecting where end of the word is.
(upcase, capitalize, upcase-initials): Update documentation to mention
limitations when working on characters.
* test/src/casefiddle-tests.el (casefiddle-tests-char-properties):
Add test cases for the newly introduced character properties.
(casefiddle-tests-casing): Update test cases which are now passing.
* test/lisp/char-fold-tests.el (char-fold--ascii-upcase,
char-fold--ascii-downcase): New functions which behave like old ‘upcase’
and ‘downcase’.
(char-fold--test-match-exactly): Use the new functions. This is needed
because otherwise fi and similar characters are turned into their multi-
-character representation.
* doc/lispref/strings.texi: Describe issue with casing characters versus
strings.
* doc/lispref/nonascii.texi: Describe the new character properties.
Some tests are marked as expected to fail.
* test/lisp/international/ucs-normalize-tests.el: New tests.
* admin/unidata/NormalizationTest.txt: Add data for tests.
* admin/unidata/README: Add URL for NormalizationTest.txt.
* admin/notes/unicode: Add note about running (and updating the data
for) the new tests. Remove note about normalization being unsupported.
* admin/unidata/UnicodeData.txt:
* admin/unidata/Blocks.txt:
* admin/unidata/BidiMirroring.txt:
* admin/unidata/BidiBrackets.txt: Update from Unicode 9.0.0beta.
* admin/unidata/unidata-gen.el (unidata-gen-files): Bind
'coding-system-for-read' to 'utf-8, as various Unicode data files
now actually use non-ASCII characters.
(unidata-setup-list, unidata-get-name): Support the new Tangut
Ideographs block.
* lisp/international/characters.el (standard-case-table): Add new
characters from Unicode 9.0.0.
(standard-category-table): Add Arabic block u+08A0..u+08FF. Add
Cyrillic Extended-C block.
(char-width-table): Update ranges per Unicode 9.0.0.
* lisp/international/fontset.el (script-representative-chars): Add
new scripts defined by Unicode 9.0.0.
(otf-script-alist): Add new OTF script tags.
* lisp/international/mule-cmds.el (ucs-names): Update ranges per
Unicode 9.0.0 additions.
* nt/configure.bat: Remove everything except the blurb about the
new build procedure.
* make-dist: Remove references to makefile.w32-in in various
directories, and to files in nt/ that were deleted.
* etc/NEWS: Mention the fact that the files were dropped.
* etc/NEWS: Update wording since Unicode 8.0 is no longer in draft
status.
* test/BidiCharacterTest.txt: Update from Unicode 8.0.
* admin/unidata/BidiMirroring.txt:
* admin/unidata/BidiBrackets.txt:
* admin/unidata/UnicodeData.txt: Update from Unicode 8.0.
* admin/unidata/Makefile.in (AWK): New, set by configure.
(all): Add charscript.el.
(blocks): New variable.
(charscript.el, ${unidir}/charscript.el): New targets.
(extraclean): Also remove generated charscript.el.
* admin/unidata/blocks.awk: New script.
* admin/unidata/Blocks.txt: New data file, from unicode.org.
* lisp/international/characters.el: Load charscript.
* src/Makefile.in (charscript): New variable.
(${charscript}): New target.
(${lispintdir}/characters.elc): Depend on charscript.elc.
(temacs$(EXEEXT)): Depend on charscript.
; * admin/unidata/README: Mention Blocks.txt.
; * .gitignore: Add lisp/international/charscript.el.
* admin/unidata/unidata-gen.el (unidata-setup-list): Don't ignore
surrogates. This avoids assigning them the default
general-category of 'Cn', i.e. unassigned codepoints.
(unidata-get-name): Give surrogates synthetic names.
* admin/unidata/Makefile.in (unifiles):
Use AM_V_at instead of AM_V_GEN, since this doesn't generate a file.
* lib-src/Makefile.in (AM_V_GEN, am__v_GEN_, am__v_GEN_0, am__v_GEN_1)
(AM_V_at, am__v_at_, am__v_at_0, am__v_at_1):
New macros, copied from ../src/Makefile.in.
* unidata/Makefile.in (AM_DEFAULT_VERBOSITY, AM_V_GEN, am__v_GEN_)
(am__v_GEN_0, am__v_GEN_1, AM_V_at, am__v_at_, am__v_at_0, am__v_at_1):
New macros, from ../src/Makefile.in.
(${top_srcdir}/src/macuvs.h, %.elc, unidata.txt)
(${unidir}/charprop.el, unifiles): Use them.
(PHONY_EXTRAS): New macro.
(.PHONY, ${unidir}/charprop.el): Use it.
(FORCE): Remove; all uses replaced by PHONY_EXTRAS manipulation.
(${unidir}/charprop.el, extraclean):
Use 'make' conditionals rather than shell.
