mirror of
https://git.savannah.gnu.org/git/emacs.git
synced 2024-12-14 09:39:42 +00:00
6129cd03e2
b73f466* lisp/cus-start.el (exec-path): Handle nil elements. (Bug#2...55ebb70Catch the imenu-unavailable error in sh-mode completion table993acb5; Minor fix for last change in characters.el30c4bb5More char-width fixes4eb4463Fix char-width-table values for some Emoji528997dKeep point when switching from and to *terminal* buffer2130005* INSTALL: Use correct Emacs release number 25.10835b1Avoid crashes due to objects read with the #n=object form4de671dImprove doc string of 'completion-at-point-functions'ceb46f0Fix crash in evaluating functionsd8374c4* src/filelock.c (current_lock_owner): Update comment.
309 lines
10 KiB
Plaintext
309 lines
10 KiB
Plaintext
-*-mode: text; coding: utf-8;-*-
|
||
|
||
Copyright (C) 2002-2016 Free Software Foundation, Inc.
|
||
See the end of the file for license conditions.
|
||
|
||
Importing a new Unicode Standard version into Emacs
|
||
-------------------------------------------------------------
|
||
|
||
Emacs uses the following files from the Unicode Character Database
|
||
(a.k.a. "UCD):
|
||
|
||
. UnicodeData.txt
|
||
. Blocks.txt
|
||
. BidiMirroring.txt
|
||
. BidiBrackets.txt
|
||
. IVD_Sequences.txt
|
||
. NormalizationTest.txt
|
||
. BidiCharacterTest.txt
|
||
|
||
First, the first 6 files need to be copied into admin/unidata/, and
|
||
then Emacs should be rebuilt for them to take effect. Rebuilding
|
||
Emacs updates several derived files elsewhere in the Emacs source
|
||
tree, mainly in lisp/international/.
|
||
|
||
When Emacs is rebuilt for the first time after importing the new
|
||
files, pay attention to any warning or error messages. In particular,
|
||
admin/unidata/unidata-gen.el will complain if UnicodeData.txt defines
|
||
new bidirectional attributes of characters, because unidata-gen.el,
|
||
bidi.c and dispextern.h need to be updated in that case; failure to do
|
||
so will cause aborts in redisplay.
|
||
|
||
Next, review the changes in UnicodeData.txt vs the previous version
|
||
used by Emacs. Any changes, be it introduction of new scripts or
|
||
addition of codepoints to existing scripts, might need corresponding
|
||
changes in the data used for filling the category-table, case-table,
|
||
and char-width-table. The additional scripts should cause automatic
|
||
updates in charscript.el, but it is a good idea to look at the results
|
||
and see if any changes in admin/unidata/blocks.awk are required.
|
||
|
||
The setting of char-width-table around line 1200 of characters.el
|
||
should be checked against the latest version of the Unicode file
|
||
EastAsianWidth.txt, and any discrepancies fixed.
|
||
|
||
Any new scripts added by UnicodeData.txt will also need updates to
|
||
script-representative-chars defined in fontset.el, and also the list
|
||
of OTF script tags in otf-script-alist, whose source is on this page:
|
||
|
||
https://www.microsoft.com/typography/otspec/scripttags.htm
|
||
|
||
Other databases in fontset.el might also need to be updated as needed.
|
||
|
||
The function 'ucs-names', defined in lisp/international/mule-cmds.el,
|
||
might need to be updated because it knows about used and unused ranges
|
||
of Unicode codepoints, which a new release of the Unicode Standard
|
||
could change.
|
||
|
||
Next, test normalization functions against NormalizationTests.txt,
|
||
in the test/ directory run:
|
||
|
||
make lisp/international/ucs-normalize-tests
|
||
|
||
See commentary in test/lisp/international/ucs-normalize-tests.el
|
||
regarding failing lines.
|
||
|
||
The file BidiCharacterTest.txt should be copied to the test suite, and
|
||
if its format has changed, the file biditest.el there should be
|
||
modified to follow suit.
|
||
|
||
Problems, fixmes and other unicode-related issues
|
||
-------------------------------------------------------------
|
||
|
||
Notes by fx to record various things of variable importance. Handa
|
||
needs to check them -- don't take too seriously, especially with
|
||
regard to completeness.
|
||
|
||
* SINGLE_BYTE_CHAR_P returns true for Latin-1 characters, which has
|
||
undesirable effects. E.g.:
|
||
(multibyte-string-p (let ((s "x")) (aset s 0 ?£) s)) => nil
|
||
(multibyte-string-p (concat [?£])) => nil
|
||
(text-char-description ?£) => "M-#"
|
||
|
||
These examples are all fixed by the change of 2002-10-14, but
|
||
there still exist questionable SINGLE_BYTE_CHAR_P in the
|
||
code (keymap.c and print.c).
|
||
|
||
* Rationalize character syntax and its relationship to the Unicode
|
||
database. (Applies mainly to symbol an punctuation syntax.)
|
||
|
||
* Fontset handling and customization needs work. We want to relate
|
||
fonts to scripts, probably based on the Unicode blocks. The
|
||
presence of small-repertoire 10646-encoded fonts in XFree 4 is a
|
||
pain, not currently worked round.
|
||
|
||
With the change on 2002-07-26, multiple fonts can be
|
||
specified in a fontset for a specific range of characters.
|
||
Each range can also be specified by script. Before using
|
||
ISO10646 fonts, Emacs checks their repertories to avoid such
|
||
fonts that don't have a glyph for a specific character.
|
||
|
||
fx has worked on fontset customization, but was stymied by
|
||
basic problems with the way the default face is dealt with
|
||
(and something else, I think). This needs revisiting.
|
||
|
||
* Work is also needed on charset and coding system priorities.
|
||
|
||
* The relevant bits of latin1-disp.el need porting (and probably
|
||
re-naming/updating). See also cyril-util.el.
|
||
|
||
* Quail files need more work now the encoding is largely irrelevant.
|
||
|
||
* What to do with the old coding categories stuff?
|
||
|
||
* The preferred-coding-system property of charsets should probably be
|
||
junked unless it can be made more useful now.
|
||
|
||
* find-multibyte-characters needs looking at.
|
||
|
||
* Implement Korean cp949/UHC, BIG5-HKSCS and any other important missing
|
||
charsets.
|
||
|
||
* Lazy-load tables for unify-charset somehow?
|
||
|
||
Actually, Emacs clears out all charset maps and unify-map just
|
||
before dumping, and they are loaded again on demand by the
|
||
dumped emacs. But, those maps (char tables) generated while
|
||
temacs is running can't be removed from the dumped emacs.
|
||
|
||
* iso-2022 charsets get unified on i/o.
|
||
|
||
With the change on 2003-01-06, decoding routines put the 'charset'
|
||
property onto decoded text, and iso-2022 encoder pay attention
|
||
to it. Thus, for instance, reading and writing by
|
||
iso-2022-7bit preserve the original designation sequences.
|
||
The property name 'preferred-charset' may be better?
|
||
|
||
We may have to utilize this property to decide a font.
|
||
|
||
* Revisit locale processing: look at treating the language and
|
||
charset parts separately. (Language should affect things like
|
||
spelling and calendar, but that's not a Unicode issue.)
|
||
|
||
* Handle Unicode combining characters usefully, e.g. diacritics, and
|
||
handle more scripts specifically (à la Devanagari). There are
|
||
issues with canonicalization.
|
||
|
||
* We need tabular input methods, e.g. for maths symbols. (Not
|
||
specific to Unicode.)
|
||
|
||
* Need multibyte text in menus, e.g. for the above. (Not specific to
|
||
Unicode -- see Emacs etc/TODO, but now mostly works with gtk.)
|
||
|
||
* Populate char-width-table correctly for Unicode characters and
|
||
worry about what happens when double-width charsets covering
|
||
non-CJK characters are unified.
|
||
|
||
* There are type errors lurking, e.g. in
|
||
Fcheck_coding_systems_region. Define ENABLE_CHECKING to find them.
|
||
|
||
* Old auto-save files, and similar files, such as Gnus drafts,
|
||
containing non-ASCII characters probably won't be re-read correctly.
|
||
|
||
|
||
Source file encoding
|
||
--------------------
|
||
|
||
Most Emacs source files are encoded in UTF-8 (or in ASCII, which is a
|
||
subset), but there are a few exceptions, listed below. Perhaps
|
||
someday many of these files will be converted to UTF-8, for
|
||
convenience when using tools like 'grep -r', but this might need
|
||
nontrivial changes to the build process.
|
||
|
||
* chinese-big5
|
||
|
||
These are verbatim copies of files taken from external sources.
|
||
They haven't been converted to UTF-8.
|
||
|
||
leim/CXTERM-DIC/4Corner.tit
|
||
leim/CXTERM-DIC/ARRAY30.tit
|
||
leim/CXTERM-DIC/ECDICT.tit
|
||
leim/CXTERM-DIC/ETZY.tit
|
||
leim/CXTERM-DIC/PY-b5.tit
|
||
leim/CXTERM-DIC/Punct-b5.tit
|
||
leim/CXTERM-DIC/QJ-b5.tit
|
||
leim/CXTERM-DIC/ZOZY.tit
|
||
leim/MISC-DIC/CTLau-b5.html
|
||
leim/MISC-DIC/cangjie-table.b5
|
||
|
||
* chinese-iso-8bit
|
||
|
||
These are verbatim copies of files taken from external sources.
|
||
They haven't been converted to UTF-8.
|
||
|
||
leim/CXTERM-DIC/CCDOSPY.tit
|
||
leim/CXTERM-DIC/Punct.tit
|
||
leim/CXTERM-DIC/QJ.tit
|
||
leim/CXTERM-DIC/SW.tit
|
||
leim/CXTERM-DIC/TONEPY.tit
|
||
leim/MISC-DIC/CTLau.html
|
||
leim/MISC-DIC/pinyin.map
|
||
leim/MISC-DIC/ziranma.cin
|
||
|
||
* cp850
|
||
|
||
This file contains non-ASCII characters in unibyte strings. When
|
||
editing a keyboard layout it's more convenient to see 'é' than
|
||
'\202', and the MS-DOS compiler requires the single byte if a
|
||
backslash escape is not being used.
|
||
|
||
src/msdos.c
|
||
|
||
* iso-2022-cn-ext
|
||
|
||
This file is externally generated from leim/MISC-DIC/cangjie-table.b5
|
||
by Big5->CNS converter. It hasn't been converted to UTF-8.
|
||
|
||
leim/MISC-DIC/cangjie-table.cns
|
||
|
||
* japanese-iso-8bit
|
||
|
||
SKK-JISYO.L is a verbatim copy of a file taken from an external source.
|
||
It hasn't been converted to UTF-8.
|
||
|
||
leim/SKK-DIC/SKK-JISYO.L
|
||
|
||
* japanese-shift-jis
|
||
|
||
This is a verbatim copy of a file taken from an external source.
|
||
It hasn't been converted to UTF-8.
|
||
|
||
admin/charsets/mapfiles/cns2ucsdkw.txt
|
||
|
||
* iso-2022-7bit
|
||
|
||
This file switches between CJK charsets, which is not encoded in UTF-8.
|
||
|
||
etc/HELLO
|
||
|
||
Each of these files contains just one CJK charset, but Emacs
|
||
currently has no easy way to specify set-charset-priority on a
|
||
per-file basis, so converting any of these files to UTF-8 might
|
||
change the file's appearance when viewed by an Emacs that is
|
||
operating in some other language environment.
|
||
|
||
etc/tutorials/TUTORIAL.ja
|
||
lisp/international/ja-dic-cnv.el
|
||
lisp/international/ja-dic-utl.el
|
||
lisp/international/kinsoku.el
|
||
lisp/international/kkc.el
|
||
lisp/international/titdic-cnv.el
|
||
lisp/language/japan-util.el
|
||
lisp/language/japanese.el
|
||
lisp/leim/quail/cyril-jis.el
|
||
lisp/leim/quail/hanja-jis.el
|
||
lisp/leim/quail/japanese.el
|
||
lisp/leim/quail/py-punct.el
|
||
lisp/leim/quail/pypunct-b5.el
|
||
|
||
This file contains just Chinese characters, and has same problem.
|
||
Also, it contains characters that cannot be encoded in UTF-8.
|
||
|
||
lisp/international/titdic-cnv.el
|
||
|
||
* utf-8-emacs
|
||
|
||
These files contain characters that cannot be encoded in UTF-8.
|
||
|
||
lisp/language/ethio-util.el
|
||
lisp/language/ethiopic.el
|
||
lisp/language/ind-util.el
|
||
lisp/language/tibet-util.el
|
||
lisp/language/tibetan.el
|
||
lisp/leim/quail/ethiopic.el
|
||
lisp/leim/quail/tibetan.el
|
||
|
||
* binary files
|
||
|
||
These files contain binary data, and are not text files.
|
||
Some of the entries in this list are patterns, and stand for any
|
||
files with the listed extension.
|
||
|
||
*.gz
|
||
*.icns
|
||
*.ico
|
||
*.pbm
|
||
*.pdf
|
||
*.png
|
||
*.sig
|
||
etc/e/eterm-color
|
||
etc/package-keyring.gpg
|
||
msdos/emacs.pif
|
||
nextstep/GNUstep/Emacs.base/Resources/emacs.tiff
|
||
nt/icons/hand.cur
|
||
|
||
|
||
This file is part of GNU Emacs.
|
||
|
||
GNU Emacs is free software: you can redistribute it and/or modify
|
||
it under the terms of the GNU General Public License as published by
|
||
the Free Software Foundation, either version 3 of the License, or
|
||
(at your option) any later version.
|
||
|
||
GNU Emacs is distributed in the hope that it will be useful,
|
||
but WITHOUT ANY WARRANTY; without even the implied warranty of
|
||
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
|
||
GNU General Public License for more details.
|
||
|
||
You should have received a copy of the GNU General Public License
|
||
along with GNU Emacs. If not, see <http://www.gnu.org/licenses/>.
|