mirror of
https://git.savannah.gnu.org/git/emacs.git
synced 2024-12-29 11:02:01 +00:00
1455 lines
62 KiB
Plaintext
1455 lines
62 KiB
Plaintext
@c -*-texinfo-*-
|
|
@c This is part of the GNU Emacs Lisp Reference Manual.
|
|
@c Copyright (C) 1998, 1999 Free Software Foundation, Inc.
|
|
@c See the file elisp.texi for copying conditions.
|
|
@setfilename ../info/characters
|
|
@node Non-ASCII Characters, Searching and Matching, Text, Top
|
|
@chapter Non-@acronym{ASCII} Characters
|
|
@cindex multibyte characters
|
|
@cindex non-@acronym{ASCII} characters
|
|
|
|
This chapter covers the special issues relating to non-@acronym{ASCII}
|
|
characters and how they are stored in strings and buffers.
|
|
|
|
@menu
|
|
* Text Representations:: Unibyte and multibyte representations
|
|
* Converting Representations:: Converting unibyte to multibyte and vice versa.
|
|
* Selecting a Representation:: Treating a byte sequence as unibyte or multi.
|
|
* Character Codes:: How unibyte and multibyte relate to
|
|
codes of individual characters.
|
|
* Character Sets:: The space of possible character codes
|
|
is divided into various character sets.
|
|
* Chars and Bytes:: More information about multibyte encodings.
|
|
* Splitting Characters:: Converting a character to its byte sequence.
|
|
* Scanning Charsets:: Which character sets are used in a buffer?
|
|
* Translation of Characters:: Translation tables are used for conversion.
|
|
* Coding Systems:: Coding systems are conversions for saving files.
|
|
* Input Methods:: Input methods allow users to enter various
|
|
non-ASCII characters without special keyboards.
|
|
* Locales:: Interacting with the POSIX locale.
|
|
@end menu
|
|
|
|
@node Text Representations
|
|
@section Text Representations
|
|
@cindex text representations
|
|
|
|
Emacs has two @dfn{text representations}---two ways to represent text
|
|
in a string or buffer. These are called @dfn{unibyte} and
|
|
@dfn{multibyte}. Each string, and each buffer, uses one of these two
|
|
representations. For most purposes, you can ignore the issue of
|
|
representations, because Emacs converts text between them as
|
|
appropriate. Occasionally in Lisp programming you will need to pay
|
|
attention to the difference.
|
|
|
|
@cindex unibyte text
|
|
In unibyte representation, each character occupies one byte and
|
|
therefore the possible character codes range from 0 to 255. Codes 0
|
|
through 127 are @acronym{ASCII} characters; the codes from 128 through 255
|
|
are used for one non-@acronym{ASCII} character set (you can choose which
|
|
character set by setting the variable @code{nonascii-insert-offset}).
|
|
|
|
@cindex leading code
|
|
@cindex multibyte text
|
|
@cindex trailing codes
|
|
In multibyte representation, a character may occupy more than one
|
|
byte, and as a result, the full range of Emacs character codes can be
|
|
stored. The first byte of a multibyte character is always in the range
|
|
128 through 159 (octal 0200 through 0237). These values are called
|
|
@dfn{leading codes}. The second and subsequent bytes of a multibyte
|
|
character are always in the range 160 through 255 (octal 0240 through
|
|
0377); these values are @dfn{trailing codes}.
|
|
|
|
Some sequences of bytes are not valid in multibyte text: for example,
|
|
a single isolated byte in the range 128 through 159 is not allowed. But
|
|
character codes 128 through 159 can appear in multibyte text,
|
|
represented as two-byte sequences. All the character codes 128 through
|
|
255 are possible (though slightly abnormal) in multibyte text; they
|
|
appear in multibyte buffers and strings when you do explicit encoding
|
|
and decoding (@pxref{Explicit Encoding}).
|
|
|
|
In a buffer, the buffer-local value of the variable
|
|
@code{enable-multibyte-characters} specifies the representation used.
|
|
The representation for a string is determined and recorded in the string
|
|
when the string is constructed.
|
|
|
|
@defvar enable-multibyte-characters
|
|
This variable specifies the current buffer's text representation.
|
|
If it is non-@code{nil}, the buffer contains multibyte text; otherwise,
|
|
it contains unibyte text.
|
|
|
|
You cannot set this variable directly; instead, use the function
|
|
@code{set-buffer-multibyte} to change a buffer's representation.
|
|
@end defvar
|
|
|
|
@defvar default-enable-multibyte-characters
|
|
This variable's value is entirely equivalent to @code{(default-value
|
|
'enable-multibyte-characters)}, and setting this variable changes that
|
|
default value. Setting the local binding of
|
|
@code{enable-multibyte-characters} in a specific buffer is not allowed,
|
|
but changing the default value is supported, and it is a reasonable
|
|
thing to do, because it has no effect on existing buffers.
|
|
|
|
The @samp{--unibyte} command line option does its job by setting the
|
|
default value to @code{nil} early in startup.
|
|
@end defvar
|
|
|
|
@defun position-bytes position
|
|
@tindex position-bytes
|
|
Return the byte-position corresponding to buffer position
|
|
@var{position} in the current buffer. This is 1 at the start of the
|
|
buffer, and counts upward in bytes. If @var{position} is out of
|
|
range, the value is @code{nil}.
|
|
@end defun
|
|
|
|
@defun byte-to-position byte-position
|
|
@tindex byte-to-position
|
|
Return the buffer position corresponding to byte-position
|
|
@var{byte-position} in the current buffer. If @var{byte-position} is
|
|
out of range, the value is @code{nil}.
|
|
@end defun
|
|
|
|
@defun multibyte-string-p string
|
|
Return @code{t} if @var{string} is a multibyte string.
|
|
@end defun
|
|
|
|
@node Converting Representations
|
|
@section Converting Text Representations
|
|
|
|
Emacs can convert unibyte text to multibyte; it can also convert
|
|
multibyte text to unibyte, though this conversion loses information. In
|
|
general these conversions happen when inserting text into a buffer, or
|
|
when putting text from several strings together in one string. You can
|
|
also explicitly convert a string's contents to either representation.
|
|
|
|
Emacs chooses the representation for a string based on the text that
|
|
it is constructed from. The general rule is to convert unibyte text to
|
|
multibyte text when combining it with other multibyte text, because the
|
|
multibyte representation is more general and can hold whatever
|
|
characters the unibyte text has.
|
|
|
|
When inserting text into a buffer, Emacs converts the text to the
|
|
buffer's representation, as specified by
|
|
@code{enable-multibyte-characters} in that buffer. In particular, when
|
|
you insert multibyte text into a unibyte buffer, Emacs converts the text
|
|
to unibyte, even though this conversion cannot in general preserve all
|
|
the characters that might be in the multibyte text. The other natural
|
|
alternative, to convert the buffer contents to multibyte, is not
|
|
acceptable because the buffer's representation is a choice made by the
|
|
user that cannot be overridden automatically.
|
|
|
|
Converting unibyte text to multibyte text leaves @acronym{ASCII} characters
|
|
unchanged, and likewise character codes 128 through 159. It converts
|
|
the non-@acronym{ASCII} codes 160 through 255 by adding the value
|
|
@code{nonascii-insert-offset} to each character code. By setting this
|
|
variable, you specify which character set the unibyte characters
|
|
correspond to (@pxref{Character Sets}). For example, if
|
|
@code{nonascii-insert-offset} is 2048, which is @code{(- (make-char
|
|
'latin-iso8859-1) 128)}, then the unibyte non-@acronym{ASCII} characters
|
|
correspond to Latin 1. If it is 2688, which is @code{(- (make-char
|
|
'greek-iso8859-7) 128)}, then they correspond to Greek letters.
|
|
|
|
Converting multibyte text to unibyte is simpler: it discards all but
|
|
the low 8 bits of each character code. If @code{nonascii-insert-offset}
|
|
has a reasonable value, corresponding to the beginning of some character
|
|
set, this conversion is the inverse of the other: converting unibyte
|
|
text to multibyte and back to unibyte reproduces the original unibyte
|
|
text.
|
|
|
|
@defvar nonascii-insert-offset
|
|
This variable specifies the amount to add to a non-@acronym{ASCII} character
|
|
when converting unibyte text to multibyte. It also applies when
|
|
@code{self-insert-command} inserts a character in the unibyte
|
|
non-@acronym{ASCII} range, 128 through 255. However, the functions
|
|
@code{insert} and @code{insert-char} do not perform this conversion.
|
|
|
|
The right value to use to select character set @var{cs} is @code{(-
|
|
(make-char @var{cs}) 128)}. If the value of
|
|
@code{nonascii-insert-offset} is zero, then conversion actually uses the
|
|
value for the Latin 1 character set, rather than zero.
|
|
@end defvar
|
|
|
|
@defvar nonascii-translation-table
|
|
This variable provides a more general alternative to
|
|
@code{nonascii-insert-offset}. You can use it to specify independently
|
|
how to translate each code in the range of 128 through 255 into a
|
|
multibyte character. The value should be a char-table, or @code{nil}.
|
|
If this is non-@code{nil}, it overrides @code{nonascii-insert-offset}.
|
|
@end defvar
|
|
|
|
The next three functions either return the argument @var{string}, or a
|
|
newly created string with no text properties.
|
|
|
|
@defun string-make-unibyte string
|
|
This function converts the text of @var{string} to unibyte
|
|
representation, if it isn't already, and returns the result. If
|
|
@var{string} is a unibyte string, it is returned unchanged. Multibyte
|
|
character codes are converted to unibyte according to
|
|
@code{nonascii-translation-table} or, if that is @code{nil}, using
|
|
@code{nonascii-insert-offset}. If the lookup in the translation table
|
|
fails, this function takes just the low 8 bits of each character.
|
|
@end defun
|
|
|
|
@defun string-make-multibyte string
|
|
This function converts the text of @var{string} to multibyte
|
|
representation, if it isn't already, and returns the result. If
|
|
@var{string} is a multibyte string or consists entirely of
|
|
@acronym{ASCII} characters, it is returned unchanged. In particular,
|
|
if @var{string} is unibyte and entirely @acronym{ASCII}, the returned
|
|
string is unibyte. (When the characters are all @acronym{ASCII},
|
|
Emacs primitives will treat the string the same way whether it is
|
|
unibyte or multibyte.) If @var{string} is unibyte and contains
|
|
non-@acronym{ASCII} characters, the function
|
|
@code{unibyte-char-to-multibyte} is used to convert each unibyte
|
|
character to a multibyte character.
|
|
@end defun
|
|
|
|
@defun string-to-multibyte string
|
|
This function returns a multibyte string containing the same sequence
|
|
of character codes as @var{string}. Unlike
|
|
@code{string-make-multibyte}, this function unconditionally returns a
|
|
multibyte string. If @var{string} is a multibyte string, it is
|
|
returned unchanged.
|
|
@end defun
|
|
|
|
@defun multibyte-char-to-unibyte char
|
|
This convert the multibyte character @var{char} to a unibyte
|
|
character, based on @code{nonascii-translation-table} and
|
|
@code{nonascii-insert-offset}.
|
|
@end defun
|
|
|
|
@defun unibyte-char-to-multibyte char
|
|
This convert the unibyte character @var{char} to a multibyte
|
|
character, based on @code{nonascii-translation-table} and
|
|
@code{nonascii-insert-offset}.
|
|
@end defun
|
|
|
|
@node Selecting a Representation
|
|
@section Selecting a Representation
|
|
|
|
Sometimes it is useful to examine an existing buffer or string as
|
|
multibyte when it was unibyte, or vice versa.
|
|
|
|
@defun set-buffer-multibyte multibyte
|
|
Set the representation type of the current buffer. If @var{multibyte}
|
|
is non-@code{nil}, the buffer becomes multibyte. If @var{multibyte}
|
|
is @code{nil}, the buffer becomes unibyte.
|
|
|
|
This function leaves the buffer contents unchanged when viewed as a
|
|
sequence of bytes. As a consequence, it can change the contents viewed
|
|
as characters; a sequence of two bytes which is treated as one character
|
|
in multibyte representation will count as two characters in unibyte
|
|
representation. Character codes 128 through 159 are an exception. They
|
|
are represented by one byte in a unibyte buffer, but when the buffer is
|
|
set to multibyte, they are converted to two-byte sequences, and vice
|
|
versa.
|
|
|
|
This function sets @code{enable-multibyte-characters} to record which
|
|
representation is in use. It also adjusts various data in the buffer
|
|
(including overlays, text properties and markers) so that they cover the
|
|
same text as they did before.
|
|
|
|
You cannot use @code{set-buffer-multibyte} on an indirect buffer,
|
|
because indirect buffers always inherit the representation of the
|
|
base buffer.
|
|
@end defun
|
|
|
|
@defun string-as-unibyte string
|
|
This function returns a string with the same bytes as @var{string} but
|
|
treating each byte as a character. This means that the value may have
|
|
more characters than @var{string} has.
|
|
|
|
If @var{string} is already a unibyte string, then the value is
|
|
@var{string} itself. Otherwise it is a newly created string, with no
|
|
text properties. If @var{string} is multibyte, any characters it
|
|
contains of charset @code{eight-bit-control} or @code{eight-bit-graphic}
|
|
are converted to the corresponding single byte.
|
|
@end defun
|
|
|
|
@defun string-as-multibyte string
|
|
This function returns a string with the same bytes as @var{string} but
|
|
treating each multibyte sequence as one character. This means that the
|
|
value may have fewer characters than @var{string} has.
|
|
|
|
If @var{string} is already a multibyte string, then the value is
|
|
@var{string} itself. Otherwise it is a newly created string, with no
|
|
text properties. If @var{string} is unibyte and contains any individual
|
|
8-bit bytes (i.e.@: not part of a multibyte form), they are converted to
|
|
the corresponding multibyte character of charset @code{eight-bit-control}
|
|
or @code{eight-bit-graphic}.
|
|
@end defun
|
|
|
|
@node Character Codes
|
|
@section Character Codes
|
|
@cindex character codes
|
|
|
|
The unibyte and multibyte text representations use different character
|
|
codes. The valid character codes for unibyte representation range from
|
|
0 to 255---the values that can fit in one byte. The valid character
|
|
codes for multibyte representation range from 0 to 524287, but not all
|
|
values in that range are valid. The values 128 through 255 are not
|
|
entirely proper in multibyte text, but they can occur if you do explicit
|
|
encoding and decoding (@pxref{Explicit Encoding}). Some other character
|
|
codes cannot occur at all in multibyte text. Only the @acronym{ASCII} codes
|
|
0 through 127 are completely legitimate in both representations.
|
|
|
|
@defun char-valid-p charcode &optional genericp
|
|
This returns @code{t} if @var{charcode} is valid (either for unibyte
|
|
text or for multibyte text).
|
|
|
|
@example
|
|
(char-valid-p 65)
|
|
@result{} t
|
|
(char-valid-p 256)
|
|
@result{} nil
|
|
(char-valid-p 2248)
|
|
@result{} t
|
|
@end example
|
|
|
|
If the optional argument @var{genericp} is non-@code{nil}, this
|
|
function also returns @code{t} if @var{charcode} is a generic
|
|
character (@pxref{Splitting Characters}).
|
|
@end defun
|
|
|
|
@node Character Sets
|
|
@section Character Sets
|
|
@cindex character sets
|
|
|
|
Emacs classifies characters into various @dfn{character sets}, each of
|
|
which has a name which is a symbol. Each character belongs to one and
|
|
only one character set.
|
|
|
|
In general, there is one character set for each distinct script. For
|
|
example, @code{latin-iso8859-1} is one character set,
|
|
@code{greek-iso8859-7} is another, and @code{ascii} is another. An
|
|
Emacs character set can hold at most 9025 characters; therefore, in some
|
|
cases, characters that would logically be grouped together are split
|
|
into several character sets. For example, one set of Chinese
|
|
characters, generally known as Big 5, is divided into two Emacs
|
|
character sets, @code{chinese-big5-1} and @code{chinese-big5-2}.
|
|
|
|
@acronym{ASCII} characters are in character set @code{ascii}. The
|
|
non-@acronym{ASCII} characters 128 through 159 are in character set
|
|
@code{eight-bit-control}, and codes 160 through 255 are in character set
|
|
@code{eight-bit-graphic}.
|
|
|
|
@defun charsetp object
|
|
Returns @code{t} if @var{object} is a symbol that names a character set,
|
|
@code{nil} otherwise.
|
|
@end defun
|
|
|
|
@defvar charset-list
|
|
The value is a list of all defined character set names.
|
|
@end defvar
|
|
|
|
@defun charset-list
|
|
This function returns the value of @code{charset-list}. It is only
|
|
provided for backward compatibility.
|
|
@end defun
|
|
|
|
@defun char-charset character
|
|
This function returns the name of the character set that @var{character}
|
|
belongs to, or the symbol @code{unknown} if @var{character} is not a
|
|
valid character.
|
|
@end defun
|
|
|
|
@defun charset-plist charset
|
|
@tindex charset-plist
|
|
This function returns the charset property list of the character set
|
|
@var{charset}. Although @var{charset} is a symbol, this is not the same
|
|
as the property list of that symbol. Charset properties are used for
|
|
special purposes within Emacs.
|
|
@end defun
|
|
|
|
@deffn Command list-charset-chars charset
|
|
This command displays a list of characters in the character set
|
|
@var{charset}.
|
|
@end deffn
|
|
|
|
@node Chars and Bytes
|
|
@section Characters and Bytes
|
|
@cindex bytes and characters
|
|
|
|
@cindex introduction sequence
|
|
@cindex dimension (of character set)
|
|
In multibyte representation, each character occupies one or more
|
|
bytes. Each character set has an @dfn{introduction sequence}, which is
|
|
normally one or two bytes long. (Exception: the @code{ascii} character
|
|
set and the @code{eight-bit-graphic} character set have a zero-length
|
|
introduction sequence.) The introduction sequence is the beginning of
|
|
the byte sequence for any character in the character set. The rest of
|
|
the character's bytes distinguish it from the other characters in the
|
|
same character set. Depending on the character set, there are either
|
|
one or two distinguishing bytes; the number of such bytes is called the
|
|
@dfn{dimension} of the character set.
|
|
|
|
@defun charset-dimension charset
|
|
This function returns the dimension of @var{charset}; at present, the
|
|
dimension is always 1 or 2.
|
|
@end defun
|
|
|
|
@defun charset-bytes charset
|
|
@tindex charset-bytes
|
|
This function returns the number of bytes used to represent a character
|
|
in character set @var{charset}.
|
|
@end defun
|
|
|
|
This is the simplest way to determine the byte length of a character
|
|
set's introduction sequence:
|
|
|
|
@example
|
|
(- (charset-bytes @var{charset})
|
|
(charset-dimension @var{charset}))
|
|
@end example
|
|
|
|
@node Splitting Characters
|
|
@section Splitting Characters
|
|
|
|
The functions in this section convert between characters and the byte
|
|
values used to represent them. For most purposes, there is no need to
|
|
be concerned with the sequence of bytes used to represent a character,
|
|
because Emacs translates automatically when necessary.
|
|
|
|
@defun split-char character
|
|
Return a list containing the name of the character set of
|
|
@var{character}, followed by one or two byte values (integers) which
|
|
identify @var{character} within that character set. The number of byte
|
|
values is the character set's dimension.
|
|
|
|
If @var{character} is invalid as a character code, @code{split-char}
|
|
returns a list consisting of the symbol @code{unknown} and @var{character}.
|
|
|
|
@example
|
|
(split-char 2248)
|
|
@result{} (latin-iso8859-1 72)
|
|
(split-char 65)
|
|
@result{} (ascii 65)
|
|
(split-char 128)
|
|
@result{} (eight-bit-control 128)
|
|
@end example
|
|
@end defun
|
|
|
|
@defun make-char charset &optional code1 code2
|
|
This function returns the character in character set @var{charset} whose
|
|
position codes are @var{code1} and @var{code2}. This is roughly the
|
|
inverse of @code{split-char}. Normally, you should specify either one
|
|
or both of @var{code1} and @var{code2} according to the dimension of
|
|
@var{charset}. For example,
|
|
|
|
@example
|
|
(make-char 'latin-iso8859-1 72)
|
|
@result{} 2248
|
|
@end example
|
|
|
|
Actually, the eighth bit of both @var{code1} and @var{code2} is zeroed
|
|
before they are used to index @var{charset}. Thus you may use, for
|
|
instance, an ISO 8859 character code rather than subtracting 128, as
|
|
is necessary to index the corresponding Emacs charset.
|
|
@end defun
|
|
|
|
@cindex generic characters
|
|
If you call @code{make-char} with no @var{byte-values}, the result is
|
|
a @dfn{generic character} which stands for @var{charset}. A generic
|
|
character is an integer, but it is @emph{not} valid for insertion in the
|
|
buffer as a character. It can be used in @code{char-table-range} to
|
|
refer to the whole character set (@pxref{Char-Tables}).
|
|
@code{char-valid-p} returns @code{nil} for generic characters.
|
|
For example:
|
|
|
|
@example
|
|
(make-char 'latin-iso8859-1)
|
|
@result{} 2176
|
|
(char-valid-p 2176)
|
|
@result{} nil
|
|
(char-valid-p 2176 t)
|
|
@result{} t
|
|
(split-char 2176)
|
|
@result{} (latin-iso8859-1 0)
|
|
@end example
|
|
|
|
The character sets @code{ascii}, @code{eight-bit-control}, and
|
|
@code{eight-bit-graphic} don't have corresponding generic characters. If
|
|
@var{charset} is one of them and you don't supply @var{code1},
|
|
@code{make-char} returns the character code corresponding to the
|
|
smallest code in @var{charset}.
|
|
|
|
@node Scanning Charsets
|
|
@section Scanning for Character Sets
|
|
|
|
Sometimes it is useful to find out which character sets appear in a
|
|
part of a buffer or a string. One use for this is in determining which
|
|
coding systems (@pxref{Coding Systems}) are capable of representing all
|
|
of the text in question.
|
|
|
|
@defun charset-after &optional pos
|
|
This function return the charset of a character in the current buffer
|
|
at position @var{pos}. If @var{pos} is omitted or @code{nil}, it
|
|
defauls to the current value of point. If @var{pos} is out of range,
|
|
the value is @code{nil}.
|
|
@end defun
|
|
|
|
@defun find-charset-region beg end &optional translation
|
|
This function returns a list of the character sets that appear in the
|
|
current buffer between positions @var{beg} and @var{end}.
|
|
|
|
The optional argument @var{translation} specifies a translation table to
|
|
be used in scanning the text (@pxref{Translation of Characters}). If it
|
|
is non-@code{nil}, then each character in the region is translated
|
|
through this table, and the value returned describes the translated
|
|
characters instead of the characters actually in the buffer.
|
|
@end defun
|
|
|
|
@defun find-charset-string string &optional translation
|
|
This function returns a list of the character sets that appear in the
|
|
string @var{string}. It is just like @code{find-charset-region}, except
|
|
that it applies to the contents of @var{string} instead of part of the
|
|
current buffer.
|
|
@end defun
|
|
|
|
@node Translation of Characters
|
|
@section Translation of Characters
|
|
@cindex character translation tables
|
|
@cindex translation tables
|
|
|
|
A @dfn{translation table} is a char-table that specifies a mapping
|
|
of characters into characters. These tables are used in encoding and
|
|
decoding, and for other purposes. Some coding systems specify their
|
|
own particular translation tables; there are also default translation
|
|
tables which apply to all other coding systems.
|
|
|
|
For instance, the coding-system @code{utf-8} has a translation table
|
|
that maps characters of various charsets (e.g.,
|
|
@code{latin-iso8859-@var{x}}) into Unicode character sets. This way,
|
|
it can encode Latin-2 characters into UTF-8. Meanwhile,
|
|
@code{unify-8859-on-decoding-mode} operates by specifying
|
|
@code{standard-translation-table-for-decode} to translate
|
|
Latin-@var{x} characters into corresponding Unicode characters.
|
|
|
|
@defun make-translation-table &rest translations
|
|
This function returns a translation table based on the argument
|
|
@var{translations}. Each element of @var{translations} should be a
|
|
list of elements of the form @code{(@var{from} . @var{to})}; this says
|
|
to translate the character @var{from} into @var{to}.
|
|
|
|
The arguments and the forms in each argument are processed in order,
|
|
and if a previous form already translates @var{to} to some other
|
|
character, say @var{to-alt}, @var{from} is also translated to
|
|
@var{to-alt}.
|
|
|
|
You can also map one whole character set into another character set with
|
|
the same dimension. To do this, you specify a generic character (which
|
|
designates a character set) for @var{from} (@pxref{Splitting Characters}).
|
|
In this case, if @var{to} is also a generic character, its character
|
|
set should have the same dimension as @var{from}'s. Then the
|
|
translation table translates each character of @var{from}'s character
|
|
set into the corresponding character of @var{to}'s character set. If
|
|
@var{from} is a generic character and @var{to} is an ordinary
|
|
character, then the translation table translates every character of
|
|
@var{from}'s character set into @var{to}.
|
|
@end defun
|
|
|
|
In decoding, the translation table's translations are applied to the
|
|
characters that result from ordinary decoding. If a coding system has
|
|
property @code{translation-table-for-decode}, that specifies the
|
|
translation table to use. (This is a property of the coding system,
|
|
as returned by @code{coding-system-get}, not a property of the symbol
|
|
that is the coding system's name. @xref{Coding System Basics,, Basic
|
|
Concepts of Coding Systems}.) Otherwise, if
|
|
@code{standard-translation-table-for-decode} is non-@code{nil},
|
|
decoding uses that table.
|
|
|
|
In encoding, the translation table's translations are applied to the
|
|
characters in the buffer, and the result of translation is actually
|
|
encoded. If a coding system has property
|
|
@code{translation-table-for-encode}, that specifies the translation
|
|
table to use. Otherwise the variable
|
|
@code{standard-translation-table-for-encode} specifies the translation
|
|
table.
|
|
|
|
@defvar standard-translation-table-for-decode
|
|
This is the default translation table for decoding, for
|
|
coding systems that don't specify any other translation table.
|
|
@end defvar
|
|
|
|
@defvar standard-translation-table-for-encode
|
|
This is the default translation table for encoding, for
|
|
coding systems that don't specify any other translation table.
|
|
@end defvar
|
|
|
|
@defvar translation-table-for-input
|
|
Self-inserting characters are translated through this translation
|
|
table before they are inserted. This variable automatically becomes
|
|
buffer-local when set.
|
|
|
|
@code{set-buffer-file-coding-system} sets this variable so that your
|
|
keyboard input gets translated into the character sets that the buffer
|
|
is likely to contain.
|
|
@end defvar
|
|
|
|
@node Coding Systems
|
|
@section Coding Systems
|
|
|
|
@cindex coding system
|
|
When Emacs reads or writes a file, and when Emacs sends text to a
|
|
subprocess or receives text from a subprocess, it normally performs
|
|
character code conversion and end-of-line conversion as specified
|
|
by a particular @dfn{coding system}.
|
|
|
|
How to define a coding system is an arcane matter, and is not
|
|
documented here.
|
|
|
|
@menu
|
|
* Coding System Basics:: Basic concepts.
|
|
* Encoding and I/O:: How file I/O functions handle coding systems.
|
|
* Lisp and Coding Systems:: Functions to operate on coding system names.
|
|
* User-Chosen Coding Systems:: Asking the user to choose a coding system.
|
|
* Default Coding Systems:: Controlling the default choices.
|
|
* Specifying Coding Systems:: Requesting a particular coding system
|
|
for a single file operation.
|
|
* Explicit Encoding:: Encoding or decoding text without doing I/O.
|
|
* Terminal I/O Encoding:: Use of encoding for terminal I/O.
|
|
* MS-DOS File Types:: How DOS "text" and "binary" files
|
|
relate to coding systems.
|
|
@end menu
|
|
|
|
@node Coding System Basics
|
|
@subsection Basic Concepts of Coding Systems
|
|
|
|
@cindex character code conversion
|
|
@dfn{Character code conversion} involves conversion between the encoding
|
|
used inside Emacs and some other encoding. Emacs supports many
|
|
different encodings, in that it can convert to and from them. For
|
|
example, it can convert text to or from encodings such as Latin 1, Latin
|
|
2, Latin 3, Latin 4, Latin 5, and several variants of ISO 2022. In some
|
|
cases, Emacs supports several alternative encodings for the same
|
|
characters; for example, there are three coding systems for the Cyrillic
|
|
(Russian) alphabet: ISO, Alternativnyj, and KOI8.
|
|
|
|
Most coding systems specify a particular character code for
|
|
conversion, but some of them leave the choice unspecified---to be chosen
|
|
heuristically for each file, based on the data.
|
|
|
|
In general, a coding system doesn't guarantee roundtrip identity:
|
|
decoding a byte sequence using coding system, then encoding the
|
|
resulting text in the same coding system, can produce a different byte
|
|
sequence. However, the following coding systems do guarantee that the
|
|
byte sequence will be the same as what you originally decoded:
|
|
|
|
@quotation
|
|
chinese-big5 chinese-iso-8bit cyrillic-iso-8bit emacs-mule
|
|
greek-iso-8bit hebrew-iso-8bit iso-latin-1 iso-latin-2 iso-latin-3
|
|
iso-latin-4 iso-latin-5 iso-latin-8 iso-latin-9 iso-safe
|
|
japanese-iso-8bit japanese-shift-jis korean-iso-8bit raw-text
|
|
@end quotation
|
|
|
|
Encoding buffer text and then decoding the result can also fail to
|
|
reproduce the original text. For instance, if you encode Latin-2
|
|
characters with @code{utf-8} and decode the result using the same
|
|
coding system, you'll get Unicode characters (of charset
|
|
@code{mule-unicode-0100-24ff}). If you encode Unicode characters with
|
|
@code{iso-latin-2} and decode the result with the same coding system,
|
|
you'll get Latin-2 characters.
|
|
|
|
@cindex end of line conversion
|
|
@dfn{End of line conversion} handles three different conventions used
|
|
on various systems for representing end of line in files. The Unix
|
|
convention is to use the linefeed character (also called newline). The
|
|
DOS convention is to use a carriage-return and a linefeed at the end of
|
|
a line. The Mac convention is to use just carriage-return.
|
|
|
|
@cindex base coding system
|
|
@cindex variant coding system
|
|
@dfn{Base coding systems} such as @code{latin-1} leave the end-of-line
|
|
conversion unspecified, to be chosen based on the data. @dfn{Variant
|
|
coding systems} such as @code{latin-1-unix}, @code{latin-1-dos} and
|
|
@code{latin-1-mac} specify the end-of-line conversion explicitly as
|
|
well. Most base coding systems have three corresponding variants whose
|
|
names are formed by adding @samp{-unix}, @samp{-dos} and @samp{-mac}.
|
|
|
|
The coding system @code{raw-text} is special in that it prevents
|
|
character code conversion, and causes the buffer visited with that
|
|
coding system to be a unibyte buffer. It does not specify the
|
|
end-of-line conversion, allowing that to be determined as usual by the
|
|
data, and has the usual three variants which specify the end-of-line
|
|
conversion. @code{no-conversion} is equivalent to @code{raw-text-unix}:
|
|
it specifies no conversion of either character codes or end-of-line.
|
|
|
|
The coding system @code{emacs-mule} specifies that the data is
|
|
represented in the internal Emacs encoding. This is like
|
|
@code{raw-text} in that no code conversion happens, but different in
|
|
that the result is multibyte data.
|
|
|
|
@defun coding-system-get coding-system property
|
|
This function returns the specified property of the coding system
|
|
@var{coding-system}. Most coding system properties exist for internal
|
|
purposes, but one that you might find useful is @code{mime-charset}.
|
|
That property's value is the name used in MIME for the character coding
|
|
which this coding system can read and write. Examples:
|
|
|
|
@example
|
|
(coding-system-get 'iso-latin-1 'mime-charset)
|
|
@result{} iso-8859-1
|
|
(coding-system-get 'iso-2022-cn 'mime-charset)
|
|
@result{} iso-2022-cn
|
|
(coding-system-get 'cyrillic-koi8 'mime-charset)
|
|
@result{} koi8-r
|
|
@end example
|
|
|
|
The value of the @code{mime-charset} property is also defined
|
|
as an alias for the coding system.
|
|
@end defun
|
|
|
|
@node Encoding and I/O
|
|
@subsection Encoding and I/O
|
|
|
|
The principal purpose of coding systems is for use in reading and
|
|
writing files. The function @code{insert-file-contents} uses
|
|
a coding system for decoding the file data, and @code{write-region}
|
|
uses one to encode the buffer contents.
|
|
|
|
You can specify the coding system to use either explicitly
|
|
(@pxref{Specifying Coding Systems}), or implicitly using a default
|
|
mechanism (@pxref{Default Coding Systems}). But these methods may not
|
|
completely specify what to do. For example, they may choose a coding
|
|
system such as @code{undefined} which leaves the character code
|
|
conversion to be determined from the data. In these cases, the I/O
|
|
operation finishes the job of choosing a coding system. Very often
|
|
you will want to find out afterwards which coding system was chosen.
|
|
|
|
@defvar buffer-file-coding-system
|
|
This variable records the coding system that was used for visiting the
|
|
current buffer. It is used for saving the buffer, and for writing part
|
|
of the buffer with @code{write-region}. If the text to be written
|
|
cannot be safely encoded using the coding system specified by this
|
|
variable, these operations select an alternative encoding by calling
|
|
the function @code{select-safe-coding-system} (@pxref{User-Chosen
|
|
Coding Systems}). If selecting a different encoding requires to ask
|
|
the user to specify a coding system, @code{buffer-file-coding-system}
|
|
is updated to the newly selected coding system.
|
|
|
|
@code{buffer-file-coding-system} does @emph{not} affect sending text
|
|
to a subprocess.
|
|
@end defvar
|
|
|
|
@defvar save-buffer-coding-system
|
|
This variable specifies the coding system for saving the buffer (by
|
|
overriding @code{buffer-file-coding-system}). Note that it is not used
|
|
for @code{write-region}.
|
|
|
|
When a command to save the buffer starts out to use
|
|
@code{buffer-file-coding-system} (or @code{save-buffer-coding-system}),
|
|
and that coding system cannot handle
|
|
the actual text in the buffer, the command asks the user to choose
|
|
another coding system (by calling @code{select-safe-coding-system}).
|
|
After that happens, the command also updates
|
|
@code{buffer-file-coding-system} to represent the coding system that
|
|
the user specified.
|
|
@end defvar
|
|
|
|
@defvar last-coding-system-used
|
|
I/O operations for files and subprocesses set this variable to the
|
|
coding system name that was used. The explicit encoding and decoding
|
|
functions (@pxref{Explicit Encoding}) set it too.
|
|
|
|
@strong{Warning:} Since receiving subprocess output sets this variable,
|
|
it can change whenever Emacs waits; therefore, you should copy the
|
|
value shortly after the function call that stores the value you are
|
|
interested in.
|
|
@end defvar
|
|
|
|
The variable @code{selection-coding-system} specifies how to encode
|
|
selections for the window system. @xref{Window System Selections}.
|
|
|
|
@defvar file-name-coding-system
|
|
The variable @code{file-name-coding-system} specifies the coding
|
|
system to use for encoding file names. Emacs encodes file names using
|
|
that coding system for all file operations. If
|
|
@code{file-name-coding-system} is @code{nil}, Emacs uses a default
|
|
coding system determined by the selected language environment. In the
|
|
default language environment, any non-@acronym{ASCII} characters in
|
|
file names are not encoded specially; they appear in the file system
|
|
using the internal Emacs representation.
|
|
@end defvar
|
|
|
|
@strong{Warning:} if you change @code{file-name-coding-system} (or
|
|
the language environment) in the middle of an Emacs session, problems
|
|
can result if you have already visited files whose names were encoded
|
|
using the earlier coding system and are handled differently under the
|
|
new coding system. If you try to save one of these buffers under the
|
|
visited file name, saving may use the wrong file name, or it may get
|
|
an error. If such a problem happens, use @kbd{C-x C-w} to specify a
|
|
new file name for that buffer.
|
|
|
|
@node Lisp and Coding Systems
|
|
@subsection Coding Systems in Lisp
|
|
|
|
Here are the Lisp facilities for working with coding systems:
|
|
|
|
@defun coding-system-list &optional base-only
|
|
This function returns a list of all coding system names (symbols). If
|
|
@var{base-only} is non-@code{nil}, the value includes only the
|
|
base coding systems. Otherwise, it includes alias and variant coding
|
|
systems as well.
|
|
@end defun
|
|
|
|
@defun coding-system-p object
|
|
This function returns @code{t} if @var{object} is a coding system
|
|
name or @code{nil}.
|
|
@end defun
|
|
|
|
@defun check-coding-system coding-system
|
|
This function checks the validity of @var{coding-system}.
|
|
If that is valid, it returns @var{coding-system}.
|
|
Otherwise it signals an error with condition @code{coding-system-error}.
|
|
@end defun
|
|
|
|
@defun coding-system-change-eol-conversion coding-system eol-type
|
|
This function returns a coding system which is like @var{coding-system}
|
|
except for its eol conversion, which is specified by @code{eol-type}.
|
|
@var{eol-type} should be @code{unix}, @code{dos}, @code{mac}, or
|
|
@code{nil}. If it is @code{nil}, the returned coding system determines
|
|
the end-of-line conversion from the data.
|
|
|
|
@var{eol-type} may also be 0, 1 or 2, standing for @code{unix},
|
|
@code{dos} and @code{mac}, respectively.
|
|
@end defun
|
|
|
|
@defun coding-system-change-text-conversion eol-coding text-coding
|
|
This function returns a coding system which uses the end-of-line
|
|
conversion of @var{eol-coding}, and the text conversion of
|
|
@var{text-coding}. If @var{text-coding} is @code{nil}, it returns
|
|
@code{undecided}, or one of its variants according to @var{eol-coding}.
|
|
@end defun
|
|
|
|
@defun find-coding-systems-region from to
|
|
This function returns a list of coding systems that could be used to
|
|
encode a text between @var{from} and @var{to}. All coding systems in
|
|
the list can safely encode any multibyte characters in that portion of
|
|
the text.
|
|
|
|
If the text contains no multibyte characters, the function returns the
|
|
list @code{(undecided)}.
|
|
@end defun
|
|
|
|
@defun find-coding-systems-string string
|
|
This function returns a list of coding systems that could be used to
|
|
encode the text of @var{string}. All coding systems in the list can
|
|
safely encode any multibyte characters in @var{string}. If the text
|
|
contains no multibyte characters, this returns the list
|
|
@code{(undecided)}.
|
|
@end defun
|
|
|
|
@defun find-coding-systems-for-charsets charsets
|
|
This function returns a list of coding systems that could be used to
|
|
encode all the character sets in the list @var{charsets}.
|
|
@end defun
|
|
|
|
@defun detect-coding-region start end &optional highest
|
|
This function chooses a plausible coding system for decoding the text
|
|
from @var{start} to @var{end}. This text should be a byte sequence
|
|
(@pxref{Explicit Encoding}).
|
|
|
|
Normally this function returns a list of coding systems that could
|
|
handle decoding the text that was scanned. They are listed in order of
|
|
decreasing priority. But if @var{highest} is non-@code{nil}, then the
|
|
return value is just one coding system, the one that is highest in
|
|
priority.
|
|
|
|
If the region contains only @acronym{ASCII} characters, the value
|
|
is @code{undecided} or @code{(undecided)}, or a variant specifying
|
|
end-of-line conversion, if that can be deduced from the text.
|
|
@end defun
|
|
|
|
@defun detect-coding-string string &optional highest
|
|
This function is like @code{detect-coding-region} except that it
|
|
operates on the contents of @var{string} instead of bytes in the buffer.
|
|
@end defun
|
|
|
|
@xref{Coding systems for a subprocess,, Process Information}, in
|
|
particular the description of the functions
|
|
@code{process-coding-system} and @code{set-process-coding-system}, for
|
|
how to examine or set the coding systems used for I/O to a subprocess.
|
|
|
|
@node User-Chosen Coding Systems
|
|
@subsection User-Chosen Coding Systems
|
|
|
|
@cindex select safe coding system
|
|
@defun select-safe-coding-system from to &optional default-coding-system accept-default-p file
|
|
This function selects a coding system for encoding specified text,
|
|
asking the user to choose if necessary. Normally the specified text
|
|
is the text in the current buffer between @var{from} and @var{to}. If
|
|
@var{from} is a string, the string specifies the text to encode, and
|
|
@var{to} is ignored.
|
|
|
|
If @var{default-coding-system} is non-@code{nil}, that is the first
|
|
coding system to try; if that can handle the text,
|
|
@code{select-safe-coding-system} returns that coding system. It can
|
|
also be a list of coding systems; then the function tries each of them
|
|
one by one. After trying all of them, it next tries the current
|
|
buffer's value of @code{buffer-file-coding-system} (if it is not
|
|
@code{undecided}), then the value of
|
|
@code{default-buffer-file-coding-system} and finally the user's most
|
|
preferred coding system, which the user can set using the command
|
|
@code{prefer-coding-system} (@pxref{Recognize Coding,, Recognizing
|
|
Coding Systems, emacs, The GNU Emacs Manual}).
|
|
|
|
If one of those coding systems can safely encode all the specified
|
|
text, @code{select-safe-coding-system} chooses it and returns it.
|
|
Otherwise, it asks the user to choose from a list of coding systems
|
|
which can encode all the text, and returns the user's choice.
|
|
|
|
@var{default-coding-system} can also be a list whose first element is
|
|
t and whose other elements are coding systems. Then, if no coding
|
|
system in the list can handle the text, @code{select-safe-coding-system}
|
|
queries the user immediately, without trying any of the three
|
|
alternatives described above.
|
|
|
|
The optional argument @var{accept-default-p}, if non-@code{nil},
|
|
should be a function to determine whether a coding system selected
|
|
without user interaction is acceptable. @code{select-safe-coding-system}
|
|
calls this function with one argument, the base coding system of the
|
|
selected coding system. If @var{accept-default-p} returns @code{nil},
|
|
@code{select-safe-coding-system} rejects the silently selected coding
|
|
system, and asks the user to select a coding system from a list of
|
|
possible candidates.
|
|
|
|
@vindex select-safe-coding-system-accept-default-p
|
|
If the variable @code{select-safe-coding-system-accept-default-p} is
|
|
non-@code{nil}, its value overrides the value of
|
|
@var{accept-default-p}.
|
|
|
|
As a final step, before returning the chosen coding system,
|
|
@code{select-safe-coding-system} checks whether that coding system is
|
|
consistent with what would be selected if the contents of the region
|
|
were read from a file. (If not, this could lead to data corruption in
|
|
a file subsequently re-visited and edited.) Normally,
|
|
@code{select-safe-coding-system} uses @code{buffer-file-name} as the
|
|
file for this purpose, but if @var{file} is non-@code{nil}, it uses
|
|
that file instead (this can be relevant for @code{write-region} and
|
|
similar functions). If it detects an apparent inconsistency,
|
|
@code{select-safe-coding-system} queries the user before selecting the
|
|
coding system.
|
|
@end defun
|
|
|
|
Here are two functions you can use to let the user specify a coding
|
|
system, with completion. @xref{Completion}.
|
|
|
|
@defun read-coding-system prompt &optional default
|
|
This function reads a coding system using the minibuffer, prompting with
|
|
string @var{prompt}, and returns the coding system name as a symbol. If
|
|
the user enters null input, @var{default} specifies which coding system
|
|
to return. It should be a symbol or a string.
|
|
@end defun
|
|
|
|
@defun read-non-nil-coding-system prompt
|
|
This function reads a coding system using the minibuffer, prompting with
|
|
string @var{prompt}, and returns the coding system name as a symbol. If
|
|
the user tries to enter null input, it asks the user to try again.
|
|
@xref{Coding Systems}.
|
|
@end defun
|
|
|
|
@node Default Coding Systems
|
|
@subsection Default Coding Systems
|
|
|
|
This section describes variables that specify the default coding
|
|
system for certain files or when running certain subprograms, and the
|
|
function that I/O operations use to access them.
|
|
|
|
The idea of these variables is that you set them once and for all to the
|
|
defaults you want, and then do not change them again. To specify a
|
|
particular coding system for a particular operation in a Lisp program,
|
|
don't change these variables; instead, override them using
|
|
@code{coding-system-for-read} and @code{coding-system-for-write}
|
|
(@pxref{Specifying Coding Systems}).
|
|
|
|
@defvar auto-coding-regexp-alist
|
|
This variable is an alist of text patterns and corresponding coding
|
|
systems. Each element has the form @code{(@var{regexp}
|
|
. @var{coding-system})}; a file whose first few kilobytes match
|
|
@var{regexp} is decoded with @var{coding-system} when its contents are
|
|
read into a buffer. The settings in this alist take priority over
|
|
@code{coding:} tags in the files and the contents of
|
|
@code{file-coding-system-alist} (see below). The default value is set
|
|
so that Emacs automatically recognizes mail files in Babyl format and
|
|
reads them with no code conversions.
|
|
@end defvar
|
|
|
|
@defvar file-coding-system-alist
|
|
This variable is an alist that specifies the coding systems to use for
|
|
reading and writing particular files. Each element has the form
|
|
@code{(@var{pattern} . @var{coding})}, where @var{pattern} is a regular
|
|
expression that matches certain file names. The element applies to file
|
|
names that match @var{pattern}.
|
|
|
|
The @sc{cdr} of the element, @var{coding}, should be either a coding
|
|
system, a cons cell containing two coding systems, or a function name (a
|
|
symbol with a function definition). If @var{coding} is a coding system,
|
|
that coding system is used for both reading the file and writing it. If
|
|
@var{coding} is a cons cell containing two coding systems, its @sc{car}
|
|
specifies the coding system for decoding, and its @sc{cdr} specifies the
|
|
coding system for encoding.
|
|
|
|
If @var{coding} is a function name, the function should take one
|
|
argument, a list of all arguments passed to
|
|
@code{find-operation-coding-system}. It must return a coding system
|
|
or a cons cell containing two coding systems. This value has the same
|
|
meaning as described above.
|
|
@end defvar
|
|
|
|
@defvar process-coding-system-alist
|
|
This variable is an alist specifying which coding systems to use for a
|
|
subprocess, depending on which program is running in the subprocess. It
|
|
works like @code{file-coding-system-alist}, except that @var{pattern} is
|
|
matched against the program name used to start the subprocess. The coding
|
|
system or systems specified in this alist are used to initialize the
|
|
coding systems used for I/O to the subprocess, but you can specify
|
|
other coding systems later using @code{set-process-coding-system}.
|
|
@end defvar
|
|
|
|
@strong{Warning:} Coding systems such as @code{undecided}, which
|
|
determine the coding system from the data, do not work entirely reliably
|
|
with asynchronous subprocess output. This is because Emacs handles
|
|
asynchronous subprocess output in batches, as it arrives. If the coding
|
|
system leaves the character code conversion unspecified, or leaves the
|
|
end-of-line conversion unspecified, Emacs must try to detect the proper
|
|
conversion from one batch at a time, and this does not always work.
|
|
|
|
Therefore, with an asynchronous subprocess, if at all possible, use a
|
|
coding system which determines both the character code conversion and
|
|
the end of line conversion---that is, one like @code{latin-1-unix},
|
|
rather than @code{undecided} or @code{latin-1}.
|
|
|
|
@defvar network-coding-system-alist
|
|
This variable is an alist that specifies the coding system to use for
|
|
network streams. It works much like @code{file-coding-system-alist},
|
|
with the difference that the @var{pattern} in an element may be either a
|
|
port number or a regular expression. If it is a regular expression, it
|
|
is matched against the network service name used to open the network
|
|
stream.
|
|
@end defvar
|
|
|
|
@defvar default-process-coding-system
|
|
This variable specifies the coding systems to use for subprocess (and
|
|
network stream) input and output, when nothing else specifies what to
|
|
do.
|
|
|
|
The value should be a cons cell of the form @code{(@var{input-coding}
|
|
. @var{output-coding})}. Here @var{input-coding} applies to input from
|
|
the subprocess, and @var{output-coding} applies to output to it.
|
|
@end defvar
|
|
|
|
@defvar auto-coding-functions
|
|
This variable holds a list of functions that try to determine a
|
|
coding system for a file based on its undecoded contents.
|
|
|
|
Each function in this list should be written to look at text in the
|
|
current buffer, but should not modify it in any way. The buffer will
|
|
contain undecoded text of parts of the file. Each function should
|
|
take one argument, @var{size}, which tells it how many characters to
|
|
look at, starting from point. If the function succeeds in determining
|
|
a coding system for the file, it should return that coding system.
|
|
Otherwise, it should return @code{nil}.
|
|
|
|
If a file has a @samp{coding:} tag, that takes precedence, so these
|
|
functions won't be called.
|
|
@end defvar
|
|
|
|
@defun find-operation-coding-system operation &rest arguments
|
|
This function returns the coding system to use (by default) for
|
|
performing @var{operation} with @var{arguments}. The value has this
|
|
form:
|
|
|
|
@example
|
|
(@var{decoding-system} . @var{encoding-system})
|
|
@end example
|
|
|
|
The first element, @var{decoding-system}, is the coding system to use
|
|
for decoding (in case @var{operation} does decoding), and
|
|
@var{encoding-system} is the coding system for encoding (in case
|
|
@var{operation} does encoding).
|
|
|
|
The argument @var{operation} should be a symbol, one of
|
|
@code{insert-file-contents}, @code{write-region}, @code{call-process},
|
|
@code{call-process-region}, @code{start-process}, or
|
|
@code{open-network-stream}. These are the names of the Emacs I/O primitives
|
|
that can do coding system conversion.
|
|
|
|
The remaining arguments should be the same arguments that might be given
|
|
to that I/O primitive. Depending on the primitive, one of those
|
|
arguments is selected as the @dfn{target}. For example, if
|
|
@var{operation} does file I/O, whichever argument specifies the file
|
|
name is the target. For subprocess primitives, the process name is the
|
|
target. For @code{open-network-stream}, the target is the service name
|
|
or port number.
|
|
|
|
This function looks up the target in @code{file-coding-system-alist},
|
|
@code{process-coding-system-alist}, or
|
|
@code{network-coding-system-alist}, depending on @var{operation}.
|
|
@end defun
|
|
|
|
@node Specifying Coding Systems
|
|
@subsection Specifying a Coding System for One Operation
|
|
|
|
You can specify the coding system for a specific operation by binding
|
|
the variables @code{coding-system-for-read} and/or
|
|
@code{coding-system-for-write}.
|
|
|
|
@defvar coding-system-for-read
|
|
If this variable is non-@code{nil}, it specifies the coding system to
|
|
use for reading a file, or for input from a synchronous subprocess.
|
|
|
|
It also applies to any asynchronous subprocess or network stream, but in
|
|
a different way: the value of @code{coding-system-for-read} when you
|
|
start the subprocess or open the network stream specifies the input
|
|
decoding method for that subprocess or network stream. It remains in
|
|
use for that subprocess or network stream unless and until overridden.
|
|
|
|
The right way to use this variable is to bind it with @code{let} for a
|
|
specific I/O operation. Its global value is normally @code{nil}, and
|
|
you should not globally set it to any other value. Here is an example
|
|
of the right way to use the variable:
|
|
|
|
@example
|
|
;; @r{Read the file with no character code conversion.}
|
|
;; @r{Assume @acronym{crlf} represents end-of-line.}
|
|
(let ((coding-system-for-read 'emacs-mule-dos))
|
|
(insert-file-contents filename))
|
|
@end example
|
|
|
|
When its value is non-@code{nil}, @code{coding-system-for-read} takes
|
|
precedence over all other methods of specifying a coding system to use for
|
|
input, including @code{file-coding-system-alist},
|
|
@code{process-coding-system-alist} and
|
|
@code{network-coding-system-alist}.
|
|
@end defvar
|
|
|
|
@defvar coding-system-for-write
|
|
This works much like @code{coding-system-for-read}, except that it
|
|
applies to output rather than input. It affects writing to files,
|
|
as well as sending output to subprocesses and net connections.
|
|
|
|
When a single operation does both input and output, as do
|
|
@code{call-process-region} and @code{start-process}, both
|
|
@code{coding-system-for-read} and @code{coding-system-for-write}
|
|
affect it.
|
|
@end defvar
|
|
|
|
@defvar inhibit-eol-conversion
|
|
When this variable is non-@code{nil}, no end-of-line conversion is done,
|
|
no matter which coding system is specified. This applies to all the
|
|
Emacs I/O and subprocess primitives, and to the explicit encoding and
|
|
decoding functions (@pxref{Explicit Encoding}).
|
|
@end defvar
|
|
|
|
@node Explicit Encoding
|
|
@subsection Explicit Encoding and Decoding
|
|
@cindex encoding text
|
|
@cindex decoding text
|
|
|
|
All the operations that transfer text in and out of Emacs have the
|
|
ability to use a coding system to encode or decode the text.
|
|
You can also explicitly encode and decode text using the functions
|
|
in this section.
|
|
|
|
The result of encoding, and the input to decoding, are not ordinary
|
|
text. They logically consist of a series of byte values; that is, a
|
|
series of characters whose codes are in the range 0 through 255. In a
|
|
multibyte buffer or string, character codes 128 through 159 are
|
|
represented by multibyte sequences, but this is invisible to Lisp
|
|
programs.
|
|
|
|
The usual way to read a file into a buffer as a sequence of bytes, so
|
|
you can decode the contents explicitly, is with
|
|
@code{insert-file-contents-literally} (@pxref{Reading from Files});
|
|
alternatively, specify a non-@code{nil} @var{rawfile} argument when
|
|
visiting a file with @code{find-file-noselect}. These methods result in
|
|
a unibyte buffer.
|
|
|
|
The usual way to use the byte sequence that results from explicitly
|
|
encoding text is to copy it to a file or process---for example, to write
|
|
it with @code{write-region} (@pxref{Writing to Files}), and suppress
|
|
encoding by binding @code{coding-system-for-write} to
|
|
@code{no-conversion}.
|
|
|
|
Here are the functions to perform explicit encoding or decoding. The
|
|
decoding functions produce sequences of bytes; the encoding functions
|
|
are meant to operate on sequences of bytes. All of these functions
|
|
discard text properties.
|
|
|
|
@deffn Command encode-coding-region start end coding-system
|
|
This command encodes the text from @var{start} to @var{end} according
|
|
to coding system @var{coding-system}. The encoded text replaces the
|
|
original text in the buffer. The result of encoding is logically a
|
|
sequence of bytes, but the buffer remains multibyte if it was multibyte
|
|
before.
|
|
|
|
This command returns the length of the encoded text.
|
|
@end deffn
|
|
|
|
@defun encode-coding-string string coding-system &optional nocopy
|
|
This function encodes the text in @var{string} according to coding
|
|
system @var{coding-system}. It returns a new string containing the
|
|
encoded text, except when @var{nocopy} is non-@code{nil}, in which
|
|
case the function may return @var{string} itself if the encoding
|
|
operation is trivial. The result of encoding is a unibyte string.
|
|
@end defun
|
|
|
|
@deffn Command decode-coding-region start end coding-system
|
|
This command decodes the text from @var{start} to @var{end} according
|
|
to coding system @var{coding-system}. The decoded text replaces the
|
|
original text in the buffer. To make explicit decoding useful, the text
|
|
before decoding ought to be a sequence of byte values, but both
|
|
multibyte and unibyte buffers are acceptable.
|
|
|
|
This command returns the length of the decoded text.
|
|
@end deffn
|
|
|
|
@defun decode-coding-string string coding-system &optional nocopy
|
|
This function decodes the text in @var{string} according to coding
|
|
system @var{coding-system}. It returns a new string containing the
|
|
decoded text, except when @var{nocopy} is non-@code{nil}, in which
|
|
case the function may return @var{string} itself if the decoding
|
|
operation is trivial. To make explicit decoding useful, the contents
|
|
of @var{string} ought to be a sequence of byte values, but a multibyte
|
|
string is acceptable.
|
|
@end defun
|
|
|
|
@defun decode-coding-inserted-region from to filename &optional visit beg end replace
|
|
This function decodes the text from @var{from} to @var{to} as if
|
|
it were being read from file @var{filename} using @code{insert-file-contents}
|
|
using the rest of the arguments provided.
|
|
|
|
The normal way to use this function is after reading text from a file
|
|
without decoding, if you decide you would rather have decoded it.
|
|
Instead of deleting the text and reading it again, this time with
|
|
decoding, you can call this function.
|
|
@end defun
|
|
|
|
@node Terminal I/O Encoding
|
|
@subsection Terminal I/O Encoding
|
|
|
|
Emacs can decode keyboard input using a coding system, and encode
|
|
terminal output. This is useful for terminals that transmit or display
|
|
text using a particular encoding such as Latin-1. Emacs does not set
|
|
@code{last-coding-system-used} for encoding or decoding for the
|
|
terminal.
|
|
|
|
@defun keyboard-coding-system
|
|
This function returns the coding system that is in use for decoding
|
|
keyboard input---or @code{nil} if no coding system is to be used.
|
|
@end defun
|
|
|
|
@deffn Command set-keyboard-coding-system coding-system
|
|
This command specifies @var{coding-system} as the coding system to
|
|
use for decoding keyboard input. If @var{coding-system} is @code{nil},
|
|
that means do not decode keyboard input.
|
|
@end deffn
|
|
|
|
@defun terminal-coding-system
|
|
This function returns the coding system that is in use for encoding
|
|
terminal output---or @code{nil} for no encoding.
|
|
@end defun
|
|
|
|
@deffn Command set-terminal-coding-system coding-system
|
|
This command specifies @var{coding-system} as the coding system to use
|
|
for encoding terminal output. If @var{coding-system} is @code{nil},
|
|
that means do not encode terminal output.
|
|
@end deffn
|
|
|
|
@node MS-DOS File Types
|
|
@subsection MS-DOS File Types
|
|
@cindex DOS file types
|
|
@cindex MS-DOS file types
|
|
@cindex Windows file types
|
|
@cindex file types on MS-DOS and Windows
|
|
@cindex text files and binary files
|
|
@cindex binary files and text files
|
|
|
|
On MS-DOS and Microsoft Windows, Emacs guesses the appropriate
|
|
end-of-line conversion for a file by looking at the file's name. This
|
|
feature classifies files as @dfn{text files} and @dfn{binary files}. By
|
|
``binary file'' we mean a file of literal byte values that are not
|
|
necessarily meant to be characters; Emacs does no end-of-line conversion
|
|
and no character code conversion for them. On the other hand, the bytes
|
|
in a text file are intended to represent characters; when you create a
|
|
new file whose name implies that it is a text file, Emacs uses DOS
|
|
end-of-line conversion.
|
|
|
|
@defvar buffer-file-type
|
|
This variable, automatically buffer-local in each buffer, records the
|
|
file type of the buffer's visited file. When a buffer does not specify
|
|
a coding system with @code{buffer-file-coding-system}, this variable is
|
|
used to determine which coding system to use when writing the contents
|
|
of the buffer. It should be @code{nil} for text, @code{t} for binary.
|
|
If it is @code{t}, the coding system is @code{no-conversion}.
|
|
Otherwise, @code{undecided-dos} is used.
|
|
|
|
Normally this variable is set by visiting a file; it is set to
|
|
@code{nil} if the file was visited without any actual conversion.
|
|
@end defvar
|
|
|
|
@defopt file-name-buffer-file-type-alist
|
|
This variable holds an alist for recognizing text and binary files.
|
|
Each element has the form (@var{regexp} . @var{type}), where
|
|
@var{regexp} is matched against the file name, and @var{type} may be
|
|
@code{nil} for text, @code{t} for binary, or a function to call to
|
|
compute which. If it is a function, then it is called with a single
|
|
argument (the file name) and should return @code{t} or @code{nil}.
|
|
|
|
When running on MS-DOS or MS-Windows, Emacs checks this alist to decide
|
|
which coding system to use when reading a file. For a text file,
|
|
@code{undecided-dos} is used. For a binary file, @code{no-conversion}
|
|
is used.
|
|
|
|
If no element in this alist matches a given file name, then
|
|
@code{default-buffer-file-type} says how to treat the file.
|
|
@end defopt
|
|
|
|
@defopt default-buffer-file-type
|
|
This variable says how to handle files for which
|
|
@code{file-name-buffer-file-type-alist} says nothing about the type.
|
|
|
|
If this variable is non-@code{nil}, then these files are treated as
|
|
binary: the coding system @code{no-conversion} is used. Otherwise,
|
|
nothing special is done for them---the coding system is deduced solely
|
|
from the file contents, in the usual Emacs fashion.
|
|
@end defopt
|
|
|
|
@node Input Methods
|
|
@section Input Methods
|
|
@cindex input methods
|
|
|
|
@dfn{Input methods} provide convenient ways of entering non-@acronym{ASCII}
|
|
characters from the keyboard. Unlike coding systems, which translate
|
|
non-@acronym{ASCII} characters to and from encodings meant to be read by
|
|
programs, input methods provide human-friendly commands. (@xref{Input
|
|
Methods,,, emacs, The GNU Emacs Manual}, for information on how users
|
|
use input methods to enter text.) How to define input methods is not
|
|
yet documented in this manual, but here we describe how to use them.
|
|
|
|
Each input method has a name, which is currently a string;
|
|
in the future, symbols may also be usable as input method names.
|
|
|
|
@defvar current-input-method
|
|
This variable holds the name of the input method now active in the
|
|
current buffer. (It automatically becomes local in each buffer when set
|
|
in any fashion.) It is @code{nil} if no input method is active in the
|
|
buffer now.
|
|
@end defvar
|
|
|
|
@defopt default-input-method
|
|
This variable holds the default input method for commands that choose an
|
|
input method. Unlike @code{current-input-method}, this variable is
|
|
normally global.
|
|
@end defopt
|
|
|
|
@deffn Command set-input-method input-method
|
|
This command activates input method @var{input-method} for the current
|
|
buffer. It also sets @code{default-input-method} to @var{input-method}.
|
|
If @var{input-method} is @code{nil}, this command deactivates any input
|
|
method for the current buffer.
|
|
@end deffn
|
|
|
|
@defun read-input-method-name prompt &optional default inhibit-null
|
|
This function reads an input method name with the minibuffer, prompting
|
|
with @var{prompt}. If @var{default} is non-@code{nil}, that is returned
|
|
by default, if the user enters empty input. However, if
|
|
@var{inhibit-null} is non-@code{nil}, empty input signals an error.
|
|
|
|
The returned value is a string.
|
|
@end defun
|
|
|
|
@defvar input-method-alist
|
|
This variable defines all the supported input methods.
|
|
Each element defines one input method, and should have the form:
|
|
|
|
@example
|
|
(@var{input-method} @var{language-env} @var{activate-func}
|
|
@var{title} @var{description} @var{args}...)
|
|
@end example
|
|
|
|
Here @var{input-method} is the input method name, a string;
|
|
@var{language-env} is another string, the name of the language
|
|
environment this input method is recommended for. (That serves only for
|
|
documentation purposes.)
|
|
|
|
@var{activate-func} is a function to call to activate this method. The
|
|
@var{args}, if any, are passed as arguments to @var{activate-func}. All
|
|
told, the arguments to @var{activate-func} are @var{input-method} and
|
|
the @var{args}.
|
|
|
|
@var{title} is a string to display in the mode line while this method is
|
|
active. @var{description} is a string describing this method and what
|
|
it is good for.
|
|
@end defvar
|
|
|
|
The fundamental interface to input methods is through the
|
|
variable @code{input-method-function}. @xref{Reading One Event},
|
|
and @ref{Invoking the Input Method}.
|
|
|
|
@node Locales
|
|
@section Locales
|
|
@cindex locale
|
|
|
|
POSIX defines a concept of ``locales'' which control which language
|
|
to use in language-related features. These Emacs variables control
|
|
how Emacs interacts with these features.
|
|
|
|
@defvar locale-coding-system
|
|
@tindex locale-coding-system
|
|
@cindex keyboard input decoding on X
|
|
This variable specifies the coding system to use for decoding system
|
|
error messages and---on X Window system only---keyboard input, for
|
|
encoding the format argument to @code{format-time-string}, and for
|
|
decoding the return value of @code{format-time-string}.
|
|
@end defvar
|
|
|
|
@defvar system-messages-locale
|
|
@tindex system-messages-locale
|
|
This variable specifies the locale to use for generating system error
|
|
messages. Changing the locale can cause messages to come out in a
|
|
different language or in a different orthography. If the variable is
|
|
@code{nil}, the locale is specified by environment variables in the
|
|
usual POSIX fashion.
|
|
@end defvar
|
|
|
|
@defvar system-time-locale
|
|
@tindex system-time-locale
|
|
This variable specifies the locale to use for formatting time values.
|
|
Changing the locale can cause messages to appear according to the
|
|
conventions of a different language. If the variable is @code{nil}, the
|
|
locale is specified by environment variables in the usual POSIX fashion.
|
|
@end defvar
|
|
|
|
@defun locale-info item
|
|
This function returns locale data @var{item} for the current POSIX
|
|
locale, if available. @var{item} should be one of these symbols:
|
|
|
|
@table @code
|
|
@item codeset
|
|
Return the character set as a string (locale item @code{CODESET}).
|
|
|
|
@item days
|
|
Return a 7-element vector of day names (locale items
|
|
@code{DAY_1} through @code{DAY_7});
|
|
|
|
@item months
|
|
Return a 12-element vector of month names (locale items @code{MON_1}
|
|
through @code{MON_12}).
|
|
|
|
@item paper
|
|
Return a list @code{(@var{width} @var{height})} for the default paper
|
|
size measured in millimeters (locale items @code{PAPER_WIDTH} and
|
|
@code{PAPER_HEIGHT}).
|
|
@end table
|
|
|
|
If the system can't provide the requested information, or if
|
|
@var{item} is not one of those symbols, the value is @code{nil}. All
|
|
strings in the return value are decoded using
|
|
@code{locale-coding-system}. @xref{Locales,,, libc, The GNU Libc Manual},
|
|
for more information about locales and locale items.
|
|
@end defun
|
|
|
|
@ignore
|
|
arch-tag: be705bf8-941b-4c35-84fc-ad7d20ddb7cb
|
|
@end ignore
|