mirror of
https://git.savannah.gnu.org/git/emacs.git
synced 2024-11-28 07:45:00 +00:00
9c3dbabee3
These are dates that admin/update-copyright did not update, or updated incorrectly.
476 lines
14 KiB
Plaintext
476 lines
14 KiB
Plaintext
\input texinfo @c -*-texinfo-*-
|
|
@c %**start of header
|
|
@setfilename ../../info/bovine.info
|
|
@set TITLE Bovine parser development
|
|
@set AUTHOR Eric M. Ludlam, David Ponce, and Richard Y. Kim
|
|
@settitle @value{TITLE}
|
|
@include docstyle.texi
|
|
|
|
@c *************************************************************************
|
|
@c @ Header
|
|
@c *************************************************************************
|
|
|
|
@c Merge all indexes into a single index for now.
|
|
@c We can always separate them later into two or more as needed.
|
|
@syncodeindex vr cp
|
|
@syncodeindex fn cp
|
|
@syncodeindex ky cp
|
|
@syncodeindex pg cp
|
|
@syncodeindex tp cp
|
|
|
|
@c @footnotestyle separate
|
|
@c @paragraphindent 2
|
|
@c @@smallbook
|
|
@c %**end of header
|
|
|
|
@copying
|
|
Copyright @copyright{} 1999--2004, 2012--2016 Free Software Foundation, Inc.
|
|
|
|
@quotation
|
|
Permission is granted to copy, distribute and/or modify this document
|
|
under the terms of the GNU Free Documentation License, Version 1.3 or
|
|
any later version published by the Free Software Foundation; with no
|
|
Invariant Sections, with the Front-Cover Texts being ``A GNU Manual,''
|
|
and with the Back-Cover Texts as in (a) below. A copy of the license
|
|
is included in the section entitled ``GNU Free Documentation License''.
|
|
|
|
(a) The FSF's Back-Cover Text is: ``You have the freedom to copy and
|
|
modify this GNU manual.''
|
|
@end quotation
|
|
@end copying
|
|
|
|
@dircategory Emacs misc features
|
|
@direntry
|
|
* Bovine: (bovine). Semantic bovine parser development.
|
|
@end direntry
|
|
|
|
@iftex
|
|
@finalout
|
|
@end iftex
|
|
|
|
@c @setchapternewpage odd
|
|
@c @setchapternewpage off
|
|
|
|
@titlepage
|
|
@sp 10
|
|
@title @value{TITLE}
|
|
@author by @value{AUTHOR}
|
|
@page
|
|
@vskip 0pt plus 1 fill
|
|
@insertcopying
|
|
@end titlepage
|
|
@page
|
|
|
|
@macro semantic{}
|
|
@i{Semantic}
|
|
@end macro
|
|
|
|
@c *************************************************************************
|
|
@c @ Document
|
|
@c *************************************************************************
|
|
@contents
|
|
|
|
@node top
|
|
@top @value{TITLE}
|
|
|
|
The @dfn{bovine} parser is the original @semantic{} parser, and is an
|
|
implementation of an @acronym{LL} parser. It is good for simple
|
|
languages. It has many conveniences making grammar writing easy. The
|
|
conveniences make it less powerful than a Bison-like @acronym{LALR}
|
|
parser. For more information, @inforef{Top, The Wisent Parser Manual,
|
|
wisent}.
|
|
|
|
Bovine @acronym{LL} grammars are stored in files with a @file{.by}
|
|
extension. When compiled, the contents is converted into a file of
|
|
the form @file{NAME-by.el}. This, in turn is byte compiled.
|
|
@inforef{top, Grammar Framework Manual, grammar-fw}.
|
|
|
|
@ifnottex
|
|
@insertcopying
|
|
@end ifnottex
|
|
|
|
@menu
|
|
* Starting Rules:: The starting rules for the grammar.
|
|
* Bovine Grammar Rules:: Rules used to parse a language.
|
|
* Optional Lambda Expression:: Actions to take when a rule is matched.
|
|
* Bovine Examples:: Simple Samples.
|
|
* GNU Free Documentation License:: The license for this documentation.
|
|
@c * Index::
|
|
@end menu
|
|
|
|
@node Starting Rules
|
|
@chapter Starting Rules
|
|
|
|
In Bison, one and only one nonterminal is designated as the ``start''
|
|
symbol. In @semantic{}, one or more nonterminals can be designated as
|
|
the ``start'' symbol. They are declared following the @code{%start}
|
|
keyword separated by spaces. @inforef{start Decl, ,grammar-fw}.
|
|
|
|
If no @code{%start} keyword is used in a grammar, then the very first
|
|
is used. Internally the first start nonterminal is targeted by the
|
|
reserved symbol @code{bovine-toplevel}, so it can be found by the
|
|
parser harness.
|
|
|
|
To find locally defined variables, the local context handler needs to
|
|
parse the body of functional code. The @code{scopestart} declaration
|
|
specifies the name of a nonterminal used as the goal to parse a local
|
|
context, @inforef{scopestart Decl, ,grammar-fw}. Internally the
|
|
scopestart nonterminal is targeted by the reserved symbol
|
|
@code{bovine-inner-scope}, so it can be found by the parser harness.
|
|
|
|
@node Bovine Grammar Rules
|
|
@chapter Bovine Grammar Rules
|
|
|
|
The rules are what allow the compiler to create tags from a language
|
|
file. Once the setup is done in the prologue, you can start writing
|
|
rules. @inforef{Grammar Rules, ,grammar-fw}.
|
|
|
|
@example
|
|
@var{result} : @var{components1} @var{optional-semantic-action1})
|
|
| @var{components2} @var{optional-semantic-action2}
|
|
;
|
|
@end example
|
|
|
|
@var{result} is a nonterminal, that is a symbol synthesized in your grammar.
|
|
@var{components} is a list of elements that are to be matched if @var{result}
|
|
is to be made. @var{optional-semantic-action} is an optional sequence
|
|
of simplified Emacs Lisp expressions for concocting the parse tree.
|
|
|
|
In bison, each time an element of @var{components} is found, it is
|
|
@dfn{shifted} onto the parser stack. (The stack of matched elements.)
|
|
When all @var{components}' elements have been matched, it is
|
|
@dfn{reduced} to @var{result}. @xref{Algorithm,,, bison, The GNU Bison Manual}.
|
|
|
|
A particular @var{result} written into your grammar becomes
|
|
the parser's goal. It is designated by a @code{%start} statement
|
|
(@pxref{Starting Rules}). The value returned by the associated
|
|
@var{optional-semantic-action} is the parser's result. It should be
|
|
a tree of @semantic{} @dfn{tags}, @inforef{Semantic Tags, ,
|
|
semantic-appdev}.
|
|
|
|
@var{components} is made up of symbols. A symbol such as @code{FOO}
|
|
means that a syntactic token of class @code{FOO} must be matched.
|
|
|
|
@menu
|
|
* How Lexical Tokens Match::
|
|
* Grammar-to-Lisp Details::
|
|
* Order of components in rules::
|
|
@end menu
|
|
|
|
@node How Lexical Tokens Match
|
|
@section How Lexical Tokens Match
|
|
|
|
A lexical rule must be used to define how to match a lexical token.
|
|
|
|
For instance:
|
|
|
|
@example
|
|
%keyword FOO "foo"
|
|
@end example
|
|
|
|
Means that @code{FOO} is a reserved language keyword, matched as such
|
|
by looking up into a keyword table, @inforef{keyword Decl,
|
|
,grammar-fw}. This is because @code{"foo"} will be converted to
|
|
@code{FOO} in the lexical analysis stage. Thus the symbol @code{FOO}
|
|
won't be available any other way.
|
|
|
|
If we specify our token in this way:
|
|
|
|
@example
|
|
%token <symbol> FOO "foo"
|
|
@end example
|
|
|
|
then @code{FOO} will match the string @code{"foo"} explicitly, but it
|
|
won't do so at the lexical level, allowing use of the text
|
|
@code{"foo"} in other forms of regular expressions.
|
|
|
|
In that case, @code{FOO} is a @code{symbol}-type token. To match, a
|
|
@code{symbol} must first be encountered, and then it must
|
|
@code{string-match "foo"}.
|
|
|
|
@table @strong
|
|
@item Caution:
|
|
Be especially careful to remember that @code{"foo"}, and more
|
|
generally the %token's match-value string, is a regular expression!
|
|
@end table
|
|
|
|
Non symbol tokens are also allowed. For example:
|
|
|
|
@example
|
|
%token <punctuation> PERIOD "[.]"
|
|
|
|
filename : symbol PERIOD symbol
|
|
;
|
|
@end example
|
|
|
|
@code{PERIOD} is a @code{punctuation}-type token that will explicitly
|
|
match one period when used in the above rule.
|
|
|
|
@table @strong
|
|
@item Please Note:
|
|
@code{symbol}, @code{punctuation}, etc., are predefined lexical token
|
|
types, based on the @dfn{syntax class}-character associations
|
|
currently in effect.
|
|
@end table
|
|
|
|
@node Grammar-to-Lisp Details
|
|
@section Grammar-to-Lisp Details
|
|
|
|
For the bovinator, lexical token matching patterns are @emph{inlined}.
|
|
When the grammar-to-lisp converter encounters a lexical token
|
|
declaration of the form:
|
|
|
|
@example
|
|
%token <@var{type}> @var{token-name} @var{match-value}
|
|
@end example
|
|
|
|
It substitutes every occurrences of @var{token-name} in rules, by its
|
|
expanded form:
|
|
|
|
@example
|
|
@var{type} @var{match-value}
|
|
@end example
|
|
|
|
For example:
|
|
|
|
@example
|
|
%token <symbol> MOOSE "moose"
|
|
|
|
find_a_moose: MOOSE
|
|
;
|
|
@end example
|
|
|
|
Will generate this pseudo equivalent-rule:
|
|
|
|
@example
|
|
find_a_moose: symbol "moose" ;; invalid syntax!
|
|
;
|
|
@end example
|
|
|
|
Thus, from the bovinator point of view, the @var{components} part of a
|
|
rule is made up of symbols and strings. A string in the mix means
|
|
that the previous symbol must have the additional constraint of
|
|
exactly matching it, as described in @ref{How Lexical Tokens Match}.
|
|
|
|
@table @strong
|
|
@item Please Note:
|
|
For the bovinator, this task was mixed into the language definition to
|
|
simplify implementation, though Bison's technique is more efficient.
|
|
@end table
|
|
|
|
@node Order of components in rules
|
|
@section Order of components in rules
|
|
|
|
If a rule has multiple components, order is important, for example
|
|
|
|
@example
|
|
headerfile : symbol PERIOD symbol
|
|
| symbol
|
|
;
|
|
@end example
|
|
|
|
would match @samp{foo.h} or the @acronym{C++} header @samp{foo}.
|
|
The bovine parser will first attempt to match the long form, and then
|
|
the short form. If they were in reverse order, then the long form
|
|
would never be tested.
|
|
|
|
@c @xref{Default syntactic tokens}.
|
|
|
|
@node Optional Lambda Expression
|
|
@chapter Optional Lambda Expressions
|
|
|
|
The @acronym{OLE} (@dfn{Optional Lambda Expression}) is converted into
|
|
a bovine lambda. This lambda has special short-cuts to simplify
|
|
reading the semantic action definition. An @acronym{OLE} like this:
|
|
|
|
@example
|
|
( $1 )
|
|
@end example
|
|
|
|
results in a lambda return which consists entirely of the string
|
|
or object found by matching the first (zeroth) element of match.
|
|
An @acronym{OLE} like this:
|
|
|
|
@example
|
|
( ,(foo $1) )
|
|
@end example
|
|
|
|
executes @code{foo} on the first argument, and then splices its return
|
|
into the return list whereas:
|
|
|
|
@example
|
|
( (foo $1) )
|
|
@end example
|
|
|
|
executes @code{foo}, and that is placed in the return list.
|
|
|
|
Here are other things that can appear inline:
|
|
|
|
@table @code
|
|
@item $1
|
|
The first object matched.
|
|
|
|
@item ,$1
|
|
The first object spliced into the list (assuming it is a list from a
|
|
non-terminal).
|
|
|
|
@item '$1
|
|
The first object matched, placed in a list. I.e., @code{( $1 )}.
|
|
|
|
@item foo
|
|
The symbol @code{foo} (exactly as displayed).
|
|
|
|
@item (foo)
|
|
A function call to foo which is stuck into the return list.
|
|
|
|
@item ,(foo)
|
|
A function call to foo which is spliced into the return list.
|
|
|
|
@item '(foo)
|
|
A function call to foo which is stuck into the return list in a list.
|
|
|
|
@item (EXPAND @var{$1} @var{nonterminal} @var{depth})
|
|
A list starting with @code{EXPAND} performs a recursive parse on the
|
|
token passed to it (represented by @samp{$1} above.) The
|
|
@dfn{semantic list} is a common token to expand, as there are often
|
|
interesting things in the list. The @var{nonterminal} is a symbol in
|
|
your table which the bovinator will start with when parsing.
|
|
@var{nonterminal}'s definition is the same as any other nonterminal.
|
|
@var{depth} should be at least @samp{1} when descending into a
|
|
semantic list.
|
|
|
|
@item (EXPANDFULL @var{$1} @var{nonterminal} @var{depth})
|
|
Is like @code{EXPAND}, except that the parser will iterate over
|
|
@var{nonterminal} until there are no more matches. (The same way the
|
|
parser iterates over the starting rule (@pxref{Starting Rules}). This
|
|
lets you have much simpler rules in this specific case, and also lets
|
|
you have positional information in the returned tokens, and error
|
|
skipping.
|
|
|
|
@item (ASSOC @var{symbol1} @var{value1} @var{symbol2} @var{value2} @dots{})
|
|
This is used for creating an association list. Each @var{symbol} is
|
|
included in the list if the associated @var{value} is non-@code{nil}.
|
|
While the items are all listed explicitly, the created structure is an
|
|
association list of the form:
|
|
|
|
@example
|
|
((@var{symbol1} . @var{value1}) (@var{symbol2} . @var{value2}) @dots{})
|
|
@end example
|
|
|
|
@item (TAG @var{name} @var{class} [@var{attributes}])
|
|
This creates one tag in the current buffer.
|
|
|
|
@table @var
|
|
@item name
|
|
Is a string that represents the tag in the language.
|
|
|
|
@item class
|
|
Is the kind of tag being create, such as @code{function}, or
|
|
@code{variable}, though any symbol will work.
|
|
|
|
@item attributes
|
|
Is an optional set of labeled values such as @code{:constant-flag t :parent
|
|
"parenttype"}.
|
|
@end table
|
|
|
|
@item (TAG-VARIABLE @var{name} @var{type} @var{default-value} [@var{attributes}])
|
|
@itemx (TAG-FUNCTION @var{name} @var{type} @var{arg-list} [@var{attributes}])
|
|
@itemx (TAG-TYPE @var{name} @var{type} @var{members} @var{parents} [@var{attributes}])
|
|
@itemx (TAG-INCLUDE @var{name} @var{system-flag} [@var{attributes}])
|
|
@itemx (TAG-PACKAGE @var{name} @var{detail} [@var{attributes}])
|
|
@itemx (TAG-CODE @var{name} @var{detail} [@var{attributes}])
|
|
Create a tag with @var{name} of respectively the class
|
|
@code{variable}, @code{function}, @code{type}, @code{include},
|
|
@code{package}, and @code{code}.
|
|
See @inforef{Creating Tags, , semantic-appdev} for the lisp
|
|
functions these translate into.
|
|
@end table
|
|
|
|
If the symbol @code{%quotemode backquote} is specified, then use
|
|
@code{,@@} to splice a list in, and @code{,} to evaluate the expression.
|
|
This lets you send @code{$1} as a symbol into a list instead of having
|
|
it expanded inline.
|
|
|
|
@node Bovine Examples
|
|
@chapter Examples
|
|
|
|
The rule:
|
|
|
|
@example
|
|
any-symbol: symbol
|
|
;
|
|
@end example
|
|
|
|
is equivalent to
|
|
|
|
@example
|
|
any-symbol: symbol
|
|
( $1 )
|
|
;
|
|
@end example
|
|
|
|
which, if it matched the string @samp{"A"}, would return
|
|
|
|
@example
|
|
( "A" )
|
|
@end example
|
|
|
|
If this rule were used like this:
|
|
|
|
@example
|
|
%token <punctuation> EQUAL "="
|
|
@dots{}
|
|
assign: any-symbol EQUAL any-symbol
|
|
( $1 $3 )
|
|
;
|
|
@end example
|
|
|
|
it would match @samp{"A=B"}, and return
|
|
|
|
@example
|
|
( ("A") ("B") )
|
|
@end example
|
|
|
|
The letters @samp{A} and @samp{B} come back in lists because
|
|
@samp{any-symbol} is a nonterminal, not an actual lexical element.
|
|
|
|
To get a better result with nonterminals, use @asis{,} to splice lists
|
|
in like this:
|
|
|
|
@example
|
|
%token <punctuation> EQUAL "="
|
|
@dots{}
|
|
assign: any-symbol EQUAL any-symbol
|
|
( ,$1 ,$3 )
|
|
;
|
|
@end example
|
|
|
|
which would return
|
|
|
|
@example
|
|
( "A" "B" )
|
|
@end example
|
|
|
|
@node GNU Free Documentation License
|
|
@appendix GNU Free Documentation License
|
|
|
|
@include doclicense.texi
|
|
|
|
@c There is nothing to index at the moment.
|
|
@ignore
|
|
@node Index
|
|
@unnumbered Index
|
|
@printindex cp
|
|
@end ignore
|
|
|
|
@iftex
|
|
@contents
|
|
@summarycontents
|
|
@end iftex
|
|
|
|
@bye
|
|
|
|
@c Following comments are for the benefit of ispell.
|
|
|
|
@c LocalWords: bovinator inlined
|