Copyright: Copyright © 1999 Bumble-Bee Software Ltd.
Citation: Copyright © 1999 Bumble-Bee Software Ltd.


Parser Generator Help Index

Parser Generator

Build Tools

YACC Language

Lex Language

YACC and Lex Library

License


Parser Generator

How To...

Setup Visual C++ Version 4.00 (including Versions 5.00 and 6.00)

Setup Visual C++ Version 1.52

Setup Borland C++Builder Version 3.0

Setup Borland C++ Version 4.52 (including Version 5.00)

Setup Other C/C++ Compilers

Solve Common Linker Problems

Menus

File Menu

Edit Menu

View Menu

Project Menu

Options Menu

Window Menu

Help Menu

Control Bars

Toolbar

Status Bar

Windows

Editor Window

Output Window


ParserWizard Dialog Box - Project Options

The following options allow you to specify the project options:

Name

Type the name of the project you want to create. The name should not include any directory information. If you attempt to enter any, then the name will be rejected. The extension .pgp (parser generated project) is added to the name by default if it is omitted.

Directory

Type the directory where you want the project file and any other files to be created. The directory must be specified as a full path.  That is it must contain a drive letter such as "c:\" etc.  If the directory you type does not exist, then Parser Generator will create it for you.  You can also use the browse button next to the edit box to display the Choose Directory dialog box.  You can use this dialog box to navigate and select an existing directory.

Target Language

The following options allow you to specify the target language for the generated parsers and/or lexical analysers:

C

Select this option to generate C compatible parsers and/or lexical analysers.  Note that these parsers and/or lexical analysers are also C++ compatible, however they do not make use of classes.

C++

Select this option to generate C++ compatible parsers and/or lexical analysers.  These parsers and/or lexical analysers are C++ specific and make use of classes.

Compiler

The following options allow you to specify the compiler that you will be using to compile the parsers and/or lexical analysers:

Microsoft Visual C++

Select this option to specify a set of AYACC and ALex options which are compatible with the Microsoft Visual C++ range of compilers.

Borland C++

Select this option to specify a set of AYACC and Alex options which are compatible with the Borland C++ range of compilers, including Borland C++Builder.  Currently, the only difference between Microsoft Visual C++ and Borland C++ concerns the Do not treat #line directive file names as strings option in both the AYACC Options and ALex Options dialog boxes.  For Borland C++ the option is selected, whereas for Microsoft Visual C++ it isn't.

Other Compiler

Select this option to specify a standard set of AYACC and Alex options.  Currently, these are equivalent to the options specified for Microsoft Visual C++.

See Also

ParserWizard Dialog Box - Project Files


ParserWizard Dialog Box - Project Files

The following options allow you to specify the project files:

Files

The following options allow you to specify what files to create:

YACC And Lex File

Select this option to create both a YACC and a Lex file.  The files are assumed to be interdependent.  That is the parser in the YACC file is assumed to use the lexical analyser in the Lex file, and vice versa.

YACC File Only

Select this option to create a YACC file only.

Lex File Only

Select this option to create a Lex file only.

Create Main Function

Select this option to create a main function.  The function will invoke the generated parser and/or lexical analyser in the correct fashion depending upon the options selected.  If a YACC file is created, then the version of main will reside there.  Otherwise it will reside in the Lex file.

Use Compact Tables

Select this option to generate parsers and/or lexical analysers which use compact tables.  These are slower, but more compact, than the default (fast) tables which are typically used.

Model

The following options allow you to specify what parser/lexical analyser model to use.  This option only applies to C parsers and lexical analysers.  C++ parsers and lexical analysers are always treated as though they were multiple instance model parsers/lexical analysers:

Single

Select this option to create single model parsers and/or lexical analysers.  This model allows only one C parser and one C lexical analyser to reside in an application.  Variables are held statically and the parser's stack and the lexical analyser's buffers can be static and/or dynamic.

Multiple

Select this option to create multiple model parsers and/or lexical analysers.  This model allows one or more C parsers and lexical analysers to reside in an application.  Parser and lexical analyser information is now held in a structure, although this is still statically allocated.  The parser's stack and the lexical analyser's buffers can be statically and/or dynamically allocated.

Multiple Instance

Select this option to create multiple instance model parsers and/or lexical analysers.  This model allows one or more C parsers and lexical analysers to reside in an application.  However in addition it also allows one or more instances of the same parser and/or lexical analyser to be active simultaneously.  Parser and lexical analyser information is still held in a structure, although usually dynamically.  That parser's stack and the lexical analyser's buffers are always dynamically allocated.

See Also

ParserWizard Dialog Box - Project Options, ParserWizard Dialog Box - YACC File, ParserWizard Dialog Box - Lex File


ParserWizard Dialog Box - YACC File

The following options allow you to specify the YACC file:

YACC File Name

Type the name of the YACC file you want to create.  The name should not include any directory information.  If you attempt to enter any, then the file name will be rejected.  The file name must have the extension .y or .yac.

Parser Name

Type the name of the parser.  This option does not apply to single model C parsers.  The name entered should be a valid C/C++ identifier.  The parser name is used to identify a structure (or class) generated by AYACC, and which represents the parser.

Parser

The following options allow you to specify what parser to generate:

Simple LR

Select this option to generate a Simple LR parser.  This is the least powerful parser that can be generated.  In many cases this parser generation method is quite adequate, although it does have trouble with certain programming language constructs.  This parser generation method is the fastest of the three, and has tables sizes comparable with an LALR parser.  It is primarily provided for completeness.

Lookahead LR

Select this option to generate a Lookahead LR parser.  This is the parser that is typically generated.  In terms of power it lies between an SLR and a CLR parser.

Canonical LR

Select this option to generate a Canonical LR parser.  This is the most powerful parser that can be generated.  It does however have considerably larger tables.  Only use in extremis.  Note that a CLR parser cannot represent every grammar.  Far from it.  If you experience shift-reduce or reduce-reduce conflicts then you will need to refine your grammar.

See Also

ParserWixard Dialog Box - Project Files, ParserWizard Dialog Box - Lex File


ParserWiazrd Dialog Box - Lex File
ParserWizard Dialog Box - Lex File

The following options allow you to specify the Lex file:

Lex File Name

Type the name of the Lex file you want to create.  The name should not include any directory information.  If you attempt to enter any, then the file name will be rejected.  The file name must have the extension .l or .lex.

Lexical Analyser Name

Type the name of the lexical analyser.  This option does not apply to single model C lexical analysers.  The name entered should be a valid C/C++ identifier.  The lexical analyser name is used to identify a structure (or class) generated by ALex, and which represents the lexical analyser.

See Also

ParserWizard Dialog Box - Project Files, ParserWizard Dialog Box - YACC File


Line Dialog Box


Provides a space for you to specify the line to be displayed.  Note that line numbers start at one.  If you specify a line number which is past the end of the document, then the last line in the document will be displayed.


Closes this dialog box and displays the specified line in the active document.


Closes this dialog box without displaying a line.


New Project Dialog Box


Provides a space for you to specify the name of the project to be created.  If the project name given is not a fully qualified path name, then the current working directory is used to create one.  If a file name extension is omitted then .pgp (parser generator project) is added by default.


Lists the target languages that a parser and/or lexical analyser can be generated for.  Select the target language that you require.  Note that you can subsequently change the target language from the Project Options dialog box.


Lists the compilers that are directly supported by Parser Generator.  Select the compiler that you will be using to compile the files created by Parser Generator.  This option has an effect on the tool options that are initially selected for the project.  Note that you can subsequently change any option from the Project Options dialog box.  At the moment, if Borland C++ is selected then the Do not treat #line directive file names as strings option in both the AYACC Options and ALex Options dialog boxes is selected.  If Microsoft Visual C++ or Other Compiler is selected then the option is left un-selected.


Closes this dialog box creating the specified project.


Closes this dialog box without creating a project.


Opens a dialog box where you can specify the project name and directory.


Edit Project Dialog Box


Lists the files in the project.  Beware of adding YACC and Lex files with the same base name to a project, for instance myfile.y and myfile.l.  If you use the default project options, then they will both generate the same output and include files, in this case myfile.c and myfile.h.  One set of files will always overwrite the other, depending upon the order in which they are built.


Closes this dialog box and saves any changes you have made.


Closes this dialog box without saving any changes you have made.


Opens a dialog box where you can add files to the project.


Deletes the specified file from the project.


Project Options Dialog Box


Opens a dialog box where you can change the AYACC options.


Opens a dialog box where you can change the ALex options.


Closes this dialog box and saves any changes you have made.


Closes this dialog box without saving any changes you have made.


Sets the AYACC and ALex options back to their default values.  Note that the Target Language and Compiler combo boxes in the New Project dialog box affects what these default values are.


AYACC Options Dialog Box


Displays the resultant options that are passed to AYACC when it is run.  This string cannot be modified directly.


Lists the different categories that options can be specified for.  Select the category that you wish to view and/or set the options for.


Closes this dialog box and saves any changes you have made.


Closes this dialog box without saving any changes you have made.


Sets the AYACC options back to their default values.  Note that the Target Language and Compiler combo boxes in the New Project dialog box affects what these default values are.


AYACC Options Dialog Box General Category


If selected, suppresses the display of the sign-on banner.  The sign-on banner includes the program name and copyright messages, along with the trailing error count message.  If this option is selected, all that will be seen in the Output window are any error messages that are generated.


If selected, specifies that a compact representation will be used to store the parser's tables.  By default a fast representation using a table lookup mechanism is used.  A compact parser also uses a slightly different way of storing Character Tokens, at least ones which consist of a single character.  Instead of converting the character to its ASCII equivalent, the character token is output as is to the tables.  It is then the job of the C/C++ compiler to convert the now character constant into an integer value.


Lists the target languages that a parser can be generated for.  Select the target language that you require.  Note that there are slight option variations depending upon which target you select.  If you select the C language, then output file names will have a default extension of .c, and the default parser model will be a Single Parser.  If, on the other hand, you select the C++ language, output file names will have a default extension of .cpp, and the default parser model will be a Multiple Instance Parser.  Indeed a C++ parser cannot have any other parser model.


Lists the available parser models that can be used. This option is only available if you have selected the C language as your Target Language.  A C++ parser is always treated similar to a Multiple Instance Parser model, except in the description below substitute class for structure.

For a C parser, the model chosen will depend upon the environment that the parser is going to be used in.  A Single Parser model allows only one parser to reside in an application.  Variables are held statically, and the stack can be static and/or dynamic.  A Multiple Parser model allows one or more parsers to reside in an application.  Parser information is now held in a structure, although this is still statically allocated.  Again the stack can be statically and/or dynamically allocated.  A Multiple Instance Parser model allows one or more parsers to reside in an application.  However, in addition it also allows one or more instances of the same parser to be active simultaneously.  Parser information is again held in a structure, although usually dynamically.  The stacks are always dynamically allocated.


Lists the available parsers that can be generated.  The Simple LR (SLR) parser is the least powerful parser that can be generated.  In many cases this parser generation method is quite adequate, although it does have trouble with certain programming language constructs.  This parser generation method is also the fastest of the three, and has table sizes comparable with an LALR parser.  It is provided primarily for completeness.  The Lookahead LR (LALR) parser is the default parser that is generated.  In terms of power it lies between an SLR and a CLR parser.  The Canonical LR (CLR) parser is the most powerful parser that can be generated.  It does however have considerably larger tables.  Only use in extremis.  Note that a CLR parser cannot represent every grammar.  Far from it.  If you experience shift-reduce and reduce-reduce conflicts then you will need to refine your grammar.


Provides a space for you to specify any other, additional options to pass to AYACC.  For a complete listing of all the options which AYACC takes, see AYACC Operation in the online help.


AYACC Options Dialog Box Files Category


If selected, specifies that the generated output and include files will be named y_tab, and the verbose file will be named y_output.  The extension added to the output and include file names is specified by the Output file name extension and Include file name extension options.  This option is used to provide compatibility with other versions of Lex.  Ordinarily, the generated output, include and verbose file names are based on the input file name, except with a different extension.


If selected, specifies that an include file is generated.  The include file name is normally based on the input file name, except with a different extension.  If the Use compatible file names y_tab and y_output option is selected, then the include file will be named y_tab instead.  The extension added to the include file name is specified by the Include file name extension option.  The include file contains definitions for the tokens defined by the %token, %left, %right and %nonassoc keywords.  It also contains the attribute type defined by the %union keyword, the parser name defined by the %name keyword where applicable plus any associated code, and code specified by the %include keyword.


If selected, specifies that a verbose file is generated.  The verbose file is normally based on the input file, except with the extension .v.  If the Use compatible file names y_tab and y_output option is selected, then the verbose file name will be named y_output instead.  The verbose file lists all of the states in the parser, along with the kernel items, actions and gotos in each state, together with any shift-reduce and reduce-reduce conflicts which occur.


Provides a space for you to specify the extension to be appended to the output file name.  The default file name extension depends upon what Target Language you have selected for your parser.  For a C parser the default extension is .c, and for a C++ parser the default extension is .cpp.

Be aware that the extension appended to a file name can govern the way in which it is compiled.  For instance, in Visual C++ files with the extensions .cpp and .cxx are compiled with the C++ compiler, whereas files with the extension .c are compiled with the C compiler.


Provides a space for you to specify the extension to be appended to the include file name.  The default file name extension is always .h, no matter what Target Language is selected.


AYACC Options Dialog Box Table Compression Category


If selected, default action states are not used to compress the parser's action table.  This will typically result in a parser that is faster, but larger in size.  The default state compression technique makes use of similarities between states.  If two states have similar actions, then it is possible to remove the actions from one state and use those in the other state (the default action state).


If selected, no error actions are introduced when finding default action states.  This will typically result in a parser that is slightly faster, but also slightly larger in size.  This option will have no effect if the No Default Action States option is selected.  Error actions allow states to choose from a wider range of states when looking for default action states.  They do this by allowing states to introduce error actions to themselves, in order to make potential default action states fit.  For instance, a state may find a preferable default action state which has maybe one or two actions which are not present in the state.  The state can then introduce a couple of error actions to cover these cases.  When the parser is running, these error actions prevent the parser from looking at the corresponding actions in the default action state.


Lists the available default reductions that can be used to compress the parser's action table.  Note that these options can change the parser that is generated by AYACC.  However, these effects are only made apparent when, or if, the parser encounters a syntax error.  In such circumstances, these options can cause the syntax error to be trapped earlier or later.  Indeed, the options can be used positively to control your error trapping strategy.

The No Default Reductions option specifies that no default reductions will be used by the states in the parser.  This is the only option which does not modify the generated parser in any way.  The Default Reductions If No Other Actions option specifies that if a state has no other actions, other than a reduction action (or actions providing the rule being reduced is the same), then the reduction will be made the default reduction.  This option is primarily for compatibility with BYACC.  The Default Reductions For Error Token Reductions Only option specifies that if a state has a reduction action for the error token, then the reduction will be made the default reduction.  The Default Reductions Favouring Error Token Reductions option specifies that if a state has a reduction action for the error token, then the reduction will be made the default reduction.  Otherwise, the most common reduction in the state will be made the default reduction.  The Default Reductions option specifies that the most common reduction in the state will be made the default reduction.  This is the default option.

In all of the cases above, whenever a default reduction is found, all reduction actions for the rule concerned will be removed from the state.


If selected, no default reductions will be introduced to states if they have a shift action for the error token.  This option is intended to prevent the parser from reducing past a potential error recovery rule.  The option only has an effect if the Default Reductions Favouring Error Token Reductions or Default Reductions options are selected for the Default Reductions option.


If selected, default goto states are not used to compress the parser's goto table.  This will typically result in a parser that is faster, but larger in size.  The default state compression technique makes use of similarities between states.  If two states have similar gotos, then it is possible to remove the gotos from one state and use those in the other state (the default goto state).


If selected, no error gotos are introduced when finding default goto states.  This will typically result in a parser that is slightly faster, but also slightly larger in size.  This option will have no effect if the No Default Goto States option is selected.  Error gotos allow states to choose from a wider range of states when looking for default goto states.  They do this by allowing states to introduce error gotos to themselves, in order to make potential default goto states fit.  For instance, a state may find a preferable default goto state which has maybe one or two gotos which are not present in the state.  The state can then introduce a couple of error gotos to cover these cases.  When the parser is running, these error gotos prevent the parser from looking at the corresponding gotos in the default goto state.


AYACC Options Dialog Box #line Directives Category


If selected, specifies that short file names will be used in the generated output and include files.  In particular this option affects any #include and #line directives that are output to the files.  This option is primarily intended to provide support for 16-bit compilers, or those that cannot handle long file names.  Short file names are those that are limited to 8 file name characters and 3 file extension characters.  These file names are also sometimes called 8.3 file names.


If selected, no #line directives are generated in the output file.  The #line directives enable the C/C++ compiler to map from the C/C++ file back to the YACC source file.  This is useful for correcting compilation errors.  Sometimes, however it is better if these mappings did not take place, for instance if you want to examine stack attributes in the debugger.


If selected, #line directive file names are not treated as strings.  Instead they will be treated as file names, similar to those used in #include directives.  The Microsoft Visual C++ compiler treats #line directive file names as strings.  The Borland C++ compiler on the other hand treats #line directive file names as proper file names.  The important point about strings is that they have an escape character '\'.  Unfortunately, in Windows this is the directory partitioning character.  Taking an example, the string "c:\\myproj\\myparser.y" is equivalent to the file name c:\myproj\myparser.y.


ALex Options Dialog Box


Displays the resultant options that are passed to ALex when it is run.  This string cannot be modified directly.


Lists the different categories that options can be specified for.  Select the category that you wish to view and/or set the options for.


Closes this dialog box and saves any changes you have made.


Closes this dialog box without saving any changes you have made.


Sets the ALex options back to their default values.  Note that the Target Language and Compiler combo boxes in the New Project dialog box affects what these default values are.


ALex Options Dialog Box General Category


If selected, suppresses the display of the sign-on banner.  The sign-on banner includes the program name and copyright messages, along with the trailing error count message.  If this option is selected, all that will be seen in the Output window are any error messages that are generated.


If selected, specifies that a compact representation will be used to store the lexical analyser's tables.  By default a fast representation using a table lookup mechanism is used.


Lists the target languages that a lexical analyser can be generated for.  Select the target language that you require.  Note that there are slight option variations depending upon which target you select.  If you select the C language, then output file names will have a default extension of .c, and the default lexical analyser model will be a Single Lexical Analyser.  If, on the other hand, you select the C++ language, output file names will have a default extension of .cpp, and the default lexical analyser model will be a Multiple Instance Lexical Analyser.  Indeed a C++ lexical analyser cannot have any other lexical analyser model.


Lists the available lexical analyser models that can be used.  The option is only available if you have selected the C language as your Target Language.  A C++ lexical analyser is always treated similar to a Multiple Instance Lexical Analyser model, except in the description below substitute class for structure.

For a C lexical analyser, the model chosen will depend upon the environment that the lexical analyser is going to be used in.  A Single Lexical Analyser model allows only one lexical analyser to reside in an application.  Variables are held statically, and the buffers can be static and/or dynamic.  A Multiple Lexical Analyser model allows one or more lexical analysers to reside in an application.  Lexical analyser information is now held in a structure, although this is still statically allocated.  Again the buffers can be statically and/or dynamically allocated.  A Multiple Instance Lexical Analyser model allows one or more lexical analysers to reside in an application.  However, in addition it also allows one or more instances of the same lexical analyser to be active simultaneously.  Lexical analyser information is again held in a structure, although usually dynamically.  The buffers are always dynamically allocated.


If selected, specifies that the generated lexical analyser will be case insensitive.  For instance an upper case A and a lower case a will be treated as the same character.  Some languages, for example BASIC, are case insensitive.  Taking the FOR keyword as an example, it is possible to specify it as the regular expression [Ff][Oo][Rr].  If, however this option is selected, it can be stated more simply as the regular expression FOR.


If selected, specifies that every regular expression which recognises a string can be matched.  This option is for use with the yyreject and yymreject functions and the yylexer::yyreject member function, all of which hereafter are referred to as yyreject.  Ordinarily, a string can only be matched by one regular expression.  If a string is recognised by two or more regular expressions, then it is always the expression that is listed first in the Lex source file that matches it.  In such circumstances, the next matched string after a call to yyreject will always be shorter than the previous one.  The one notable exception being a string of length one.  In which case, the character is matched by the default rule, output, and then discarded.

With this option selected, the next matched string after a call to yyreject may be the same length as the previous one.  If there is a regular expression further down the Lex source file which recognises the string then it is matched.  Otherwise, as before the next matched string will be shorter.  A consequence of using this option is that no unmatched expression warnings will be generated by Lex.


Provides a space for you to specify any other, additional options to pass to ALex.  For a complete listing of all the options which ALex takes, see ALex Operation in the online help.


ALex Options Dialog Box Files Category


If selected, specifies that the generated output and include files will be named lex_yy.  The extension added to the file names is specified by the Output file name extension and Include file name extension options.  This option is used to provide compatibility with other versions of Lex.  Ordinarily, the generated output and include file names are based on the input file name, except with a different extension.


If selected, specifies that an include file is generated.  The include file name is normally based on the input file name, except with a different extension.  If the Use compatible file name lex_yy option is selected, then the include file will be named lex_yy instead.  The extension added to the include file name is specified by the Include file name extension option.  The include file contains definitions for the start states defined by the %start and %exclusive keywords.  It also contains the lexical analyser name defined by the %name keyword where applicable plus any associated code, and code specified by the %include keyword.


Provides a space for you to specify the extension to be appended to the output file name.  The default file name extension depends upon what Target Language you have selected for your lexical analyser.  For a C lexical analyser the default extension is .c, and for a C++ lexical analyser the default extension is .cpp.

Be ware that the extension appended to a file name can govern the way in which it is compiled.  For instance, in Visual C++ files with the extensions .cpp and .cxx are compiled with the C++ compiler, whereas files with the extension .c are compiled with the C compiler.


Provides a space for you to specify the extension to be appended to the include file name.  The default file name extension is always .h, no matter what Target Language is selected.


ALex Options Dialog Box Table Compression Category


If selected, no DFA state minimisation is performed.  This can lead to the faster compilation of Lex source files, however in most cases it also leads to a subsequent increase in the size of the resultant lexical analyser.


If selected, default states are not used to compress the lexical analyser's state transition table.  This will typically result in a lexical analyser that is faster, but larger in size.  The default state compression technique makes use of similarities between states.  If two states have similar transitions, then it is possible to remove the transitions from one state and use those in the other state (the default state).


If selected, no error transitions are introduced when finding default states.  This will typically result in a lexical analyser that is slightly faster, but also slightly larger in size.  This option will have no effect if the No Default States option is selected.  Error transitions allow states to choose from a wider range of states when looking for default states.  They do this by allowing states to introduce error transitions to themselves, in order to make potential default states fit.  For instance, a state may find a preferable default state which has maybe one or two character transitions which are not present in the state.  The state can then introduce a couple of error transitions to cover these cases.  When the lexical analyser is running, these error transitions prevent the lexical analyser from looking at the corresponding transitions in the default state.


If selected, no default transitions are introduced for states.  This will typically result in a lexical analyser that is slightly faster, but also larger in size.  Default transitions are introduced if a state has a large number of transitions going to the same state.  In which case, these transitions can be removed and replaced by a single default transition.  Note that the introduction of a default transition for a state can also lead to the introduction of error transitions.  That is, if the state does not have a transition for every character, then an error transition must be introduced for the ones that aren't.  These simply prevent the lexical analyser from using the default transition.


ALex Options Dialog Box #line Directives Category


If selected, specifies that short file names will be used in the generated output and include files.  In particular this option affects any #include and #line directives that are output to the files.  This option is primarily intended to provide support for 16-bit compilers, or those that cannot handle long file names.  Short file names are those that are limited to 8 file name characters and 3 file extension characters.  These file names are also sometimes called 8.3 file names.


If selected, no #line directives are generated in the output file.  The #line directives enable the C/C++ compiler to map from the C/C++ file back to the Lex source file.  This is useful for correcting compilation errors.  Sometimes, however it is better if these mappings did not take place.


If selected, #line directive file names are not treated as strings.  Instead they will be treated as file names, similar to those used in #include directives.  The Microsoft Visual C++ compiler treats #line directive file names as strings.  The Borland C++ compiler on the other hand treats #line directive file names as proper file names.  The important point about strings is that they have an escape character '\'.  Unfortunately, in Windows this is the directory partitioning character.  Taking an example, the string "c:\\myproj\\mylexer.l" is equivalent to the file name c:\myproj\mylexer.l.


Editor Options Dialog Box


Provides a space for you to specify the number of spaces which correspond to a tab stop.  A tab stop can be from 1 to 62 spaces in size.


If selected, specifies that the tab stop character is kept when the tab key is pressed.


If selected, specifies that an equivalent number of spaces are inserted when the tab key is pressed.  This option also has an effect on any files which are opened.  All tab stops in the file are converted into an equivalent number of spaces.  Note that these changes are only made permanent when the file is subsequently saved.


If selected, specifies that the vertical scroll bar is displayed.


If selected, specifies that the horizontal scroll bar is displayed.


If selected, specifies that the contents of the window tracks the scroll box as it is dragged.


If selected, specifies that all modified files are saved first before any tools are run.  If this option is not selected then the files held on disk will be used, not the ones held in memory.


If selected, specifies that a prompt is displayed when a file is about to be saved.  This allows you to confirm the action, or cancel it.


If selected, specifies that modified files are automatically reloaded.  These are files that have been modified by another application, for instance one of the build tools.  File modification is checked for when a build process finishes and when Parser Generator is made the active window.  If this option is not selected then you will be prompted before the file is reloaded.


Provides a space for you to specify the size of the undo buffer in 1K increments.  The buffer can be from 0 to 63K bytes in size.  A value of 0 means that there is no undo buffer i.e. no actions can be undone.  The larger the buffer the more actions that can be stored in it.


Closes this dialog box and saves any changes you have made.


Closes this dialog box without saving any changes you have made.


Tools Dialog Box


Lists the contents of the Tools menu.


Provides a space for you to specify the program associated with the selected tool.  This edit box is automatically updated to reflect the new tool whenever the tool selection changes.


Provides a space for you to specify the menu text for the selected tool.  To specify a keyboard shortcut, type an ampersand '&' before the character in question.  The character will then appear underlined in the Tools menu.  If no character is specified, then the shortcut is assumed to be the first character in the item,  This edit box is automatically updated to reflect the new tool whenever the tool selection changes.


Provides a space for you to specify the arguments to be passed to the selected tool.  It is also possible to use argument macros when specifying the arguments.  These allow the arguments to automatically change depending upon the context in which the tool is run.  The valid argument macros are $File, $FileName, $FileDir, $Proj, $ProjDir, $Line, $Col.  For more information see Argument Macros in the online help.  This edit box is automatically updated to reflect the new tool whenever the tool selection changes.


Provides a space for you to specify the initial directory where the selected tool is to be run from.  If this edit box is empty, then the tool is assumed to be run from the current working directory.  This edit box is automatically updated to reflect the new tool whenever the tool selection changes.


If selected, specifies that you will be prompted to enter arguments for the tool when it is run.  This check box is automatically updated to reflect the new tool whenever the tool selection changes.


Opens a dialog box where you can add a tool to the Tools menu.  The new tool is added to the bottom of the menu, and the Command line and Menu text edit boxes are initialised automatically.


Deletes the specified tool from the Tools menu.


Moves the specified tool up one in the Tools menu.


Moves the specified tool down one in the Tools menu.


Closes this dialog box and saves any changes you have made.


Closes this dialog box without saving any changes you have made.


Tool Arguments Dialog Box


Provides a space for you to specify the arguments to be passed to the tool.  It is also possible to use argument macros when specifying the arguments.  These allow the arguments to automatically change depending upon the context in which the tool is run.  The valid argument macros are $File, $FileName, $FileDir, $Proj, $ProjDir, $Line, $Col.  For more information see Argument Macros in the online help.


Closes this dialog box running the tool with the arguments specified.


Closes this dialog box without running the tool.


Colour Dialog Box


Displays a list of items for which you can specify the foreground and background colours.


Displays a sample of the selected item as it would appear in the application.


Restores the selected item's foreground and background colours to their default values.


Restores every item's foreground and background colours to their default values.


If selected, YACC syntax colouring items are displayed.  This option also specifies whether YACC syntax colouring occurs in the application.


If selected, Lex syntax colouring items are displayed.  This option also specifies whether Lex syntax colouring occurs in the application.


If selected, C/C++ syntax colouring items are displayed.  This option also specifies whether C/C++ syntax colouring occurs in the application.  This applies to C/C++ code embedded in YACC and Lex files as well.


Sets the selected item's foreground colour to black.


Sets the selected item's foreground colour to white.


Sets the selected item's foreground colour to red.


Sets the selected item's foreground colour to lime.


Sets the selected item's foreground colour to blue.


Sets the selected item's foreground colour to yellow.


Sets the selected item's foreground colour to fuchsia.


Sets the selected item's foreground colour to aqua.


Sets the selected item's foreground colour to grey.


Sets the selected item's foreground colour to silver.


Sets the selected item's foreground colour to maroon.


Sets the selected item's foreground colour to green.


Sets the selected item's foreground colour to navy.


Sets the selected item's foreground colour to olive.


Sets the selected item's foreground colour to purple.


Sets the selected item's foreground colour to teal.


Sets the selected item's background colour to black.


Sets the selected item's background colour to white.


Sets the selected item's background colour to red.


Sets the selected item's background colour to lime.


Sets the selected item's background colour to blue.


Sets the selected item's background colour to yellow.


Sets the selected item's background colour to fuchsia.


Sets the selected item's background colour to aqua.


Sets the selected item's background colour to grey.


Sets the selected item's background colour to silver.


Sets the selected item's background colour to maroon.


Sets the selected item's background colour to green.


Sets the selected item's background colour to navy.


Sets the selected item's background colour to olive.


Sets the selected item's background colour to purple.


Sets the selected item's background colour to teal.


Closes this dialog box and saves any changes you have made.


Closes this dialog box without saving any changes you have made.


Setup Visual C++ Version 4.00

Carry out the following steps to enable Microsoft Visual C++ Version 4.00 to compile and link Parser Generator created files.

Directory Settings

You will need to perform the following steps in Microsoft Visual C++ Version 4.00.  You will only need to do them once.

1	Choose the Options command from the Tools menu.  This should display the Options dialog box.

2	Click the Directories tab.

3	Click Include Files in the Show Directories For box.

4	In the Directories box click the last empty directory.  Type the name of the directory where you installed Parser Generator, followed by "\include".  For instance, if you installed Parser Generator in "C:\Program Files\Parser Generator", then you would type "C:\Program Files\Parser Generator\include".  Note though that case is not important when specifying directory names.

5	Click Library Files in the Show Directories For box.

6	In the Directories box select the last empty directory.  Type the name of the directory where you installed Parser Generator, followed by "\lib\msdev".  For instance, if you installed Parser Generator in "C:\Program Files\Parser Generator", then you would type "C:\Program Files\Parser Generator\lib\msdev".  Note though that case is not important when specifying directory names.

7	Click Source Files in the Show Directories For box.

8	In the Directories box select the last empty directory.  Type the name of the directory where you installed Parser Generator, followed by "\source".  For instance, if you installed Parser Generator in "C:\Program Files\Parser Generator", then you would type "C:\Program Files\Parser Generator\source".  Note though that case is not important when specifying directory names.

9	Click the OK button.  The Options dialog box should close accepting your changes.

Visual C++ should now be able to find the include files yacc.h and lex.h and the YACC and Lex library files.

Project Settings

For each Visual C++ project that you create you will need to perform the following steps in Microsoft Visual C++ Version 4.00.

1	Choose the Settings command from the Build menu.  This should display the Project Settings dialog box.

2	Click Win32 Debug in the Settings For box.

3	Click the C/C++ tab.

4	Click General in the Category box.

5	In the Preprocessor Definitions box type, at the end of the current text, ",YYDEBUG".

6	Click the Link tab.

7	Click General in the Category box.

8	In the Object/Library Modules box type, at the end of the current text, " library", where library depends on the run-time library that is selected for the project configuration.

Run-time Library	Library

Debug Single-Threaded	yld.lib

Debug Multithreaded	ylmtd.lib

Debug Multithreaded DLL	ylmtdlld.lib

Typically Single-Threaded libraries are used in Console Applications and Multithreaded libraries are used in Windows applications.  If you use MFC in a shared DLL, then you will need to use the Multithreaded DLL library.  Beware when using this library.  The streams yyin, yyout, yylexerr, yylexdebugout, yyparseerr and yyparsedebugout are initialised to NULL.  In order to assign them to their default values you will need to call the functions yylexinit and yyparseinit.  The yylexinit function initialises the lexical analyser variables, that is yyin, yyout, yylexerr and yylexdebugout, and the yyparseinit function initialises the parser variables yyparseerr and yyparsedebugout.  Note that this applies to single model C parsers and lexical analysers.  The equivalent multiple model parser and lexical analyser functions are yymlexinit and yymparseinit.  Instead of initialising a variable, they initialise the relevant member of the yymparse_t or yymlex_t structure.  There are no initialisation functions for C++ parsers and lexical analysers, since the relevant member variables are able to be initialised correctly in the constructor.  Alternatively, you can initialise the streams yourself.  To check the Run-time library that is used by your application you will need to click the C/C++ tab, and then click Code Generation in the Category box.

9	Click Win32 Release in the Settings For box.

10	In the Object/Library Modules box type, at the end of the current text, " library", where library depends on the run-time library that is selected for the project configuration.

Run-time Library	Library

Single-Threaded	yl.lib

Multithreaded	ylmt.lib

Multithreaded DLL	ylmtdll.lib

Typically Single-Threaded libraries are used in Console Applications and Multithreaded libraries are used in Windows applications.  If you use MFC in a shared DLL, then you will need to use the Multithreaded DLL library.  Beware when using this library.  A number of streams used by, or in the library are initialised to NULL.  For information on how to assign them their default values see step 8 above.  To check the Run-time library that is used by your application you will need to click the C/C++ tab, and then click Code Generation in the Category box.

11	Click the OK button.  The Project Settings dialog box should close accepting your changes.

Visual C++ should now be able to import the required YACC and Lex functions and variables from the specified libraries.

Note

Remember to include Parser Generator created files in your Visual C++ project.  For instance, if you have compiled the YACC file "myparser.y", then the resultant output file "myparser.c" (if a C parser is being generated) should be included in your Visual C++ project.  With Visual C++ version 5.00 and higher, if you have generated an include file, then you should include this as well.  Failure to include these files usually leads to the generation of a number of unresolved symbol errors by the linker.

A final note, is beware of what Visual C++ project you create to include your Parser Generator created files in.  YACC and Lex are commonly used to create command line (or MS-DOS like) applications.  In the 32-bit environment these applications are termed Console applications.  They are different to other Windows (GUI) applications.  An indication that you have used the wrong application type is if you get a WinMain unresolved symbol error from the linker.


Setup Visual C++ Version 1.52

Carry out the following steps to enable Microsoft Visual C++ Version 1.52 to compile and link Parser Generator created files.

Directory Settings

You will need to perform the following steps in Microsoft Visual C++ Version 1.52.  You will only need to do them once.

1	Choose the Directories command from the Options menu.  This should display the Directories dialog box.

2	In the Include Files Path box type, at the end of the current text, a semicolon ';', followed by the name of the directory where you installed Parser Generator, followed by "\include". For instance, if you installed Parser Generator in "C:\Program Files\Parser Generator", then you would type something like ";C:\PROGRA~1\PARSER~1\include".  Note though that case is not important when specifying directory names.  More importantly however, you must specify the directory as a short path name.  Visual C++ version 1.52 is a 16-bit application and cannot handle long path names.  You can get the short path name for a directory, or indeed any file, from Windows Explorer.  Simply right click the directory in question and select the Properties command.  In the displayed dialog box, the short path name is specified by the MS-DOS name.

3	In the Library Files Path box type, at the end of the current text, a semicolon ';', followed by the name of the directory where you installed Parser Generator, followed by "\lib\msvc". For instance, if you installed Parser Generator in "C:\Program Files\Parser Generator", then you would type something like ";C:\PROGRA~1\PARSER~1\lib\msvc".  Note though that case is not important when specifying directory names.  More importantly however, you must specify the directory as a short path name.  Visual C++ version 1.52 is a 16-bit application and cannot handle long path names.  You can get the short path name for a directory, or indeed any file, from Windows Explorer.  Simply right click the directory in question and select the Properties command.  In the displayed dialog box, the short path name is specified by the MS-DOS name.

4	Click the OK button.  The Directories dialog box should close accepting your changes.

Visual C++ should now be able to find the include files yacc.h and lex.h and the YACC and Lex library files.

Project Settings

For each Visual C++ project that you create you will need to perform the following steps in Microsoft Visual C++ Version 1.52.

1	Choose the Project command from the Options menu.  This should display the Project Options dialog box.

2	Click the Compiler button.  This should display the C/C++ Compiler Options dialog box.

3	Click the Debug Specific button.

4	Click Preprocessor in the Category box.

5	In the Symbols And Macros To Define box type, at the end of the current text, ", YYDEBUG".

6	Click the OK button.  The C/C++ Compiler Options dialog box should close accepting your changes.

7	Click the Linker button.  This should display the Linker Options dialog box.

8	Click the Debug Specific button.

9	In the Libraries box type, at the end of the current text, ", library", where library depends on the memory model that is selected for the project.  Note that it is not essential to match the library to the memory model, and in general the large memory model library will suffice.

Memory Model	Library

Tiny	tyld.lib

Small	syld.lib

Medium	myld.lib

Compact	cyld.lib

Large	lyld.lib

Huge	hyld.lib

10	Click the Release Specific button.

11	In Libraries box type, at the end of the current text, ", library", where library depends on the memory model that is selected for the project.

Memory Model	Library

Tiny	tyl.lib

Small	syl.lib

Medium	myl.lib

Compact	cyl.lib

Large	lyl.lib

Huge	hyl.lib

12	If you are going to redefine YACC and Lex library functions in your source files then you will need to perform the following steps.  This should add the /NOE option to the linker command line.  You need to do this, if not the linker will come up with an error.

1	Click the Common to Both box.

2	Click the Prevent Use of Extended Dictionary box if it is not already checked.

13	Click the OK button.  The Linker Options dialog box should close accepting your changes.

14	Click the OK button.  The Project Options dialog box should close accepting your changes.

Visual C++ should now be able to import the required YACC and Lex functions and variables from the specified libraries.

Note

Remember to include Parser Generator created files in your Visual C++ project.  For instance, if you have compiled the YACC file "myparser.y", then the resultant output file "myparser.c" (if a C parser is being generated) should be included in your Visual C++ project.  Failure to include these files usually leads to the generation of a number of unresolved symbol errors by the linker.


Setup Borland C++Builder Version 3.0

Carry out the following steps to enable Borland C++Builder to compile and link Parser Generator created files.

Setting up the Project

For each C++Builder project which uses a parser or lexical analyser created by Parser Generator you will need to perform the following tasks:

1	Specify the location of the YACC and Lex library header and source files.

2	Define YYDEBUG if the project is a debug project.

3	Include the relevant YACC and Lex library in the project.

The following steps carry out these tasks.  Note that they assume that the Project Manager window is visible.  If it isn't then you can display it by selecting the Project Manager command from the View menu.

1	Right click the relevant project in the Project Manager window.

2	Click the Options command in the pop-up menu.  This should display the Project Options dialog box.

3	Click the Directories/Conditionals tab.

4	In the Include Path box on the right hand side type, at the end of the current text, a semicolon ';', followed by the name of the directory where you installed Parser Generator, followed by "\include".  For instance, if you installed Parser Generator in "C:\Program Files\Parser Generator", then you would type ";C:\Program Files\Parser Generator\include".  Note though that case is not important when specifying directory names.

4	In the Debug Source Path box on the right hand side type, at the end of the current text, a semicolon ';', followed by the name of the directory where you installed Parser Generator, followed by "\source".  For instance, if you installed Parser Generator in "C:\Program Files\Parser Generator", then you would type ";C:\Program Files\Parser Generator\source".  Note though that case is not important when specifying directory names.

5	If you are building a debug version of your application, then in the Conditional Defines box on the right hand side type, at the end of the current text ";YYDEBUG".  If there is no text in the box then the semicolon ';' should be omitted.

Note

It is also worth remembering, at this point, what other conditionals are specified here.  They will be used later on to determine what YACC and Lex library to include in the project.

6	Click the OK button.  The dialog box should close accepting your changes.

7	Again, right click the relevant project in the Project Manager window.

8	This time though, click the Add command in the pop-up menu.  This should display the Add To Project dialog box.

9	Click "Library file (*.lib)" in the Files Of Type box.

10	In the dialog box locate the Borland C++Builder YACC and Lex libraries.  These are found in the "lib\builder" sub-directory off of the main Parser Generator directory.  For instance, if you installed Parser Generator in the "C:\Program Files\Parser Generator" directory, then this will be "C:\Program Files\Parser Generator\lib\builder".  A list of libraries should appear in the dialog box.

11	Click the YACC and Lex library that you need to include in your project.

Library	Conditionals	Description

yl.lib	Single-threaded library.

yld.lib	YYDEBUG	Single-threaded debug library.

ylmt.lib	_MT	Multithreaded library.

ylmtd.lib	_MT;YYDEBUG	Multithreaded debug library.

ylmtdll.lib	_MT;_RTLDLL	Multithreaded library with the run-time library residing in a DLL.

ylmtdlld.lib	_MT;_RTLDLL;YYDEBUG	Multithreaded debug library with the run-time library residing in a DLL.

Debug variants of the library have the extension "d".

You can determine what YACC and Lex library you need to include in your project by the conditionals that have been specified for it.  If you have not remembered them from step 6, then you will need to display the Project Options dialog box again.  Note that YYDEBUG is always specified by yourself.

Beware when using the ylmtdll.lib and ylmtdlld.lib libraries.  The streams yyin, yyout, yylexerr, yylexdebugout, yyparseerr and yyparsedebugout are initialised to NULL.  In order to assign them to their default values you will need to call the functions yylexinit and yyparseinit.  The yylexinit function initialises the lexical analyser variables, that is yyin, yyout, yylexerr and yylexdebugout, and the yyparseinit function initialises the parser variables yyparseerr and yyparsedebugout.  Note that this applies to single model C parsers and lexical analysers.  The equivalent multiple model parser and lexical analyser functions are yymlexinit and yymparseinit.  Instead of initialising a variable, they initialise the relevant member of the yymparse_t or yymlex_t structure.  There are no initialisation functions for C++ parsers and lexical analysers, since the relevant member variables are able to be initialised correctly in the constructor.  Alternatively, you can initialise the streams yourself.

12	Click the OK button.  The dialog box should close adding the library to the project.

You should now be able compile and link Parser Generator created files using Borland C++Builder.

It is not strictly necessary to carry out the above steps through the Project Manager window.  If you have only one project in your Project Group, or the relevant project is the active project (indicated by the project name appearing in the title bar, and in a bold font in the Project Manager window), then you can use the Project menu instead.  The Project Manager pop-up menu commands and their corresponding Project menu commands are shown below.

Pop-up Command	Project Menu Equivalent Command

Options	Options

Add	Add To Project

Note

Remember to include Parser Generator created files in your C++Builder project.  For instance, if you have compiled the YACC file "myparser.y", then the resultant output file "myparser.c" (if a C parser is being generated) should be included in your C++Builder project.  Failure to include these files usually leads to the generation of a number of unresolved symbol errors by the linker.

A final note, is beware of what C++Builder project you create to include your Parser Generator created files in.  YACC and Lex are commonly used to create command line (or MS-DOS like) applications.  In the 32-bit environment these applications are termed Console applications.  They are different to other Windows (GUI) applications.  An indication that you have used the wrong application type is if you get a WinMain unresolved symbol error from the linker.


Setup Borland C++ Version 4.52

Carry out the following steps to enable Borland C++ Version 4.52 to compile and link Parser Generator created files.

Building the YACC and Lex Libraries

Parser Generator does not come with any pre-built Borland YACC and Lex libraries.  In order to use a parser or lexical analyser created by Parser Generator you must also build the relevant library.  There is an alternative to building a library, and that is to include the library source files directly into your project.  It is however preferable to build a library.  To build a Borland YACC and Lex library you will need to perform the following steps.  Note that they assume that Borland C++ is already up and running and is the active application:

1	Choose the Open Project command from the Project menu.  This should display the Open Project File dialog box.

2	In the Directories box locate the Borland YACC and Lex Library Project File.  This is found in the "borland" sub-directory off of the main Parser Generator directory.  If you installed Parser Generator in the "C:\Program Files\Parser Generator" directory, then this will be in "C:\Program Files\Parser Generator\borland".  Note though that Borland C++ uses short path names, so this is more likely to appear as "C:\PROGRA~1\PARSER~1\borland".  The Project File "yllib.ide" should appear in the File Name box.  Note that the Project File uses relative paths, so it doesn't matter where Parser Generator has been installed.

3	Click the Project File "yllib.ide" in the File Name box.

4	Click the OK button.  The dialog box should close and the yllib.ide project should be opened, displaying the Project window.

5	The project that is now loaded assumes that the Borland C++ compiler has been installed in the "c:\bc45" directory.  If you have installed Borland C++ Version 5, or did not install Borland C++ Version 4.52 in the default directory then you will need to perform the following steps:

1	Choose the Project command from the Options menu.  This should display the Project Options dialog box.

2	Click the Directories topic in the Topics box.

3	In Include box on the right hand side, over-type "c:\bc45" with the directory where Borland C++ is installed.  If you installed Borland C++ in the default directory then this will be "c:\bc5".

4	You can also do the same with the Library box, although it is not necessary.

6	Right click the library you wish to build.  A descriptive name appears next to each library.  Match the description given to the application you are intending to build.  If you are building a Win32 Console application then you can use the Win32 GUI libraries instead.  There are no DOS (Overlay) libraries.  If you want to build one of these then you will need to create a new target.

7	Click the Build Node command in the pop-up menu.  This should display the Compile Status dialog box and commence compilation.

8	When compilation finishes click the OK button.  The Compile Status dialog box should disappear, and the Message window be displayed.  The built library will be located in the "lib\borland" sub-directory off of the main Parser Generator directory.

You may of course want to build more than one library at once.  For instance you may want to build the Release and Debug libraries for a particular model, or you may want to do the whole lot in one go.

There are couple of points worth mentioning about building YACC and Lex libraries:

1	When building libraries, use the Build Node rather than the Make Node command.  All of the object files are located in the same directory, so the Make Node command will use any object files that exist, rather than creating them again from scratch.

2	If you are building the Release version of a library then you will get two warning messages.  You can safely ignore these.  They are to do with parameters never being used in functions.  These errors do not occur in the Debug version because the parameters in question are wrapped in assertions.  These functions can typically be redefined by you to extend the functionality of the parsers and lexical analysers which use them.  In these cases the unused parameters may be useful.

Setting up the Project

For each Borland C++ project which uses a parser or lexical analyser created by Parser Generator you will need to perform the following tasks:

1	Specify the location of the YACC and Lex library header files.

2	Include the relevant YACC and Lex library in your targets.

3	Define YYDEBUG for any debug targets.

To specify the location of the YACC and Lex library header files you will need to perform the following steps.  The header file location is specified through a target.  If you have a SourcePool target then you will typically use this one.  Any changes made to this target will filter down to other targets which use it.  You will usually only need to perform these steps once for each project.

1	Right click the relevant target in the Project window.  If you have a SourcePool target then you will typically choose this one.  Any changes made to this target will filter down to other Standard targets.

2	Click the Edit Local Options in the pop-up menu.  This should display the Options dialog box.

3	Click the Directories topic in the Topics box.

4	In the Include box on the right hand side type, at the end of the current text, a semicolon ';', followed by the name of the directory where you installed Parser Generator, followed by "\include".  For instance, if you installed Parser Generator in "C:\Program Files\Parser Generator", then you would type something like ";C:\PROGRA~1\PARSER~1\include".  Note though that case is not important when specifying directory names.  More importantly however, you must specify the directory as a short path name.  Borland C++ version 4.52 cannot handle long path names.  You can get the short path name for a directory, or indeed and file, from Windows Explorer.  Simply right click the directory in question and select the Properties command.  In the displayed dialog box, the short path name is specified by the MS-DOS name.

5	Click the OK button.  The dialog box should close accepting your changes.

To include a YACC and Lex library in a target you will need to perform the following steps.  You will be more likely to add libraries to Standard targets rather that SourcePool targets.  For instance, you may want to add a Debug library to one target and a Release library to another one.  For this reason the steps below may be repeated a number of times for each project.

1	Right click the relevant target in the Project window.

2	Click the Add Node command in the pop-up menu.  This should display the Add To Project List dialog box.

3	Click "Libraries (*.lib)" in the List Files of Type.

4	In the Directories box locate the Borland YACC and Lex Libraries.  These are found in the "lib\borland" sub-directory off of the main Parser Generator directory.  If you installed Parser Generator in the "C:\Program Files\Parser Generator" directory, then this will be in "C:\Program Files\Parser Generator\lib\borland".  Note though that Borland C++ uses short path names, so this is more likely to appear as "C:\PROGRA~1\PARSER~1\lib\borland".  A List of libraries should appear in the File Name box.

5	Click the YACC and Lex library in the File Name box you wish to include in your project.  If there are no files listed, or the one you want is not there then you need to build the relevant library.  A list of the available libraries which can be built is shown below:

Library	Description

ylws.lib	Windows 3.x (16) Small

ylwm.lib	Windows 3.x (16) Medium

ylwc.lib	Windows 3.x (16) Compact

ylwl.lib	Windows 3.x (16) Large

ylw32.lib	Win32 GUI

ylw32mt.lib	Win32 GUI Multithread

ylt.lib	DOS (Standard) Tiny

yls.lib	DOS (Standard) Small

ylm.lib	DOS (Standard) Medium

ylc.lib	DOS (Standard) Compact

yll.lib	DOS (Standard) Large

ylh.lib	DOS (Standard) Huge

Debug variants of the library have the extension "d".  You can use the Win32 GUI libraries for Win32 Console applications.

Note that if you are using Borland C++ to build a DLL containing a parser or lexical analyser, then you cannot link one of the above libraries with the DLL.  If you do so, then the DLL will not work correctly.  In fact, it will most probable cause an access violation when you come to use the parser or lexical analyser that is contained within it.  This is because the libraries are intended for linking with an application (or .exe file).  In order to create a library which will link correctly with a DLL you can either use TargetExpert to change the Target Type of an existing target, or you can use the New Target command in the Project menu to add a new target to the project.

6	Click the OK button.  The dialog box should close accepting your changes.

If you have added a Debug version of a YACC and Lex library to a target, then you need to define YYDEBUG for that target.  To do this you will need to perform the following steps.

1	Right click the relevant target in the Project window.

2	Click the Edit Local Options command in the pop-up menu.  This should display the Options dialog box.

3	If the Compiler topic in the Topics box is collapsed, then double click it to expand it.

4	Click the Compiler sub-topic Defines in the Topics box.

5	In the Defines box on the right hand side, type "YYDEBUG".

6	Click the OK button.  The dialog box should close accepting your changes.

You should now be able compile and link Parser Generator created files using Borland C++.

Note

Remember to include Parser Generator created files in your Borland C++ project.  For instance, if you have compiled the YACC file "myparser.y", then the resultant output file "myparser.c" (if a C parser is being generated) should be included in your Borland C++ project.  Failure to include these files usually leads to the generation of a number of unresolved symbol errors by the linker.


Setup Other C/C++ Compilers

Parser Generator comes prepared to work with the Visual C++ and Borland C++ compilers.  This means that there are versions of the YACC and Lex library which can be used by the tools in question.  There are also statements in the library header files, namely yacc.h, lex.h and their multiple equivalents myacc.h and mlex.h which fit into the Visual C++ environment.  For instance, each function in the YACC and Lex library has a specifier YYCDECL.  If a Visual C++ compiler is being used, then this is defined as being __cdecl.  In any other circumstances it is defined as being nothing.

Most of the table driving code for the parser's and lexical analyser's generated by Parser Generator are contained in a library.  The source files for which are located in the source directory.  In order to get your compiler to work with Parser Generator it must get access to these files.  This can be done in one of two ways.  Both have their advantages and disadvantages:

1	Include the required library source files directly into your C/C++ project.  The simplest way is just to include the whole lot and then remove the files you don't need or want.  For instance, if you have created a lexical analyser with a redefined yywrap function, then you cannot include the library source file "yyswrap.c" in your project.  Doing so would result in the linker complaining of a duplicate symbol definition.  Incidentally, in case you hadn't guessed, "yyswrap.c" contains the default version of yywrap.

2	Use your compiler to create a (static) library project.  Then include all of the library source files in the project and build it.  Additionally, If you are building a debug version of the library then you must define YYDEBUG.  You can typically do this through a project or command line option.  The base name for the generated YACC and Lex library is "yl", with the appropriate library file name extension.  In most cases this is .lib.  If you are building a debug version of the library then suffix the name with "d" i.e. "yld".  If you are dealing with library models then use the following prefixes:

Model	Prefix

Tiny	t

Small	s

Medium	m

Compact	c

Large	l

Huge	h

The include files for the library are located in the include directory.  You will also need to tell your compiler where this directory is located.

The advantage of the first method is that it is quick and easy to get going.  However it can lead to functions (and variables) being included in your project when they don't really need to be.  This will cause your final executable to be larger than it otherwise would.  For long term usage, the second method is preferable.

Finally, once your libraries have been built (or not as the case may be), you will need to perform the following tasks in your C/C++ compiler.  There is one set of tasks which is performed once, and another set which is performed for each project which you create.  These tasks broadly follow those laid out in the Setup Visual C++ Version 4.00, Setup Visual C++ Version 1.52, Setup Borland C++Builder Version 3.0 and Setup Borland C++ Version 4.52 help topics.

You will need to perform the following tasks once in your C/C++ compiler:

1	Specify the location of the YACC and Lex library header files.  You will already have done this if you built the libraries for your compiler.

2	Specify the location of the YACC and Lex libraries used by your compiler.  If you have not built any libraries, and are going to include the library source files directly, then you do not need to do this.

Every time you create a new C/C++ project with your compiler you will need to perform the following tasks.  You will only need to do them if you are including a parser or lexical analyser created by Parser Generator.  Remember that you must also include any Parser Generator created files in your C/C++ project too.  i.e. if you are compiling the YACC file "myfile.y", then you must include "myfile.c", or "myfile.cpp" if it is given a .cpp extension, in your C/C++ project.

1	Define YYDEBUG for the debug version of the project.

2	Either link the project with the relevant YACC and Lex library, or if you have not created any libraries, include the relevant library source files in the project.

See Also

Setup Visual C++ Version 4.00, Setup Visual C++ Version 1.52, Setup Borland C++Builder Version 3.0, Setup Borland C++ Version 4.52


Solve Common Linker Problems

There are a couple of common problems which crop up when using Parser Generator.  These occur at the linking phase when the C/C++ compiler is building the application containing the parser/lexical analyser.

1	There are a large number of unresolved symbol linker errors.

There are three possible causes for this problem.

1	You have not included the generated parser/lexical analyser in your C/C++ project.  Typically, if you compile a YACC or Lex file using Parser Generator you will get a C or C++ file as output.  You need to include this file in your project.  For example if you compiled the YACC file "myfile.y", then the (default) output file "myfile.c" should be included in your C/C++ project.

2	You are not linking the YACC and Lex library with your project.  The library contains most of the table driving code required by the parser/lexical analyser.  The library to be linked depends upon what compiler you are using and the version of the project you are building (debug or release).  In order to specify the correct library for your project see Setup Visual C++ Version 4.00, Setup Visual C++ Version 1.52, Setup Borland C++Builder Version 3.0, Setup Borland C++ Version 4.52 or Setup Other C/C++ Compilers.

3	You have created a small Lex based application and have not defined main().  Unlike other versions of YACC and Lex where you have two distinct libraries, one for YACC and the other for Lex, with Parser Generator you get just the one library.  As such, it is only possible to have one version of main() defined in the library.  By default this version calls the function yyparse, i.e. it is assuming that you have built a default (fast) single model C parser.  In order to fix the problem simply define main() in your Lex application and call the relevant function i.e. if you have built a default (fast) lexical analyser, then you would call yylex:

int main(void)

{

    return yylex();
}

In general though, if you are creating small YACC and Lex applications and you do not use a default (fast) single model C parser in the application, then you will need to implement a new version of main.

2	There are two unresolved symbol linker errors: yyrule, yysymbol.

This problem is caused by YYDEBUG being undefined when building a debug version of your C/C++ project.  The two symbols yyrule and yysymbol contain debugging information, which is used by the functions in the YACC and Lex library.  They are only defined if YYDEBUG itself is defined.  To see how to specify YYDEBUG for your project see Setup Visual C++ Version 4.00, Setup Visual C++ Version 1.52, Setup Borland C++Builder Version 3.0 or Setup Borland C++ Version 4.52.

3	There is an unresolved symbol linker error: WinMain.

This problem is caused, in most cases, by selecting the wrong project type when creating your C/C++ project.  For instance, YACC and Lex are commonly used to create command line (or MS-DOS like) applications.  In the 32-bit Windows environment these applications are termed Console applications.  They are different to other Windows (GUI) applications.  If you have selected a Windows application, when what you really wanted was a Console application, then you will get an unresolved WinMain symbol error by the linker.  The reason is that whilst a Console application, similar to a normal C/C++ program, has an entry point of main(), a Windows application has an entry point of WinMain() instead.


File Menu Commands

The File menu offers the following commands:

Command	Description

New	Creates a new document.

Open	Opens an existing document.

Close	Closes an opened document.

Save	Saves an opened document using the same file name.

Save As	Saves an opened document to a specified file name.

Save All	Saves all modified files.

Print	Prints a document.

Print Preview	Displays the document on the screen as it would appear printed.

Page Setup	Selects a printer and printer connection.

Send	Sends the active document through electronic mail.

Exit	Exits Parser Generator.


Edit Menu Commands

The Edit menu offers the following commands:

Command	Description

Undo	Reverse previous editing operation.

Redo	Redo previous editing operation.

Cut	Deletes data from the document and moves it to the clipboard.

Copy	Copies data from the document to the clipboard.

Paste	Pastes data from the clipboard into the document.

Delete	Deletes data from the document.

Select All	Selects the entire document.

Find	Find text in document.

Replace	Replace text in document.

Read Only	Toggles the read only status of the active document.


View Menu Commands

The View menu offers the following commands:

Command	Description

Line	Goes to the specified line.

Next Error	Goes to the line containing the next error generated by the build process.

Previous Error	Goes to the line containing the previous error generated by the build process.

Toggle Bookmark	Toggles a bookmark at the cursor position in the document.

Next Bookmark	Goes to the next bookmark in the document

Previous Bookmark	Goes to the previous bookmark in the document.

Clear All Bookmarks	Clears all bookmarks in the document.

Toolbar	Shows or hides the toolbar.

Status Bar	Shows or hides the status bar.

Syntax Colouring	Selects the syntax colouring to be applied to the document.


Project Menu Commands

The Project menu offers the following commands:

Command	Description

ParserWizard	Creates a new project and skeleton YACC and/or Lex files.

New	Creates a new project.

Open	Opens an existing project.

Edit	Changes the files that are in the open project.

Close	Closes the open project.

Compile File	Compiles the file using the appropriate AYACC or ALex tool.

Build	Compiles the files in the project where necessary according to the time stamp.

Rebuild All	Compiles all files in the project irrespective of time stamp.

Stop Build	Stops the build process.


Options Menu Commands

The Options menu offers the following commands:

Command	Description

Project	Sets the build options for the tools AYACC and ALex.

Editor	Sets the editor preferences

Tools	Sets the commands in the Tool menu.

Colour	Sets the window colours.

Font	Sets the windows font.


Window Menu Commands

The Window menu offers the following commands, which enable you to arrange multiple views of multiple documents in the application window:

Command	Description

New Window	Creates a new window that views the same document.

Split	Split the active window into panes.

Arrange Icons	Arranges icons of minimised windows.

Cascade	Arranges windows in an overlapped fashion.

Tile Horizontally	Arranges windows horizontally in non-overlapped tiles.

Tile Vertically	Arrange windows vertically in non-overlapped tiles.

Close All	Closes all open windows.

Output	Displays the Output window.

Window 1, 2, ...	Goes to specified window.


Help Menu Commands

The Help menu offers the following commands, which provide you assistance with this application:

Command	Description

Help Topics	Offers you an index to topics on which you can get help.

Parser Generator	Displays help on Parser Generator.

Build Tools	Displays help for the build tools.

YACC Language	Displays help on the YACC language.

Lex Language	Displays help on the Lex language.

License	Displays the license help topic.

About Parser Generator	Displays the version number of this application.


File New command
New Command (File Menu)

Use this command to create a new document in Parser Generator.

You can open an existing document with the Open command.

Shortcuts

Toolbar:

Keys:	CTRL+N


File Open command
Open Command (File Menu)

Use this command to open an existing document in a new window.  You can open multiple documents at once.  Use the Window menu to switch among the multiple open documents.  See Window 1, 2, ... command.

You can create new documents with the New command.

Shortcuts

Toolbar:

Keys:	CTRL+O


File Close command
Close Command (File Menu)

Use this command to close all windows containing the active document.  Parser Generator suggests that you save changes to your document before you close it.  If you close a document without saving, you lose all changes made since the last time you saved it.  Before closing an untitled document, Parser Generator displays the Save As dialog box and suggests that you name and save the document.

You can also close a document by using the Close icon on the document's window, as shown below:


File Save command
Save Command (File Menu)

Use this command to save the active document to its current name and directory.  When you save a document for the first time, Parser Generator displays the Save As dialog box so you can name your document.  If you want to change the name and directory of an existing document before you save it, choose the Save As command.

Shortcuts

Toolbar:

Keys:	CTRL+S


File Save As command
Save As Command (File Menu)

Use this command to save and name the active document.  Parser Generator displays the Save As dialog box so you can name your document.

To save a document with its existing name and directory, use the Save command.


File Save All command
Save All Command (File Menu)

Use this command to save any modified documents.  If a document is being saved for the first time, Parser Generator displays the Save As dialog box so you can name it.  A modified document is marked with a star ‘*’ in the title bar.  Note that a new (empty) document is not deemed as modified and so therefore will not be saved.

Shortcuts

Toolbar:


Send Command (File Menu)

Use this command to send the active document through electronic mail.  This command presents a mail window with the active document attached to it.  You may then fill out the To: field, Subject: field, etc., and add text to the body of the message if you wish.  When you are finished you may click the "Send" button to send the message.


File 1, 2, 3, 4 command
File 1, 2, 3, 4 Command (File Menu)

Use the numbers and filenames listed at the bottom of the File menu to open the last four documents you closed.  Choose the number that corresponds with the document you want to open.


File Exit command
Exit Command (File Menu)

Use this command to end your Parser Generator session.  You can also use the Close command on the application Control menu.  Parser Generator prompts you to save documents with unsaved changes.

Shortcuts

Mouse:	Double-click the application's Control menu button.

Keys:	ALT+F4


Undo Command (Edit Menu)

Use this command to reverse the last editing action, if possible.  The number of actions that are undoable depends on the size of the undo buffer and the scale of the action.

Shortcuts

Toolbar:

Keys:	CTRL+Z or

ALT+BACKSPACE


Redo Command (Edit Menu)

Use this command to redo the last undone editing action.

Shortcuts

Toolbar:

Keys:	CTRL+Y


Cut Command (Edit Menu)

Use this command to remove the currently selected data from the document and put it on the clipboard.  This command is unavailable if there is no data currently selected.

Cutting data to the clipboard replaces the contents previously stored there.

Shortcuts

Toolbar:

Keys:	CTRL+X or

SHIFT+DEL


Copy Command (Edit Menu)

Use this command to copy selected data onto the clipboard.  This command is unavailable if there is no data currently selected.

Copying data to the clipboard replaces the contents previously stored there.

Shortcuts

Toolbar:

Keys:	CTRL+C or

CTRL+INS


Paste Command (Edit Menu)

Use this command to insert a copy of the clipboard contents at the insertion point.  This command is unavailable if the clipboard is empty.

Shortcuts

Toolbar:

Keys:	CTRL+V or

SHIFT+INS


Delete Command (Edit Menu)

Use this command to delete text.  If text is selected then this is deleted.  If not the character to the right of the cursor is deleted instead.  Note that the deleted text is not placed on the clipboard.

Shortcuts

Keys:	DEL


Select All Command (Edit Menu)

Use this command to select the entire contents of the active document.

Shortcuts

Keys:	CTRL+A


Find Command (Edit Menu)

Use this command to find some text in the document.  Parser Generator displays the Find dialog box in which you enter the search parameters.

Shortcuts

Keys:	CTRL+F


Replace Command (Edit Menu)

Use this command to find and replace text in the document.  Parser Generator displays the Replace dialog box in which you enter the search parameters.

Shortcuts

Keys:	CTRL+H


Read Only Command (Edit Menu)

Use this command to mark a document as being read only.  The read only status of the document is indicated by a check mark next to the command.  There is also a indicator in the status bar.  Note that the Output window is, by default read only.  This cannot be changed.


Line Command (View Menu)

Use this command to go to a line in the document.  Parser Generator displays the Line dialog box.

Shortcuts

Keys:	CTRL+G


Next Error Command (View Menu)

Use this command to go to the next error generated by the build process.  The source file containing the error is opened and the offending line is highlighted.  Note that the window is scrolled if necessary.  When the end of the error list has been reached the first error is gone back to.  Use the Previous Error command to correspondingly go to the previous error.

Shortcuts

Keys:	F4


Previous Error Command (View Menu)

Use this command to go to the previous error generated by the build process.  The source file containing the error is opened and the offending line is highlighted.  Note that the window is scrolled if necessary.  When the start of the error list has been reached the last error is gone back to.  Use the Next Error command to correspondingly go to the nex error.

Shortcuts

Keys:	SHIFT+F4


Toggle Bookmark Command (View Menu)

Use this command to toggle a bookmark at the cursor in the document.  Placed bookmarks can subsequently be navigated using the Next Bookmark command and Previous Bookmark command.

Shortcuts

Keys:	CTRL+F2


Next Bookmark Command (View Menu)

Use this command to go to the next bookmark in the document.

Shortcuts

Keys:	F2


Previous Bookmark Command (View Menu)

Use this command to go to the previous bookmark in the document.

Shortcuts

Keys:	SHIFT+F2


Clear All Bookmarks Command (View Menu)

Use this command to clear all bookmarks in the document.


View Toolbar command
Toolbar Command (View Menu)

Use this command to display and hide the Toolbar, which includes buttons for some of the most common commands in Parser Generator, such as File Open.  A check mark appears next to the menu item when the Toolbar is displayed.

See Toolbar for help on using the toolbar.


View Status Bar Command
Status Bar Command (View Menu)

Use this command to display and hide the Status Bar, which describes the action to be executed by the selected menu item or depressed toolbar button, and keyboard latch state. A check mark appears next to the menu item when the Status Bar is displayed.

See Status Bar for help on using the status bar.


Syntax Colouring Sub-Menu (View Menu)

The Syntax Colouring sub-menu offers the following commands:

Command	Description

YACC	Selects YACC syntax colouring for the document.

Lex	Selects Lex syntax colouring for the document.

C/C++	Selects C/C++ syntax colouring for the document.

None	Selects no syntax colouring for the document.


YACC Command (Syntax Colouring Sub-Menu)

Use this command to select YACC syntax colouring for the active document.  The command will have a check mark next to it if the document is already YACC syntax coloured.  Files with the extension .y or .yac that are opened are YACC syntax coloured by default. New files that are created are also YACC syntax coloured.  Note that this may change when the file is subsequently saved, at which point the syntax colouring applied is based upon the file's extension.  The colours used can be changed by selecting the Colour command from the Options menu.


Lex Command (Syntax Colouring Sub-Menu)

Use this command to select Lex syntax colouring for the active document.  The command will have a check mark next to it if the document is already Lex syntax coloured.  Files with the extension .l or .lex that are opened are Lex syntax coloured by default.  The colours used can be changed by selecting the Colour command from the Options menu.


C/C++ Command (Syntax Colouring Sub-Menu)

Use this command to select C/C++ syntax colouring for the active document.  The command will have a check mark next to it if the document is already C/C++ syntax coloured.  Files with the extension .c, .cpp, .cxx, .h, .hpp or .hxx that are opened are C/C++ syntax coloured by default.  The colours used can be changed by selecting the Colour command from the Options menu.


None Command (Syntax Colouring Sub-Menu)

Use this command to select no syntax colouring for the active document.  The command will have a check mark next to it if the document is already non-syntax coloured.


ParserWiazrd Command (Project Menu)

Use this command to create a new project.  The command displays the ParserWizard dialog box.  This dialog box allows more information about the project to be entered than the dialog box displayed by the New command.  This command also creates a skeleton YACC and/or Lex file as part of the project creation process and includes them in the project.


New Command (Project Menu)

Use this command to create a new project.  This command displays the New Project dialog box.


Open Command (Project Menu)

Use this command to open an existing project.


Edit Command (Project Menu)

Use this command to edit the current project.  This command displays the Edit Project dialog box.


Close Command (Project Menu)

Use this command to close the current project.


Compile File Command (Project Menu)

Use this command to compile the document.

Shortcuts

Toolbar:

Key:	CTRL+F8


Build Command (Project Menu)

Use this command to build the project.

Shortcuts

Toolbar:

Key:	SHIFT+F8


Rebuild All Command (Project Menu)

Use this command to re-build the project completely.

Shortcuts

Toolbar:

Key:	ALT+F8


Stop Build Command (Project Menu)

Use this command to stop the build process.

Shortcuts

Toolbar:


Project 1, 2, 3, 4 command
Project 1, 2, 3, 4 Command (Project Menu)

Use the numbers and filenames listed at the bottom of the Project menu to open the last four projects you closed.  Choose the number that corresponds with the project you want to open.


Project Command (Options Menu)

Use this command to change the project options.  This command displays the Project Options dialog box.


Editor Command (Options Menu)

Use this command to change the editor options.  This command displays the Editor Options dialog box.


Tools Command (Options Menu)

Use this command to change the Tool menu.  This command displays the Tools dialog box.


Colour Command (Options Menu)

Use this command to change the colours used in the editor and Output windows.  This command displays the Colour dialog box.


Font Command (Options Menu)

Use this command to change the font used in the editor and Output windows.


New Window Command (Window Menu)

Use this command to open a new window with the same contents as the active window.  You can open multiple document windows to display different parts or views of a document at the same time.  If you change the contents in one window, all other windows containing the same document reflect those changes.  When you open a new window, it becomes the active window and is displayed on top of all other open windows.


Split Command (Window Menu)

Use this command to split the active window into panes.  You may then use the mouse or the keyboard arrows to move the splitter bars.  When you are finished, press the mouse button or enter to leave the splitter bars in their new location.  Pressing escape keeps the splitter bars in their original location.


Arrange Icons Command (Window Menu)

Use this command to arrange the icons for minimised windows at the bottom of the main window.  If there is an open document window at the bottom of the main window, then some or all of the icons may not be visible because they will be underneath this document window.


Cascade Command (Window Menu)

Use this command to arrange multiple opened windows in an overlapped fashion.


Tile Horizontally Command (Window Menu)

Use this command to arrange multiple opened windows in a horizontal non-overlapped fashion.


Tile Vertically Command (Window Menu)

Use this command to arrange multiple opened windows in a vertical non-overlapped fashion.


Close All Command (Window Menu)

Use this command to close all opened windows in the application.  If you have modified a file then you will be prompted to save it before the window is closed.


Output Command (Window Menu)

Use this command to display the Output window.

Shortcuts

Key:	ALT+0


Window 1, 2, ... Command (Window Menu)

Parser Generator displays a list of currently open document windows at the bottom of the Window menu.  A check mark appears in front of the document name of the active window.  Choose a document from this list to make its window active.


Help Topics Command (Help Menu)

Use this command to display the opening screen of Help.  From the opening screen, you can jump to step-by-step instructions for using Parser Generator and various types of reference information.

Once you open Help, you can click the Contents button whenever you want to return to the opening screen.


About Parser Generator Command (Help Menu)

Use this command to display the copyright notice and version number of your copy of Parser Generator.


Find Box (Toolbar)

Type the text that you wish to search for in the active document.  The control stores the last 16 searches that you have made.  You can start the search by pressing the ENTER key.  This will perform a downward search.  To perform an upward search of the document hold the SHIFT key down while pressing the ENTER key.  Alternatively, you can choose the Find Next command and Find Previous command buttons which are situated on the toolbar.


Find Next Command (Toolbar)

Use this command to find the next occurrence of some text in the active document.  The text to be searched for is specified by the Find box on the toolbar.  To search up the document use the Find Previous command.

Shortcuts

Keys:	F3


Find Previous Command (Toolbar)

Use this command to find the previous occurrence of some text in the active document.  The text to be searched for is specified by the Find box on the toolbar.  To search down the document use the Find Next command.

Shortcuts

Keys:	SHIFT+F3


Help Using Help Command
Context Help Command

Use the Context Help command to obtain help on some portion of Parser Generator.  When you choose the command, the mouse pointer will change to an arrow and question mark.  Then click somewhere in the Parser Generator window, such as another Toolbar button.  The Help topic will be shown for the item you clicked.

Shortcuts

Keys:	SHIFT+F1


Toolbar

The toolbar is displayed across the top of the application window, below the menu bar.  The toolbar provides quick mouse access to many tools used in Parser Generator.

To hide or display the Toolbar, choose Toolbar from the View menu (ALT, V, T).

Click	To

Open a new document.

Open an existing document.  Parser Generator displays the Open dialog box, in which you can locate and open the desired file.

Save the active document with its current name.  If you have not named the document, Parser Generator displays the Save As dialog box.

Save all modified documents with their current name.  If you have not named a document, Parser Generator displays the Save As dialog box.

Print the active document.

Reverse the last editing.

Redo the previously undone editing action.

Remove selected data from the document and stores it on the clipboard.

Copy the selection to the clipboard.

Insert the contents of the clipboard at the insertion point.

The text you wish to find in the document.

Find the next occurrence of the text in the document.

Find the previous occurrence of the text in the document.

Compile the file.  The relevant tool will be used depending on the file extension.

Conditionally build the current project.

Rebuild the whole project.

Stop the build process.


Status Bar

The status bar is displayed at the bottom of the Parser Generator window.  To display or hide the status bar, use the Status Bar command in the View menu.

The left area of the status bar describes actions of menu items as you use the arrow keys to navigate through menus.  This area similarly shows messages that describe the actions of toolbar buttons as you depress them, before releasing them.  If after viewing the description of the toolbar button command you wish not to execute the command, then release the mouse button while the pointer is off the toolbar button.

The right areas of the status bar indicate which of the following keys are latched down:

Indicator	Description

OVR	The text overwrite mode is selected.

READ	The document is read-only.

CAP	The Caps Lock key is latched down.

NUM	The Num Lock key is latched down.

00000	The current line in the document

000	The current column in the document.


Editor Window

The editor window allows you to create and modify text files.

Moving the Cursor

It is possible to move the cursor around the editor window in a number of ways.  The simplest is to click the mouse over the point where you want to place it.  There are also a number of key combinations that can be used to move the cursor about whilst you are typing.  These are listed below:

Key Combination	Description

Left Arrow	Move the cursor left one character.

Right Arrow	Move the cursor right one character.

Up Arrow	Move the cursor up one line.

Down Arrow	Move the cursor down one line.

Home	Move the cursor to the start of the line.

End	Move the cursor to the end of the line.

Page Up	Scroll the window up one page.

Page Down	Scroll the window down one page.

Ctrl + Left Arrow	Move the cursor one word to the left.

Ctrl + Right Arrow	Move the cursor one word to the right.

Ctrl + Up Arrow	Scroll the window up one line.

Ctrl + Down Arrow	Scroll the window down one line.

Ctrl + Home	Move the cursor to the start of the file.

Ctrl + End	Move the cursor to the end of file.

Ctrl + Page Up	Scroll one window width to the right.

Ctrl + Page Down	Scroll one window width to the left.

Selecting Text

It is also possible to select text in a number of ways.  The simplest is to hold the left mouse button down, drag the mouse until the required text is selected, and then release the button.  Note that if you move the mouse outside the window then it will scroll automatically, allowing you to select large portions of text.  If you just want to select a word then you can simply double click on it.

Another way to select text is to use the Shift key. When the Shift key is held down one end of the selection is anchored at that point.  The other end can then be moved around using the key combinations described in Moving the Cursor.  Note that you can also use the Shift key with the mouse, just click the left mouse button over where you want selection to go.

Pressing the Escape key will clear any selection.

Entering Text

The typewriter keys on the keyboard are used to enter text into the file.  There are a couple of special keys which are also used for entering (or removing) characters.  These are:

Key Combination	Description

Enter	Inserts a new line in the file.

Tab	Inserts a tabulation character.

Delete	Deletes the character immediately to the right of the cursor.

Backspace	Deletes the character immediately to the left of the cursor.

Insert	Toggles the Insert/Overwrite mode.

Note that when any text is entered, the text that is currently selected is always deleted.

Indenting Text

When the enter key is pressed, and a new line is entered in the text file, it is automatically indented.  The indentation is always based on the indentation used in the previous line.  It is also possible to indent blocks of text, and to also un-indent them.  You first need to select some text.  Note that the selection must span two or more lines.  You then press the Tab key to indent the text, and Shift + Tab to un-indent it.


Output Window

The Output window displays the results of the build process.  The Output window is always marked as being read-only, and cannot be modified.  In all other respects you can use Output widow much as you would use an Editor window.

You can get help on a particular error by pressing the F1 key.  Note that the error must be the current selected error.  Errors can be navigated by using the Next Error command and Previous Error command in the View menu.


Argument Macros

Argument macros are useful for specifying arguments to pass to tools which depend on some type of context, for example the currently active document.  All argument macros start with the dollar character ’$’, and have a number of trailing letters.  Case is important.  It does not matter what characters follow a macro.  Note that the largest macro will always be matched, so the string "$FileName" will always match the macro $FileName and not $File.

$File

Specifies the complete path and file name of the active document. So, if the document "c:\progs\myproj\myfile.y" was active then the macro $File would be substituted with "c:\progs\myproj\myfile.y".  If there is no active document, or the document does not have a file name associated with it, then $File would be substituted with "".

$FileName

Specifies the name of the active document.  So, if the document "c:\progs\myproj\myfile.y" was active then the macro $FileName would be substituted with "myfile.y".  If there is no active document, or the document does not have a file name associated with it, then $File would be substituted with "".

$FileDir

Specifies the directory which contains the active document.  So, if the document "c:\progs\myproj\myfile.y" was active then the macro $FileDir would be substituted with "c:\progs\myproj\".  Note that the directory has a trailing back-slash ‘\’.  If there is no active document, or the document does not have a file name associated with it, then $FileDir would be substituted with "".

$Proj

Specifies the name of the currently open project.  The name does not include any file extension.  So, if the project "c:\progs\myproj\myproj.pgp" was open then the macro $Proj would be substituted with "myproj".  If there is no currently open project, then $Proj would be substituted with "".

$ProjDir

Specifies the directory which contains the currently open project.  So, if the project "c:\progs\myproj\myproj.pgp" was open then the macro $ProjDir would be substituted with "c:\progs\myproj\".  Note that the directory has a trailing back-slash ‘\’.  If there is no currently open project, then $ProjDir would be substituted with "".

$Line

Specifies the line number that the cursor is on in the active document.  Line numbers start at 1.  All numbers are specified in decimal notation.  If there is no active document, then $Line would be substituted with "".

$Col

Specifies the column that the cursor is on in the active document.  Columns start at 1.  All numbers are specified in decimal notation.  If there is no active document, then $Col would be substituted with "".


Title Bar

The title bar is located along the top of a window.  It contains the name of the application, the project and document.

To move the window, drag the title bar.  Note: You can also move dialog boxes by dragging their title bars.

A title bar may contain the following elements:

Application Control-menu button

Document Control-menu button

Maximize button

Minimize button

Name of the application

Name of the project

Name of the document

Restore button


Scroll Bars

Displayed at the right and bottom edges of the document window.  The scroll boxes inside the scroll bars indicate your vertical and horizontal location in the document.  You can use the mouse to scroll to other parts of the document.


Size Command (System Menu)

Use this command to display a four-headed arrow so you can size the active window with the arrow keys.

After the pointer changes to the four-headed arrow:

1	Press one of the DIRECTION keys (left, right, up, or down arrow key) to move the pointer to the border you want to move.

2	Press a DIRECTION key to move the border.

3	Press ENTER when the window is the size you want.

Note:  This command is unavailable if you maximize the window.

Shortcuts

Mouse:	Drag the size bars at the corners or edges of the window.


Move Command (Control Menu)

Use this command to display a four-headed arrow so you can move the active window or dialog box with the arrow keys.

Note:  This command is unavailable if you maximize the window.


System Minimize Command
Minimize Command (Application Control Menu)

Use this command to reduce the Parser Generator window to an icon.

Shortcuts

Mouse:	Click the minimize icon [image] on the title bar.


Maximize Command (System Menu)

Use this command to enlarge the active window to fill the available space.

Shortcuts

Mouse:	Click the maximize icon [image] on the title bar; or double-click the title bar.


Next Window Command (Document Control Menu)

Use this command to switch to the next open document window.  Parser Generator determines which window is next according to the order in which you opened the windows.

Shortcuts

Keys:	CTRL+F6


Previous Window Command (Document Control Menu)

Use this command to switch to the previous open document window.  Parser Generator determines which window is previous according to the order in which you opened the windows.

Shortcuts

Keys:	SHIFT+CTRL+F6


Close Command (Control Menus)

Use this command to close the active window or dialog box.

Double-clicking a Control-menu box is the same as choosing the Close command.

Note:  If you have multiple windows open for a single document, the Close command on the document Control menu closes only one window at a time.  You can close all windows at once with the Close command on the File menu.

Shortcuts

Keys: 	CTRL+F4 closes a document window

ALT+F4 closes the window or dialog box


Restore Command (Control Menu)

Use this command to return the active window to its size and position before you chose the Maximize or Minimize command.


Next Pane Command
Next Pane

<< Write application-specific help here. >>


Prev Pane Command
Prev Pane

<< Write application-specific help here. >>


No Help Available

No help is available for this area of the window.


No Help Available

No help is available for this message box.


YACC Language

Reference Topics

Introduction

Tokens

Grammar Summary

Grammar

Declarations Section

Declarations Section

Token Declaration

Start Declaration

Type Declaration

Union Declaration

Name Declaration

Include Declaration

Defines

Rules Section

Rules Section

Grammar Rule

Production

Grammar Symbol

Action Symbol

Precedence Symbol

Symbol Destructor Rule

Ambiguity Resolution

Error Recovery

Operator Precedence Grammars

Programs Section

Programs Section

Quick Reference Topics

Multiple C Parsers


YACC Introduction
Introduction

YACC is short for Yet Another Compiler Compiler.  A pun on the number of compiler, or parser, construction tools that were being created at the time.  It is a tool that, given a BNF (Backus-Naur Form) style specification of a grammar, can generate a corresponding parser.  It is worth noting that YACC will not accept every grammar presented to it.  Far from it.  However the class of grammars that it does accept is generally powerful enough for most programming needs.

YACC was originally written by S. C. Johnson on a UNIX platform.  It is closely tied in with Lex.  A lexical analyser generating tool.  Since then there have been many flavours of YACC implemented.  Perhaps the most notable being BISON and BYACC.

YACC generates what are termed LR parsers.  This means that they scan the input from left to right, the L bit, and produce a rightmost derivation from the bottom up, the R bit.  LR parsers are also called bottom-up parsers.  They are somewhat different to LL parsers.  Similar to LR parsers these also scan the input from left to right, but this time construct a leftmost derivation instead.  LL parsers are also called top down parsers.  They have the distinct advantage that they can generally be implemented by hand.  There are a number of techniques for doing this including predictive parsing and recursive descent parsing.  There are now also a number of tools which can construct LL parsers automatically.  Having said all this, it is generally a time consuming task to code a parser by hand, and an LR parser construction technique is inherently more powerful than an LL one.

For both types of parser, either LR or LL, there is generally an extra piece of information which specifies how many lookahead tokens the parser uses to decide what action to perform.  For instance an LR(1) parser uses one token of lookahead.  This an important point because YACC generates a parser which uses one token of lookahead as well.  That is the parser must decide, given the symbols that it has already seen, and the current lookahead token what it needs to do next.

There are three main LR parser construction methods that are available.  These are:

1	SLR(x)	Simple LR.  This is the least powerful parser construction method, although it will generate parsers for a number of grammars.  It does however have trouble with certain programming language constructs.

2	LALR(x)	Lookahead LR.  This parser construction method lies somewhere between the SLR and CLR construction methods in power.  It generates a parser with exactly the same number of states as an SLR parser, however it is able to generate parsers from a wider range of grammars.  This is the table construction technique that is commonly used by YACC.  Note that whilst YACC can typically only generate LALR parsers, AYACC can generate SLR and CLR parsers as well.

3	CLR(x)	Canonical LR.  This is the most powerful parser construction method.  It does however generate parsers which have a large number of states.  Where an LALR parser might have a few hundred states, an equivalent CLR parser would have several thousand.

Where x specifies the number of lookahead tokens.  In cases where x is equal to 1, the number is usually omitted.  Thus LALR is equivalent to LALR(1).  If an LR(x) parser construction method can construct an LR(x) parser for a grammar, then that grammar is said to be LR(x) grammar.

The parsers generated by YACC are specified in another programming language.  The original YACC generated output for C and RATFOR, a language which can be automatically translated to FORTRAN.  Currently, AYACC generates output for C and C++.  Note that whilst a C parser can be compiled in a C++ environment, the C++ parser itself makes use of classes.  There are other versions of YACC which generate output for other languages, including Ada, JAVA and Delphi etc.  The language that YACC generates output for is sometimes termed the target language.  Note that the actions associated with the rules, and any other code which is copied verbatim into the output file must also be in the target language.

One major difference between AYACC and YACC, and indeed other versions of YACC is that most of parser driving code is hidden in a library.  The generated output file contains the driving tables for the parser, definitions for the buffers, a wrapper function yyparseraction which contains the actions, and any other code that is copied from the source file to the output file.  The format of the generated output file looks like:

declarations

buffer definitions

int yyparseaction(int action)

{

actions

}

programs

parser driving tables

There are advantages and disadvantages to using a library to hold the parser driving code.  Placing the code in a library offers a considerable amount of flexibility, and means that you can tinker with how the parser itself behaves.  Whether you would want to do this or not is another matter.  Suffice to say that the opportunity is there if it is needed.  The major disadvantage to having the code in a library is the need to actually link the library in with the application you are developing.  This can be awkward if you are unfamiliar with the development environment, or you have to actually rebuild the libraries.  Library rebuilding is not particularly difficult these days, especially with the plethora of compiler development environments.  All that you generally need to do is create the relevant project, in this case a (static) library, add the library source files to it and perform a rebuild operation.


YACC Tokens
Tokens

The tokens recognised by YACC are listed below:

Keywords

Identifiers

Character Tokens

Integers

Tags

Actions

Attributes

Comments

Program Code

Declaration Code Block

Code Block

Other Tokens


YACC Keywords
Keywords

A YACC keyword starts with a percent sign % and has one or more trailing letters.  The percent sign is a special character in YACC and is referred to as the YACC escape character.  It is used to introduce a number of tokens.  The currently recognised keywords are:

%include, %left, %name, %nonassoc, %prec, %right, %start, %token, %type, %union

Compatibility

The case of the letters in a keyword is not important.  For instance %token, %Token and %TOKEN all specify the %token keyword.  There are also a number of synonyms for the YACC keywords:

Keyword	Synonym

%left	%<

%nonassoc	%2, %binary

%prec	%=

%right	%>

%token	%term, %0

Note that in most cases the synonyms have a trailing non-letter after the percent sign.  For the synonyms which do consist of letters, all upper and lower case letter combinations are possible.  The backslash character \ can also be used in place of the % character for all keywords.  For instance \token specifies the %token keyword.


YACC Identifiers
Identifiers

A YACC identifier starts with a letter or underscore and has zero or more trailing letters, digits and underscores.  Case is important, so the identifier “AA” is different to “aa”.  All characters in an identifier are significant.  The only limiting factor is the size of the line buffer used by the compiler.

Identifiers are used to represent nonterminal symbols.  They are also used, along with Character Tokens, to represent tokens, or terminal symbols.  The identifier error is predefined as a token by YACC and is used for implementing error recovery.


YACC Character Tokens
Character Tokens

A character token provides an alternative means of specifying a token in a rule.  It is a shorthand for defining tokens in the declarations section.  A character token is surrounded by single quotes and contains one or more characters.  Note, however that a character token will usually only contain one character.  Non-printable or special characters can be defined using an escape character.  In YACC, as in C this is the backslash \.  The list of valid escape sequences together with their meanings is shown below:

Name	Character	Escape

New-line	NL (LF)	\n

Horizontal tab	HT	\t

Vertical tab	VT	\v

Backspace	BS	\b

Carriage return	CR	\r

Form feed	FF	\f

Audible alert	BEL	\a

Backslash	\	\\

Question mark	?	\?

Single quote	'	\'

Double quote	"	\"

Octal number	ooo	\ooo

Hex number	hh	\xhh

An octal number consists of one, two or three octal digits (0 - 7).  A hex number consists of one or more hex digits (0 - 9, a - f, A - F).  A character token cannot cross a line boundary.  On order to specify the newline character, use the newline escape sequence \n.

If you specify a single character in a character token, then it will be assigned a token number.  This is taken as being the ASCII value of the character.  Beware of character tokens with more than one character.  They do not have a token number implicitly associated with them.  Instead they are treated like other (identifier) tokens.

Compatibility

Character tokens can also be surrounded by double quotes instead of single quotes.  The meaning is still the same.  It is still a character token, and not a string.


YACC Integers
Integers

There are three types of integers.  Octal, decimal and hexadecimal.  Octal integers start with a 0 and have zero or more trailing octal digits (0 - 7).  A decimal integer starts with any digit except zero (1 - 9) and has zero or more trailing digits (0 - 9).  A hexadecimal integer starts with either 0x or 0X and has one or more trailing hexadecimal digits (0 - 9, a - f, A - F).  Below are some examples of integers:

0       // zero

0713    // octal

99      // decimal

0x7f    // hexadecimal

Integers are used for defining the token numbers for tokens (terminal symbols).  They are stored internally as a 32-bit unsigned integer.

See Also

Token Declaration


YACC Tags
Tags

A tag is delimited by left and right angled brackets < and >.  The tag then contains a letter or underscore followed by zero or more letters, digits or underscores.  Case is important, so the tag <int> is different to <INT>.  All characters are significant.

Tags are used for associating types with nonterminals and tokens.

See Also

Token Declaration, Type Declaration


YACC Actions
Actions

An action is delimited by braces { and }.  Any characters inside are copied to the relevant point in the output file.  Note that there may be some processing first to cope with attributes.  An action is only recognised inside the Rules Section of a YACC source file.  In the Declarations Section a left bracket signifies the start of a Declaration Code Block.

An action is aware of certain C and C++ constructs.  In particular it is aware that the braces { and } are used for forming compound statements.  As such an action keeps track of the number of left and right braces that it has seen.  Only when the counts are the same, and the closing brace encountered, is the action considered complete.

An action is also aware of certain C and C++ tokens.  These are string constants, character constants and comments (both line and block).  For both string and character constants, the action is aware of the backslash escape character.  If any braces { and } are encountered inside any of these tokens then they are ignored i.e. they are not added to the running count, or treated as a terminating character.

Actions are executed whenever a rule is reduced, or alternatively, when a symbol is discarded by the parser during error recovery.  This can either be because a symbol has been popped off of the stack, or because the current lookahead token has been discarded.

From an action it is possible to access symbol attributes.  This is done by using Attribute tokens in the action.  For actions associated with rules, you can access the attributes of the symbols on both the left and right-hand-sides of the rule.  Note that when a rule is reduced, the symbols on the right-hand-side of the rule are popped off of the stack, and the nonterminal on the left-hand-side of the rule, along with its associated attribute, is pushed on it.  From this perspective, it is usual for rules to reference (i.e. not modify) attributes on the right-hand-side, and assign to attributes on the left.  For actions associated with symbol destructors, you can access the attribute of the symbol that is about to be discarded.  Symbol destructors are typically used for performing some form of clean up, such as deleting a syntax tree node.

In general though, it is possible for an action to access all attributes for symbols on the stack.  This has limited use for symbol destructor actions, however it does give certain advantages to actions associated with rules.  Namely, it allows inherited attributes to be implemented by YACC.  These allow information to be passed into rules, rather than out, as is normally the case.

Compatibility

An action can alternatively start with the equals sign =.  Such an action is terminated with a semicolon.  The action is also aware of string constants, character constants and comments, however in this case it is the semicolon character which is being ignored.  The action is not aware of braces { and } and their relevance in C and C++ for forming compound statements.

See Also

Declaration Code Block


Attributes

Attribute tokens are only recognised from within side Actions.  They are not recognised anywhere else.  They allow an action to make reference to symbol attributes.  This can either be for symbols that are already on the stack, for symbols that shortly will be, when a rule is reduced, or for symbols that are about to be discarded when the parser attempts to recover from a syntax error.  The meaning of attributes also vary slightly, depending upon the context in which they are used.  Naturally, this is dictated by the position of the enclosing action.  There are three main situations in which an action can be used:

1	As an action associated with a rule.  The action is performed whenever the rule is reduced.  That is the symbols on the right-hand-side of the rule are popped off of the stack, and the nonterminal symbol on the left-hand-side is pushed on it.

2	As an embedded action.  The action results in a (hidden) nonterminal symbol being pushed onto the stack.

3	As a symbol destructor action.  The action is performed whenever a grammar symbol associated with the action is discarded by the parser when it is attempting to recover from a syntax error.  This can either be because a symbol has been popped off of the stack, or the current lookahead token has been discarded.

In addition, attributes themselves can be split into two types.  Those for nonterminal symbols (and tokens in the case of symbol destructors) on the left-hand-side of rules, and those for symbols on the right-hand-side of rules.  In either case an attribute is always introduced with a dollar sign $.  Below are descriptions for the two types of attribute in relation to (normal) actions.  Following that, are descriptions highlighting the differences with embedded actions and symbol destructor actions.

The first, and simplest type of attribute, refers to the attribute for the nonterminal symbol on the left-hand-side of a rule.  That is, the nonterminal symbol which precedes the colon.  In this case, the attribute is represented by two dollar signs $$.  For example, the rule below implements the storage-class-specifier rule for a C compiler.  In each action, the required storage class specifier is assigned to the attribute for the nonterminal on the left-hand-side of the rule.

StorageClassSpecifier

    : AUTO      { $$ = SCS_AUTO; }

    | REGISTER  { $$ = SCS_REGISTER; }

    | STATIC    { $$ = SCS_STATIC; }

    | EXTERN    { $$ = SCS_EXTERN; }

    | TYPEDEF   { $$ = SCS_TYPEDEF; }

    ;

Where SCS_AUTO, SCS_REGISTER etc. are predefined constants.  Note that the assignment to the $$ attribute is not made permanent until the StorageClassSpecifier rule has been reduced.  This occurs after the action associated with the rule has been performed.  Normally this is all well and good, however there are a couple of action functions in the YACC and Lex Library which prevent this from happening.  Currently, the functions which do this are yyabort, yyaccept, yyforceerror and yythrowerror.  If you call one of these functions then any assignment to the $$ attribute will be lost.

The second, and slightly more complicated type of attribute, refers to the attributes for the symbols on the right-hand-side of a rule.  The symbols are numbered from left to right, starting at one, so the attribute for the first symbol is $1, the second $2, and so on.  It is also possible to have an optional plus sign + between the $ and the number.  So, for instance, it is possible to have $+1 and $+2 etc.  As an example, the rule below implements an operator precedence rule for the addition operator.  The action uses $1 and $3 to make reference to the values stored in the two sub-expressions on either side of the plus sign.  Adding the two values together, the result is then stored in $$, the attribute for the Expr nonterminal on the left-hand-side of the rule.  Although $2 refers to the attribute for the plus sign, operators, and other such symbols, do not normally need an attribute:

Expr: Expr '+' Expr { $$ = $1 + $3; } ;

Unlike the $$ attribute, all assignments to $n attributes are permanent.  The effect of assigning to a $n attribute takes place immediately.

With right-hand-side rule attributes, you are not just limited to accessing the right-hand-side symbols for that particular rule.  Since the attributes for the symbols are held on an attribute stack, you can therefore access any attributes that are on it.  It is just that when you step outside the attributes for the immediate right-hand-side, you have to be a little more careful.  Indeed, the ability to do this allows the implementation of inherited attributes.  Although these types of attribute are not used often in simple grammars, they become increasing important as grammars increase in complexity.

More fully, a right-hand-side attribute consists of a dollar sign $, followed by an optional plus + or minus - sign, followed by a decimal number.  Where a decimal number is said to consist of one or more decimal digits (0 - 9).  The attribute $1 always refers to the first symbol in the right-hand-side of a rule.  Note that it is possible for a right-hand-side to be empty, and consist of no symbols.  In which case $1 is meaningless in this context.  If the right-hand-side consists of m symbols, then $m refers to the last symbol in the rule.  This is also the topmost symbol on the stack.  Attributes below $1 i.e. at $0, $-1 etc. are referring to symbols on the stack below the symbols for the current rule.  Attributes above $m are referring to (non-existent) symbols on the stack above the current rule.  It is very rare that you will ever find cause to go above $m.  Furthermore, unless checks are made, it is also potentially dangerous since you could be accessing unallocated or invalid memory.  Taking the Expr example above, the state of the stack when the action is being evaluated is as follows:

Symbol	Attribute

Expr	$3	stack top

'+'	$2

Expr	$1

x	$0

y	$-1

…	…

z	$b	stack bottom

Where x, y and z are arbitrary grammar symbols, and b is the bottom most attribute number.   Note that although b is quite easily calculated, when actions make reference to symbols below the rule that is being reduced, it is generally to within one or two known symbols.

When a rule does not have an action associated with it, then it uses a default copy action.  This copies the attribute for the first symbol in the right-hand-side of the rule ($1) to the attribute on the left-hand-side of the rule ($$).  When there are no symbols on the right-hand-side of the rule, then no default copy action takes place.

For embedded actions, the meaning of attributes varies slightly.  Instead of $$ referring to the nonterminal symbol on the left-hand-side of the rule in which the action is embedded, it refers to the attribute for the (hidden) nonterminal which is pushed onto the stack when the embedded action rule is reduced.  The meaning of the $n attributes remain the same.  That is they refer to the symbols on the right-hand-side of the rule.  The following example shows how attributes vary for embedded actions and (normal) actions.

A:  B   C   { } D   { }

    $1  $2  $$	embedded action

$$  $1  $2  $3  $4	normal action

The second line shows the attributes accessible from the embedded action.  Note that the embedded action does not have access to A, or D.  The third line shows the attributes accessible from the (normal) action.  In this case it does have access to the attributes for A and D.  It also has access to the attribute for the embedded action ($3).

For symbol destructor actions, it is only really the $$ attribute that is typically used.  This refers either to the attribute for the grammar symbol that is about to be popped off of the stack, or the lookahead token which is about to be discarded.  It is possible to use the $n attributes, however whether you would want to, or could find a reason to, is another matter.  If such a case ever arises, the $0 attribute refers to the topmost symbol on the stack.  For grammar symbols that are about to be popped off of the stack, this is nominally equivalent to the $$ attribute, however there is a very subtle difference between the two which may not be apparent.  When a symbol destructor action is performed, the attribute for the grammar symbol which is being discarded is copied into a temporary location.  The is location of the $$ attribute.  Upon completion of the symbol destructor action, this attribute is then copied back again.  If, for any reason, a modification was made to the $0 attribute, then this modification would now be lost!  Otherwise, you can access and make modifications to $n attributes at will.  Note that in the case where lookahead tokens are discarded, yylval, which holds the attribute for the lookahead token is copied to and from the location used by $$ attribute.  As such, the stack is not altered in any by modifications to the $$ attribute.

For all attributes it is possible to specify a type for the attribute.  Usually it is not necessary to do this since the attribute type is implied by the symbol to which the attribute is referring.  For instance, if a nonterminal symbol Statement has been declared of type <node>, and an attribute makes reference to a Statement symbol then it knows that the attribute has a type <node>.

In those cases where the attribute type is not known, it can be specified by placing the equivalent of a Tag between the lead dollar $, and the following $ or attribute number.  For completeness, this is specified as a left angle bracket <, followed by a letter or underscore character _, followed by one or more letters, digits or underscore characters, and finally followed by a right angled bracket >.  For example $<node>$, $<symbol>1.

For embedded actions, the type of the (hidden) nonterminal symbol, and hence by association the $$ attribute is not known.  In these situations, the (hidden) nonterminal symbol assumes the same type as the nonterminal symbol on the left-hand-side of the rule in which the action is embedded.  For instance:

A: B { $$ = 1; } C { /* do something */ }

Here $$ in the embedded action has the same type as that of A.  Furthermore, the action for A, also knows that the $2 attribute, which makes reference to the embedded action, also has the same type as A.  In those circumstances where the embedded action does not have the same type as the nonterminal symbol on the left-hand-side, then it is necessary to resort to inserting type information in the attributes.  Alternatively, you can pre-type the embedded action by placing a tag immediately before the action.  For more information see also Action Symbol.

See Also

Actions, Tags, Action Symbol


YACC Comments
Comments

There are two ways to specify comments in AYACC.  One is block based the other line based.  A block comment starts with /* and ends with */.  Any characters in between, including the newline are considered as comment.  The line based comment starts with // and is terminated, naturally with a newline.  Note that comments are not copied to the output file.


YACC Program Code
Program Code

In the rules section, the character sequence %% indicates the start of program code.  Any characters between here and the end-of-file are copied into the output file.

Compatibility

The backslash character \ can be used in place of the % character.  As such \\, \% and %\ are synonyms for the %% sequence which indicates the start of program code.

See Also

Programs Section


YACC Declaration Code Block
Declaration Code Block

A declaration code block is delimited by braces { and }.  Any characters inside are copied to the relevant point in the output or include file.  The destination will depend upon the context in which the declaration code block is being used and whether or not an include file is being generated. A declaration code block is only recognised in the declarations section of the YACC source file.  In the rules section a left brace { signifies the start of an Action.

A declaration code block is aware of certain C and C++ constructs.  In particular it is aware that the braces { and } are used for forming compound statements and structures etc.  As such a declaration code block keeps track of the number of left and right braces that it has seen.  Only when the counts are the same, and the closing bracket encountered, is the declaration code block considered complete.

A declaration code block is also aware of certain C and C++ tokens.  These are string constants, character constants and comments (both line and block).  For both string and character constants, a declaration code block is aware of the backslash \ escape character.  If any braces { and } are encountered inside any of these tokens then they are ignored i.e. they are not added to the running count, or treated as a terminating character.

See Also

Union Declaration


YACC Block Code
Code Block

A code block is delimited by %{ and %}.  Any characters inside are copied to the relevant point in the output file.

%{

/* comment */

int variable;

void function(void);

%}

A code block is recognised in both the declarations and rules sections.  In the rules section, if the code block occurs before any action, then the characters are copied into the top of the yyparseaction function.  Remember that this function is called whenever a rule is reduced and an action is performed.  Otherwise the characters are copied after the last action that was seen.  Since each action is contained in a case statement, with a following break statement, any code copied after it will not be run.  There are ways around this, like using a goto statement, but this sort of defeats the object.

A code block is aware of certain C and C++ tokens.  These are string constants, character constants and comments (both line and block).  For both string and character constants, block code is aware of the backslash escape character.  If any %} character sequences are encountered inside any of these tokens then they are ignored.

Compatibility

In YACC, a code block can only be placed in the declarations section and at the start of the rules section.  Also, code placed at the start of the rules section is copied into the top of the generated yyparse function.   This is different to AYACC where it is copied into the top of the yyparseaction function.

The backslash character \ can be used in place of the % character.  As such \{, and \} are synonyms for the %{ and %} character sequences.

See Also

Defines


YACC Other Tokens
Other Tokens

The other tokens recognised by AYACC are:

%%  ,  :  ;  |

Compatibility

The backslash character \ can be used in place of the % character.  As such \\, \% and %\ are synonyms for the %% token.


YACC Grammar Summary
Grammar Summary

grammar :

declarations-section rules-section [programs-section]

Declarations Section

declarations-section :

[declaration-list]

declaration-list :

declaration-list declaration

declaration

declaration :

token-decl

start-decl

type-decl

union-decl

name-decl

include-decl

token-decl :

associativity [tag] init-token-list

associativity :

%left

%nonassoc

%right

%token

init-token-list :

init-token-list [,] init-token

init-token

init-token :

identifier [integer]

character-token [integer]

start-decl :

%start identifier

type-decl :

%type tag init-nonterminal-list

init-nonterminal-list :

init-nonterminal-list [,] init-nonterminal

init-nonterminal

init-nonterminal :

identifier

union-decl :

%union declaration-code-block

name-decl :

%name identifier [declaration-code-block] [declaration-code-block]

include-decl :

%include declaration-code-block

Rules Section

rules-section :

%% rule-list

rule-list :

rule-list rule

rule

rule :

grammar-rule

symbol-destructor-rule

grammar-rule :

nonterminal : production-list ;

nonterminal :

identifier

production-list :

production-list | production

production

production :

[symbol-list]

symbol-list :

symbol-list symbol

symbol

symbol :

grammar-symbol

action-symbol

precedence-symbol

grammar-symbol :

identifier

character-token

action-symbol :

[tag] action [action-destructor]

action-destructor :

~ action

precedence-symbol :

%prec token

token :

identifier

character-token

symbol-destructor-rule :

symbol-destructor-list : action ;

symbol-destructor-list :

symbol-destructor-list | symbol-destructor

symbol-destructor

symbol-destructor :

~ grammar-symbol

Programs Section

programs-section :

program-code

Compatibility

Rules must now always be terminated with a semicolon.  If not then a syntax error will be generated by the compiler.


YACC Grammar
Grammar

Syntax

grammar :

declarations-section rules-section [programs-section]

Rules

Declarations Section, Rules Section, Programs Section

Remarks

A YACC source file consists of three distinct sections: the declarations section, rules section and programs section.  All three sections are separated by the character sequence %%, sometimes called a mark.

declarations

%%

rules

%%

programs

The second mark, and the following programs are optional.  Typically any declarations go in the declarations section, the grammar rules and their associated actions in the rules section and any user routines go in the (optional) programs section.

Note that the contents of the declarations section is optional, however there must be at least one grammar rule in the rules section in order for the YACC source file to compile correctly.  As already stated, the second mark and subsequent programs section is also optional.  This leaves the minimum YACC source file as simply:

%%

A : /* empty */

  ;

The parser that is generated from such a grammar does not do much at all.  All it will do, in effect, is accept empty files, i.e. it will need to see the $end (or done) token immediately without any other intervening tokens.

See Also

Grammar Summary, Code Block


YACC Declarations Section
Declarations Section

Syntax

declarations-section :

[declaration-list]

declaration-list :

declaration-list declaration

declaration

declaration :

token-decl

start-decl

type-decl

union-decl

name-decl

include-decl

Rules

Token Declaration, Start Declaration, Type Declaration, Union Declaration, Name Declaration, Include Declaration

Remarks

The declarations section is where any information is set-up prior to defining the rules for the parser.  You can declare functions and define variables which will be used in actions by placing the relevant code in a Code Block or Include Declaration.

You can declare the tokens that will be used in your grammar using a Token Declaration.  Furthermore you can associate type information with the tokens, as well as explicitly setting token numbers and associativity and precedence levels for specifying operator precedence grammars.  You can specify the starting rule for the grammar using a Start Declaration.  You can associate types with nonterminal symbols using a Type Declaration.  You can represent more than one type on the attribute stack using a Union Declaration, and if you are building a C++ parser, or a multiple model, or multiple instance model C parser, then you can specify the name of the parser using a Name Declaration.

See Also

Grammar, Code Block


YACC Type Declaration
Type Declaration

Syntax

type-decl :

%type tag init-nonterminal-list

init-nonterminal-list :

init-nonterminal-list [,] init-nonterminal

init-nonterminal

init-nonterminal :

identifier

Tokens

Tags, Identifiers

Remarks

A type declaration declares nonterminal symbols for use in your grammar, and associates a type with the declared symbols.  It is only necessary to use a nonterminal declaration if you are going to have more than one type represented on the attribute stack.  For instance some nonterminals might have an attribute of type int, where as some others might have an attribute of type node *, where node is a syntax tree node structure.

Note that it is not strictly necessary to use a type declaration when using multiple types.  It does however, along with a Token Declaration, provide extra type checking, cutting down on the number errors which can occur.

The Tag following the %type keyword specifies the type which is to be associated with the nonterminal symbols.  The contents of the tag, that is the tag without the surrounding angled brackets, should match up with a member of the union defined by YYSTYPE.  The definition of which can be done explicitly by placing the necessary code in a Code Block, or through using a Union Declaration.  The example below shows a snippet of what might appear in the declarations section of a YACC file defining a C compiler:

%union {

    unsigned int type;

    NODE *node;

    SYMBOL *symbol;

    ...
}

%type <type> TypeQualifier

%type <node> Statement

%type <symbol> Constant

Note that no checks are made on whether the tag is valid and corresponds to a member of the YYSTYPE union.  Indeed no checks are made to ensure that YYSTYPE actually corresponds to a union.  Such errors will be detected by the C/C++ compiler when the generated output is finally compiled by it.

It is possible to declare an identifier as being a nonterminal symbol more than once, however it is not possible to re-declare the type associated with the symbol.  It is not possible to declare an identifier as being a nonterminal symbol if it has already been declared as a token.  For example:

%token A

%type <T1> a

%type <T1> A    // error, already declared as a token

%type <T1> a    // ok

%type <T2> a    // error, different type

If you use a type declaration in your grammar, then this indicates to YACC that all attributes ($$, $n) used in actions should have an associated type.  This also applies to the default copy action which is used when there is no action is associated with a rule.  Note that there is no default copy action for empty rules i.e.:

A: /* useless */ ;

In those cases where an untyped attribute is encountered in an action a warning is issued by the compiler, rather than an error.  This will typically lead, however, to some form of error being reported by the C/C++ compiler when the generated output is finally compiled by it.

See Also

Declarations Section


YACC Token Declaration
Token Declaration

Syntax

token-decl :

associativity [tag] init-token-list

associativity :

%left

%nonassoc

%right

%token

init-token-list :

init-token-list [,] init-token

init-token

init-token :

identifier [integer]

character-token [integer]

Tokens

Identifiers, Tags, Character Tokens, Integers

Remarks

A token declaration declares tokens, or terminal symbols, for use in your grammar.  If an identifier is going to be used as a token then it must be declared as such using a token declaration.  By default, an identifier which has not been declared as a token or nonterminal symbol is assumed to be a nonterminal symbol if it is encountered in the rules section.  A character token is always assumed to be a token.  It is included in a token declaration so that other information can be associated with it.  The token declaration below declares the tokens IF and ELSE:

%token IF, ELSE

Tokens are usually assigned a token number when they are declared.  This is the value which identifies the token to the parser.  When a compact parser is being built certain character tokens are not automatically assigned a token number.  Instead the character constant corresponding to the character token is used in the parser's tables.  It is then the C/C++ compilers responsibility to convert the character constant into an integer, which is the token number.  Note that compact parsers use a different table storage method to fast parsers, which enables them to do this.

Identifiers declared as tokens are always automatically assigned a token number, regardless of whether a default (fast) parser or a compact parser is being built.  The number assigned is always one greater than the number assigned to the previous token.  If no token has been declared previously then the token is assigned the number 257.  Note that type 1 character constants as defined below have no effect on this numbering scheme, and will never be considered as a previous token.  A #define statement is generated in the include or output file which defines the token.  In the IF, ELSE example above, assuming this was the first token declaration, the generated #define statements would look like:

#define IF 257

#define ELSE 258

Character tokens are little more complicated than identifiers.  There are three different categories which depend upon the number, and type characters which make up the character token:

1	The character token consists of a single quoted character.  This includes characters escaped using the backslash character \ and characters expressed as octal and hexadecimal numbers using the \ooo and \xhh character escapes.  Examples of these types of character token are: 'a', '+', '\n', '\123' and '\x7f'.  If you are building a default (fast) parser, then the character token will be assigned a token number equivalent to the ASCII value of the character.  In the case of octal and hexadecimal escape sequences, the token number is specified explicitly by the octal or hexadecimal number.  If you are building a compact parser no token number will be assigned to the character token.  This is because the character token is copied verbatim, where necessary, into the tables in the generated output file.

2	The character token consists of two or more quoted characters forming a valid C/C++ identifier.  A valid C/C++ identifier consists of a letter, digit or underscore character _, followed by, in this case, one or more letters, digits and underscore characters.  Examples of these types of character token are: 'ab', 'k99' and 'a_very_long_token'.  The character token will be assigned a token number one greater than that assigned to the previous token.  For this purpose this does not include type 1 character tokens above.  It does however include identifiers that have been declared as tokens.  If no tokens have previously been declared, the token has a token number of 257.  A #define statement is also placed in the generated output file to define the token.  The surrounding quotes are effectively stripped off for this purpose.  If an include file is generated for the parser it will be placed in there instead.  The example below declares the 'LET' and 'identifier' keywords:

%token 'LET' 'identifier'

Assuming that no tokens have previously been declared YACC would generate the two following #define statements in the generated include or output file, one for the 'LET' token and the other for the 'identifier' token:

#define LET 257

#define identifier 258

Be aware that it is possible to declare a token LET and a token 'LET'.  They will both be considered as separate tokens by YACC, and will each generate a separate #define statement for LET in the output file.  It is only when the C/C++ file is compiled that a redefinition error will be reported by the compiler.

3	Any other character tokens except types one and two above.  Examples are: '==', 'abc!' and '99'.  In most respects the character tokens behave similarly to type 2 character tokens, except that no #define statements are generated.

It is possible to change the number assigned to a token.  This is done by following the token with the relevant number.  Note that it is not possible to change the number associated with a character token containing a single character (a type 1 character token).  Be careful of assigning numbers of less than 256 to tokens.  Character tokens containing a single character are assigned numbers in this range.  Also do not, under any circumstances, assign the values 0 or 256 to a token.  The value 0 is special and indicates the $end token to the parser.  This means that there are no more tokens for the parser to read.  Likewise, the value 256 indicates the error token.  This is used by the parser for implementing error recovery.

If you do assign a number to a token then be aware that the numbering of any following tokens may change.  If a token is not explicitly assigned a number, then as stated, it is assigned a number one greater than the previously declared token's number.  For example:

%token IF       // 257 - first token so assigned 257

%token THEN     // 258

%token FOR 500  // 500

%token 'a'      // 121 - no effect on numbering

%token 'ab'     // 501 - contains more than one character

%token NEXT     // 502

AYACC checks for invalid token number assignments, for instance the assignment of the same token number to two or more different tokens.  There are however cases where it cannot detect token number clashes.  These occur when building compact parsers and single character, or type 1, character constants are used.  For instance, the two declarations:

%token 'a'      // ASCII value of 97

%token ID 97

are valid and will be accepted by AYACC.  The token 'a' is converted to a number by the C/C++ compiler, rather than AYACC.  With the usual ASCII representation, this will be 97, resulting in a clash with ID.

It is possible to associate a type with a token.  It is only necessary to do this if you are going to have more than one type represented on the attribute stack.  For instance some tokens might have an attribute of type int, where as some others might have an attribute of type symbol *, where symbol is a symbol table entry structure.

Note that it is not strictly necessary to associate types with tokens when using multiple types.  It does however, along with a Type Declaration, provide extra type checking, cutting down on the number errors which can occur.

The Tag following the associativity keyword specifies the type which is to be associated with the tokens.  The contents of the tag, that is the tag without the surrounding angled brackets, should match up with a member of the union defined by YYSTYPE.  The definition of which can be done explicitly by placing the necessary code in a Code Block, or through using a Union Declaration.

Note that no checks are made on whether the tag is valid and corresponds to a member of the YYSTYPE union.  Indeed no checks are made to ensure that YYSTYPE actually corresponds to a union.  Such errors will be detected by the C/C++ compiler when the generated output is finally compiled by it.

If the %left, %right and %nonassoc keywords are used to declare a token then it will be assigned a precedence and associativity.  Tokens declared first will be given a lower precedence than those declared last.  Tokens in the same token declaration have the same precedence.  A token's precedence and associativity is only used when trying to resolve ambiguous grammars.  They are typically used when specifying an operator precedence grammar for expressions.  These have the advantage of being faster than normal (predictive) parsers.

Keyword	Description

%left	Tokens (operators) have left associativity, i.e. A op B op C will be treated as (A op B) op C.  Most binary operators are left associative (addition, subtraction, multiplication, division etc.).

%right	Tokens (operators) have right associativity, i.e. A op B op C will be treated as A op (B op C).

%nonassoc	Tokens (operators) are non-associative, i.e. it is not possible to have A op B op C.  It is however possible to have A op B op' C op D, where op' is some other operator.  The case that is always mentioned is the FORTRAN equivalence operator ".EQ.":

IF A .EQ. B .EQ. C THEN

The above example is illegal in FORTRAN.  Compare this to the C/C++ equivalent, which although valid, has questionable use:

if (a == b == c) {

It is possible to declare an identifier or character token as being a token more than once, however there are some limitations when doing this.  They are listed below, together with some examples:

1	It is not possible to declare an identifier as being a token if it has already been declared as a nonterminal symbol.  For example:

%type <T1> A

%token A  // error, already declared as a nonterminal symbol

2	If a token number is explicitly assigned to a token, then it must occur for the first declaration of the token.  It is not necessary to assign the number in subsequent declarations.  For example:

%token IF 300

%token THEN         // 301

%token IF           // ok (300)

%token IF 400       // error, different token number

%token THEN 500     // error, different token number

3	If a type is associated with a token, it is not possible to subsequently re-type the token.  Note that omitting the tag from a token declaration does not constitute re-typing.  In general, a token can be typed in any token declaration.  For example:

%token <symbol> ID

%token NUMBER

%token <node> ID    // error, different type

%token ID           // ok

%token <int> NUMBER // ok

4	If a precedence and associativity is assigned to a token through one of the keywords %left, %right or %nonassoc, it is not possible to subsequently assign a new precedence and associativity to the token.  Note that the %token keyword does not confer any precedence and associativity to a token, and nor does it change any existing precedence and associativity that the token might have.  For example:

%left ADD

%token MULTIPLY

%left ADD           // error, different precedence!

%token ADD          // ok

%left MULTIPLY      // ok

%right MULTIPLY     // error, different precedence and associativity

If you do associate types with tokens in your grammar, then this indicates to YACC that all attributes ($$, $n) used in actions should have an associated type.  This also applies to the default copy action which is used when there is no action is associated with a rule.  Note that there is no default copy action for empty rules i.e.:

A: /* useless */ ;

In those cases where an untyped attribute is encountered in an action a warning is issued by the compiler, rather than an error.  This will typically lead, however, to some form of error being reported by the C/C++ compiler when the generated output is finally compiled by it.

Compatibility

The token declaration keywords have a number of synonyms:

Keyword	Synonym

%left	%<

%nonassoc	%2, %binary

%right	%>

%token	%term, %0

See Also

Declarations Section, Type Declaration


YACC Start Declaration
Start Declaration

Syntax

start-decl :

%start identifier

Tokens

Identifiers

Remarks

A start declaration declares the starting rule for the grammar.  It can only be used in the declarations section of the YACC source file.  If this declaration is not used then the first rule defined in the rules section is taken as the starting rule.  The example below declares term as the starting rule.  In effect this will mean that the addition and subtraction operators are no longer available.  Note though that the compiler will warn that the expr rules are never reduce (used) in the grammar.

%{

#define YYSTYPE double
%}

%token NUMBER

%start term

%%

expr

    : expr '+' term     { $$ = $1 + $3; }

    | expr '-' term     { $$ = $1 - $3; }

    | term

    ;

term

    : term '*' factor { $$ = $1 * $3; }

    | term '/' factor { $$ = $1 / $3; }

    | factor

    ;

factor

    : '(' expr ')'

    | - factor { $$ = -$1; }

    | NUMBER

    ;

It is not possible to use a start declaration more than once in a YACC source file, even if the identifier being declared as the starting rule is the same.  It is not possible to declare an identifier as the starting rule if it has already been declared as a token.  Note that by its very nature a start declaration declares the identifier as a nonterminal symbol.

See Also

Declarations Section


YACC Union Declaration
Union Declaration

Syntax

union-decl :

%union declaration-code-block

Tokens

Declaration Code Block

Remarks

A union declaration allows more than one type to be represented on the attribute stack.  Typically, it is only necessary to do this if you are creating a complex grammar.  For instance, the union declaration below shows some of the types that might be represented on an attribute stack for a C compiler:

%union {

    symbol_t *symbol;

    node_t *node;

    flags flags;

    other attribute type members…
}

Where, symbol is a pointer to a symbol table entry, node is a pointer to a syntax tree node and flags is an arbitrary type for holding type qualifier (const, volatile) etc. information.

The contents of the declaration code block are copied, verbatim, into a C/C++ union in the generated include or output file.  If no include file is generated, then the union is placed in the output file.  The contents of the declaration code block are not checked in any way, so it is your responsibility to ensure that the contents are correct.

The members of the union should correspond exactly to types associated with nonterminal symbols and tokens.  Nonterminal symbol type information is assigned using a Type Declaration, and token type information is assigned using a Token Declaration.  In both cases it is the Tag, or rather the tag without the surrounding angled brackets, associated with the declaration which specifies the type.  For instance, the three declarations below show how type information might be specified for some symbols in a C compiler:

%token <symbol> ID

%type <node> StatementList

%type <flags> TypeQualifier

Again, no checks are made on whether union members match up to types.  Instead, such errors will be detected when the generated parser is compiled by the C/C++ compiler.  For example, the following misspelling will not be detected by AYACC.  Instead it will think there is a new type symbo:

%type <symbo> INTEGER

For simple grammars, where only one type is used, it is usual to declare the attribute type by defining YYSTYPE.  This can be done by placing the relevant #define statement in a Code Block or the Declaration Code Block associated with an Include Declaration.  For instance, the code block below declares the attribute stack to be of type double:

%{

#define YYSTYPE double
%}

If YYSTYPE is not defined, then it assumes the type int by default.  For more information see Defines.

As an alternative to using a union declaration, you can duplicate its effect by defining YYSTYPE instead.  The include declaration below replicates the effects of using a union declaration:

%include {

#ifndef YYSTYPE

union tagYYSTYPE {

    attribute type members…

};

#define YYSTYPE union tagYYSTYPE

#endif
}

It is not possible to use a union declaration more than once in a YACC file.  If you attempt to use more than one union declaration, then an error will be generated.

See Also

Declarations Section, Type Declaration, Token Declaration, Defines


YACC Name Declaration
Name Declaration

Syntax

name-decl :

%name identifier [declaration-code-block] [declaration-code-block]

Tokens

Identifiers, Declaration Code Block

Remarks

This declaration declares the name of the parser.  It is not necessary to do this if you are generating a single model C parser.  However if you are generating a multiple, or multiple instance model C parser, or a C++ parser then you will need to do so.  The name declaration below declares the name of the parser as myparser:

%name myparser

For a multiple or multiple instance model C parser, the name specifies the name of a yymparse_t structure generated in the output file.  If you have generated an include file as well, then a corresponding declaration for the structure will appear within it.  As an alternative to using a name declaration, you can duplicate its effect by defining YYPARSENAME instead.  The two Include Declarations below replicate the effects of using a name declaration.  The first is for a multiple model C parser, and the second is for a multiple instance model C parser:

%include {

extern yymparse_t YYNEAR name;

#ifndef YYPARSENAME

#define YYPARSENAME name

#endif

}

%include {

extern YYCONST yymparse_t YYNEAR name;

#ifndef YYPARSENAME

#define YYPARSENAME name

#endif
}

Where name is the name of the parser.  Note that the only difference between the two include declarations is the presence of the YYCONST modifier for multiple instance model parsers.  The structure generated for a multiple instance model parser is used as a template for creating other parsers, and as such is never modified.

For a C++ parser, a declaration name specifies the name of a new C++ class.  This new class will be derived from one of the existing YACC and Lex library classes.  For instance, if you have generated a default (fast) parser, then the new class will be derived from yyfparser.  Otherwise, if you have generated a compact parser it will be derived from yycparser.

If you generated an include file for your parser, then the definition for the new class will be placed within it.  Otherwise it will be placed within the output file.  A default constructor is always generated for the new class.  The definition of which is always placed in the output file.

It is possible to place additional member functions and member variables in the new class.  You can do this by placing a Declaration Code Block after the name declaration.  The contents of the declaration code block will be copied, verbatim, into the class definition.  By default, the members will be declared as public.  You can of course change this by placing the relevant private / protected declarations in the class.

It is also possible to place additional code in the default constructor for the new class.  In this case, it is the presence of a second declaration code block which provides the source.  The contents of the code block will be copied, verbatim, into the constructor.  Note that the default constructor calls the member function yytables.  This function initialises the table array member variables for the class with their required values.  If you add other constructors to the class, be sure to call this function, otherwise the parser will not work.  For instance, the name declaration below will place an int member variable in the class myparser, and initialise it to zero in the constructor:

%name myparser

// class definition

{

    int m_count;    // add an m_count member variable...
}

// constructor

{

    m_count = 0;    // and initialise it to zero
}

Similar to multiple model and multiple instance model C parsers, it is also possible to define YYPARSENAME as the name of the new C++ class.  However, in order to do this you will also need to generate a class definition, and a constructor.  The include declaration and Code Block below replicate the effects of using a name declaration:

%include {

class YYFAR name : public yyparser {

public:

    name();

protected:

    void yytables();

    virtual int yyaction(int action);

#ifdef YYDEBUG

    void YYFAR* yyattribute1(int index) const;

    void yyinitdebug(void YYFAR** p, int count) const;

#endif

public:

    other class definition code…

};

#ifndef YYPARSENAME

#define YYPARSENAME name

#endif

}

%{

YYPARSENAME::YYPARSENAME()

{

    yytables();

    other constructor code…

}

%}

Where name is the name of the parser, and yyparser is the base class of the lexical analyser.  This will either be yyfparser or yycparser, depending upon whether you have generated a default (fast) parser or a compact parser.  The member functions yytables, yyaction, yyattribute1 and yyinitdebug are automatically generated by AYACC.  Obviously, it is much easier to use a name declaration, rather than typing in the new class by hand.

It is only possible to specify one name declaration in a YACC file.  If you attempt to use more than one name declaration, then an error will be generated.

If you specify a name declaration for a single model C parser, then it is ignored.  If any declaration code blocks are placed after the name declaration, then the first block is copied into the generated include file, or the output file if no include file is generated.  The second block, if any, is always copied into the output file.

If a name declaration for a multiple or multiple instance model C parser has any following declaration code blocks, then they are copied into the generated include and output files as specified for single model C parser's above.

See Also

Declarations Section, Include Declaration, Code Block, Defines, yymparse_t, yyfparser, yycparser


YACC Include Declaration
Include Declaration

Syntax

include-decl :

%include declaration-code-block

Tokens

Declaration Code Block

Remarks

An include declaration places the code specified in the Declaration Code Block into the generated include file.  If an include file is not being generated, then the code is placed in the output file instead.  For example, the include declaration below places a declaration for the function myfun in the generated include file:

%include {

    void myfun(void);
}

In order to generate an include file it is necessary to specify the -i option when running the AYACC tool.  You can do this from Parser Generator by selecting the Generate Include File option in the AYACC Options dialog box.  You can reach this dialog box by first choosing the Project command from the Options menu, and then clicking the AYACC button in the Project Options dialog box.

See Also

Declarations Section


YACC Defines
Defines

It is possible to define a number of symbols in the output file which will affect how the parser behaves.  This is done using the C preprocessor #define statement:

#define name value

Typically you will place such definitions in a Code Block in the declarations section of your YACC source file.  It is possible to place some definitions elsewhere, however it is recommended that they should all be placed in the declarations section.

A list of the symbols that can be defined, together with their descriptions is shown below.  Note that they have been split up into a number of different groups.  The first is for all parsers, the second for C++ parsers only, and the last two for C multiple model parsers.

All Parsers

Name	Description

YYSTACK_SIZE	Defines the size of the initial (static) stack.  This includes both the state stack, and the accompanying attribute stack.  If you do not define YYSTACK_SIZE, then it is assigned an initial value of 100.

YYSTYPE	Defines the attribute stack type.  If you do not define YYSTYPE, then it is defined as an int by default.  Note that YYSTYPE is also defined by the Union Declaration.

C++ Parsers

Name	Description

YYPARSENAME	Defines the name of the parser.  You must define this symbol if you are building a C++ parser.  It is however much more preferable to use a Name Declaration to this for you.  With a C++ parser it is not just the name of the parser that you have to define, but a whole class on which to hang the generated parser.  If you wanted to do things long hand then you would need to generate the following code (or something similar) in your output and include files:

Include File

class YYFAR name : public yyparser {

public:

    name();

protected:

    void yytables();

    virtual void yyaction(int action);

#ifdef YYDEBUG

    void YYFAR* yyattribute1(int index) const;

    void yyinitdebug(void YYFAR** p, int count) const;

#endif

public:

};

#ifndef YYPARSENAME

#define YYPARSENAME name

#endif

Output File

YYPARSENAME::YYPARSENAME()
{

    yytables();
}

Where name is the name of the parser, and yyparser is the base class of the parser.  The base class depends upon whether you have built a default (fast) parser or a compact parser.  If you have built a fast parser, then the base class should be yyfparser.  If you have built a compact parser then the base class should be yycparser.  Definitions for the yytables, yyaction, yyattribute1 and yyinitdebug member functions are generated automatically by AYACC.

Multiple C Parsers

Name	Description

YYPARSENAME	Defines the name of the parser.  You must define this symbol if you are building a multiple parser.  There is no default value.  Note that YYPARSENAME is also defined by a Name Declaration.  This declaration has some advantages over defining YYPARSENAME directly.

YYMPARSEDATA	Defines the custom data to associate with the parser.  This value is assigned to the yymdata member of the yymparse_t structure.  Note that the yymdata member is of type void *.  If YYMPARSEDATA is undefined then it assumes the value NULL.  Also note that this member is used by the default yymgettoken function.  This function assumes that the yymdata member points to a valid yymlex_t structure from where it can get input by calling the yymlex function.

Multiple C Parsers (Service Functions)

Service functions are called by the parser when they need to perform a task i.e. they want to get the next lookahead token or they want to report a syntax error etc.  They are implemented in the yymparse_t structure as a set of pointers to functions.  This means that it is possible to easily plug in a new, replacement function, if required.

Name	Description

YYMDISCARD	Defines the pointer to the token discarding reporting routine.  This value is assigned to the yymdiscard member of the yymparse_t structure.  The yymdiscard member is of type void (*)(yymparse_t *yy, int token).  If YYMDISCARD is undefined then it assumes the value yymdiscard.  That is it points to the function yymdiscard.

YYMERROR	Defines the pointer to the main error reporting routine.  This value is assigned to the yymerror member of the yymparse_t structure.  The yymerror member is of type void (*)(yymparse_t *yy, const char *text).  If YYMERROR is undefined then it assumes the value yymerror.  That is it points to the function yymerror.

YYMGETTOKEN	Defines the pointer to the token input routine.  This value is assigned to the yymgettoken member of the yymparse_t structure.  The yymgettoken member is of type int (*)(yymparse_t *yy).  If YYMGETTOKEN is undefined it assumes the value yymgettoken.  That is it points to the function yymgettoken.  Note, that as stated above, the default implementation of yymgettoken makes use of the YYMPARSEDATA member.

YYMSTACKOVERFLOW	Defines the pointer to the stack overflow error reporting routine.  This value is assigned to the yymstackoverflow member of the yymparse_t structure.  The yymstackoverflow member is of type void (*)(yymparse_t *yy).  If YYMSTACKOVERFLOW is undefined then it assumes the value yymstackoverflow.  That is it points to the function yymstackoverflow.

YYMSYNTAXERROR	Defines the pointer to the syntax error reporting routine.  This value is assigned to the yymsyntaxerror member of the yymparse_t structure.  The yymsyntaxerror member is of type void (*)(yymparse_t *yy).  If YYMSYNTAXERROR is undefined then it assumes the value yymsyntaxerror.  That is it points to the function yymsyntaxerror.

See Also

Declarations Section, Code Block, yymparse_t


YACC Rules Section
Rules Section

Syntax

rules-section :

%% rule-list

rule-list :

rule-list rule

rule

rule :

grammar-rule

symbol-destructor-rule

Rules

Grammar Rule, Symbol Destructor Rule

Remarks

The rules section is where the grammatical rules for the parser are defined.  Any symbol destructors, called when a symbol is discarded during error recovery are also defined here.  The rules section is introduced with the first %% mark in the YACC source file.

If a Code Block is placed before the first rule in the rules section, then the enclosed text in the code block is copied into the top of the generated yyparseaction, yymparseaction or yyparser::yyaction function.  This function is called whenever the parser needs to execute an action.  Otherwise the text in the code block is copied after the previous action in the generated output file.  However, note that these actions have a trailing break statement, so the code will never be executed.

The first grammar rule defined in the rules section is taken as being the starting rule, unless otherwise declared by a Start Declaration.  Note that it is only Grammar Rules which define them.  Symbol Destructor Rules simply provide a means of cleaning up symbols during error recovery.

There must be at least one grammar rule (excluding symbol destructor rules) in the rules section in order for YACC to successfully create a parser.

See Also

Grammar, Code Block


YACC Grammar Rule
Grammar Rule

Syntax

grammar-rule :

nonterminal : production-list ;

production-list :

production-list | production

production

nonterminal :

identifier

Rules

Production

Tokens

Identifiers, Actions

Remarks

A grammar rule defines a grammatical rule for the parser.  A rule consists of two parts - a left hand side (lhs) and a right hand side (rhs).  In YACC these are separated by a colon : character.  The left hand side consists of a nonterminal symbol, whilst the right hand side consists of zero or more symbols.  These can be either nonterminal symbols or tokens, and are represented in the above syntax by the Production rule.  The nonterminal in the left hand side of the rule is said to consist of the symbols in the right hand side of the rule.  The example below shows an IF statement rule for a BASIC interpreter.  For clarity, nonterminal symbols are shown in italics, and tokens are shown in fixed width font:

statement : IF expression THEN ;

The nonterminal statement is said to consist of the keyword IF, followed by an expression (nonterminal), and the keyword THEN.

It is possible for more than one rule with the same left hand side to be defined in a grammar.  For example, in addition to an IF statement it might also be nice to have a LET statement:

statement : LET VARIABLE = expression ;

Here a statement can now consist of an IF statement or a LET statement.  Since it is quite common to define rules with the same left hand side it is possible to group the right hand sides together using the | character as a separator.  For example, the FOR and LET statements above could be grouped as follows:

statement

: IF expression THEN

| LET VARIABLE = expression

;

It is also possible to specify recursive rules for grammars.  In its simplest form this is where the nonterminal on the left hand side of a rule appears in the right hand side.  The three examples below show different types of recursive (or iterater) rules:

interpreter_unit_list

: interpreter_unit_list interpreter_unit

| interpreter_unit

;

interpreter_unit_list

: interpreter_unit_list interpreter_unit

| /* empty */

;

interpreter_unit_list

: interpreter_unit interpreter_unit_list

| interpreter_unit

;

The first example is what is termed a left recursive rule.  This is determined by the position of the recursive nonterminal in the right hand side.  In this case interpreter_unit_list is on the left, hence it is left recursive.  The second example is also left recursive, however there is an empty rule.  This means that an interpreter_unit_list can consist of zero or more interpreter_unit’s.  The first example on the hand will always consist of at least one interpreter_unit.

The third, and last example is however right recursive - interpreter_unit_list is on the right.  Be careful about using right recursive rules in YACC.  In the example above the parser’s stack will continually grow by one for each interpreter_unit seen.  It is only when there are no more interpreter_unit’s that the recursive rule can be used to start reducing rules and popping symbols off of the stack.  This is not to say don’t use right recursive rules, but to strongly favour left recursive ones where ever possible.  In some circumstances though right recursive rules are required.  Consider for instance prefix operators, and in particular unary minus.  A rule for this operator might look something like:

unary_expression: - unary-expression ;

There are ways around this, but in general, it’s safe to say, you don’t see a large number of unary minus signs in front of expressions.

Compatibility

In AYACC all grammar rules must be terminated by a semicolon ; character.  In other versions of YACC the semicolon is optional.

See Also

Rules Section, Ambiguity Resolution


YACC Production
Production

Syntax

production :

[symbol-list]

symbol-list :

symbol-list symbol

symbol

symbol :

grammar-symbol

action-symbol

precedence-symbol

Rules

Grammar Symbol, Action Symbol, Precedence Symbol

Remarks

A production represents the right hand side of a rule, and can consist of zero or more symbols.  A symbol in this case is quite broad and covers Grammar Symbols for representing nonterminal symbols and tokens, Action Symbols for representing embedded actions as well as actions associated with rules and Precedence Symbols for use when specifying operator precedence grammars.

Note that the grammar symbol rule is also used to specify the symbols for a Symbol Destructor Rule.

See Also

Grammar Rule


YACC Grammar Symbol
Grammar Symbol

Syntax

grammar-symbol :

identifier

character-token

Tokens

Identifiers, Character Tokens

Remarks

A grammar symbol represents a symbol in the right-hand-side of a rule.  It also represents a symbol in the left-hand-side of a symbol destructor rule i.e. it specifies what symbol the destructor action is associated with.  In both these cases a grammar symbol refers to either a nonterminal symbol or a token.

An Identifier can represent either a nonterminal symbol or a token.  In order for an identifier to represent a token it must be declared as such in the declarations section using a Token Declaration.  If an identifier has not already been declared as a token or nonterminal symbol, then it is declared as a nonterminal symbol at this point.  It is not necessary to define rules for nonterminal symbols before they appear in other rules.  For attribute purposes, nonterminals introduced in the rules section do not have a type, as specified in the declarations section using a Type Declaration.

A Character Token can only represent a token.  If a character token has not already been declared as a token then it is declared as one at this point.  What this entails depends upon the format of the character token, and whether a compact lexical analyser is being constructed or not.  There are three different types of character token, which are described below:

1	The character token consists of a single quoted character.  This includes characters escaped using the backslash character \ and characters expressed as octal and hexadecimal numbers using the \ooo and \xhh character escapes.  Examples of these types of character token are: 'a', '+', '\n', '\123' and '\x7f'.  If you are building a default (fast) parser, then the character token will be assigned a token number equivalent to the ASCII value of the character.  In the case of octal and hexadecimal escape sequences, the token number is specified explicitly by the octal or hexadecimal number.  If you are building a compact parser no token number will be assigned to the character token.  This is because the character token is copied verbatim, where necessary, into the tables in the generated output file.

2	The character token consists of two or more quoted characters forming a valid C/C++ identifier.  A valid C/C++ identifier consists of a letter, digit or underscore character _, followed by, in this case, one or more letters, digits and underscore characters.  Examples of these types of character token are: 'ab', 'k99' and 'a_very_long_token'.  The character token will be assigned a token number one greater than that assigned to the previous token.  For this purpose this does not include type 1 character tokens above.  It does however include identifiers that have been declared as tokens.  If no tokens have previously been declared, the token has a token number of 257.  A #define statement is also placed in the generated output file to define the token.  The surrounding quotes are effectively stripped off for this purpose.  If an include file is generated for the parser it will be placed in there instead.  The example below represents a LET statement in a BASIC interpreter:

statement: 'LET' 'identifier' '=' expression;

Assuming that no tokens have previously been declared YACC would generate the two following #define statements in the generated output file, one for the 'LET' token and the other for the 'identifier' token:

#define LET 257

#define identifier 258

Be aware that it is possible to declare a token LET and a token 'LET'.  They will both be considered as separate tokens by YACC, and will each generate a separate #define statement for LET in the output file.  It is only when the C/C++ file is compiled that a redefinition error will be reported by the compiler.

3	Any other character tokens except types one and two above.  Examples are: '==', 'abc!' and '99'.  In most respects the character tokens behave similarly to type 2 character tokens, except that no #define statements are generated.

See Also

Production, Symbol Destructor Rule


YACC Action Symbol
Action Symbol

Syntax

action-symbol :

[tag] action [action-destructor]

action-destructor :

~ action

Tokens

Tags, Actions

Remarks

An action symbol represents an action in the right-hand side of a rule, here referred to as a production.  The position of the action symbol, and the presence of a tag and/or action destructor has an effect on how the action is treated.

If the action symbol is the last symbol specified for a production, not including a Precedence Symbol, and it does not have an associated tag, or action destructor, then the action symbol is treated as an action.  This means that the code within the action will be executed whenever the production is reduced by the parser.  For example, the action in the rule below adds the number stored in the first expression on the right-hand-side to that in the second expression and stores the result in the left-hand-side.  When a rule is reduced, the symbols on the right-hand-side are replaced by the one on the left:

expr: expr '+' expr { $$ = $1 + $3; }

    ;

If the action symbol is not the last symbol specified for a production, or it has an associated tag or action destructor, then it treated somewhat differently.  Such actions are usually termed embedded actions, and are used as a notational shorthand for writing rules of the form:

A: { /* do something */ }

 ;

B: C A D

 ;

Using an embedded action, this can be written much more simply as:

A: B { /* do something */ } C

 ;

These types of construct are quite common, and are typically used for setting up attributes required by following symbols.  They provide a way of implementing inherited attributes, which are not readily supported by YACC.  As you may or may not be aware, YACC is far more comfortable with synthesised attributes.  That is information being passed from right to left, rather than left to right.  Thankfully, synthesised attributes are the more common of the two.

As well as being more compact, the action in the second example also has the advantage of being able to easily extract information from symbols to its left.  For instance, if B had an attribute which the action needed to access, then all it would need to do would be to make reference to $1.  The same cannot be said of the first example, which is not quite so obvious, clear or safe.  Since C is immediately to the left of A, its attribute lies just before the first attribute of A (assuming it had one) at $0.  If there was an intervening symbol between C and A, say E perhaps, then C’s attribute would lie at $-1 etc.  There is also the possibility that other rules may make use of A as well, in which case they had better make sure that when A goes looking, it picks up the right attribute.

The (hidden) nonterminal symbols introduced by embedded actions start at $$1, $$2 etc.  You can see these symbols if you generate a verbose file and examine its contents.  Additionally, these names will be used when debug information is output by the parser.

An example of the use of an embedded action is in the installation of a local symbol table in a compound statement for the C language.  After an opening bracket has been seen, it is necessary to install the new local symbol table.  In the embedded action, symtab is assumed to be a global variable which points to the current symbol table.  The function installsymtab creates a new symbol table.  Note that it is passed symtab as a parameter.  This is so that the symbol tables can be chained together.  When the compiler needs to look up a symbol, it first looks in the current symbol table.  If it can't find the symbol there, it then looks in the previous enclosing symbol table.  The compiler continues to do this until it finds a symbol or there are no more symbol tables in the chain.  At which point it installs a new symbol in the current symbol table and/or raises an error.  The final action in the rule is performed when the rule is reduced.  The action firsts creates a new compound statement node, passing it the required node type, followed by the pointer to the local symbol table, and the statement list node.  Note that in cases where there are no statements in the list, then this parameter will be NULL.  The action then returns symtab back to its previous value:

statement:

    '{'

        { symtab = installsymtab(symtab); }

    DeclarationList StatementListOpt '}'

        { $$ = newnode(NT_COMPOUND, symtab, $4);

          symtab = symtab->parent; }

In the example above, no references to any attributes are made in the embedded action.  In particular, no reference is made to the $$ attribute, which refers to the (hidden) nonterminal associated with the embedded action.  By default these attributes always assume the type of the nonterminal on the left-hand-side of the rule in which the action is embedded.  In the example above, $$ would assume the same type is that for the Statement nonterminal.  It is possible to override this default typing by prefixing the action with a tag which identifies its type. For instamce, the above example could be re-written using the $$ attribute and tag prefixes as:

statement:

    '{'

        <symtab> { symtab = installsymtab(symtab); $$ = symtab; }

    DeclarationList StatementListOpt '}'

        { $$ = newnode(NT_COMPOUND, $2, $4);

          symtab = $2->parent; }

Note that the $2 attribute in the statement action is automatically assigned the type symtab.

The last thing that needs to be mentioned about embedded actions is that they can have an associated action destructor.  This is an action that will be called when the nonterminal symbol associated with the embedded action is popped off of the stack when the parser is attempting error recovery.  An action destructor is implemented by following the action with a tilde character ~, and then the action to be carried out.  The attribute associated with the nonterminal that is being popped off of the stack can be accessed using $$.  For instance, if an action destructor were to be used in the C compound statement example, then it might look something like:

statement:

    '{'

        <symtab> { symtab = installsymtab(symtab); $$ = symtab; }

            ~ { symtab = $$->parent; delete($$); }

    DeclarationList StatementListOpt '}'

        { $$ = newnode(NT_COMPOUND, $2, $4);

          symtab = $2->parent; }

First, symtab, the global symbol table variable is restored to its previous value, and then the newly created, but no longer needed symbol table is deleted.  Note that the destructor does not affect attribute numbering, so $4 in the Statement action still makes reference to the node specified by the StatementListOpt nonterminal symbol.

Action destructors and associated symbol destructors provide a convenient way to deal with potential memory leaks which can occur in applications.  Although this wont, in general, be an issue for applications which run, do some processing, and terminate.  It most certainly will be for applications which remain active for a considerable amount of time.  In such cases destructors are typically associated with actions which allocate memory.  In the example above, the action destructor is used to delete the symbol table that was allocated by the call to installsymtab.  Similarly, there will also be a destructor for the statement nonterminal symbol.  This will be responsible for deleting the node that was created in it's action.  It is not necessary to declare action (or symbol) destructors when they are not needed.  In such cases, when one of these symbols is popped off of the stack, no action is performed.

For more information see also Symbol Destructor Rule and Error Recovery.

See Also

Production, Precedence Symbol


YACC Precedence Symbol
Precedence Symbol

Syntax

precedence-symbol :

%prec token

token :

identifier

character-token

Tokens

Actions, Identifiers, Character Tokens

Remarks

A precedence symbol allows the default precedence and associativity associated with a rule to be redefined.  Precedence symbols are used when specifying operator precedence grammars and/or for resolving ambiguous grammars.  For example, the following rule is typically used when specifying an unary minus expression for an operator precedence grammar.  Since the minus sign '-' is usually used as the subtraction operator, it is necessary to give the unary minus rule a higher precedence and right associativity.  This is achieved through using the pseudo token UMINUS:

expr: '-' expr %prec UMINUS

Ordinarily, the precedence and associativity of a rule is specified by the first token in the left-hand-side of the rule.  If the left-hand-side does not contain any tokens, then the rule will not have a precedence and associativity.  Whether tokens have a precedence and associativity associated with them depends upon how they have been declared using a Token Declaration.  If tokens are declared using the %left, %right or %nonassoc keywords then they will have a precedence and associativity, otherwise if they have been declared with the %token keyword they will not.

For more information see Ambiguity Resolution.

It is not possible re-declare the precedence and associativity of a rule more than once.  For example:

%left A

%left B

%%

a: b %prec A            // ok

a: c %prec A %prec A    // ok, same precedence and associativity

a: d %prec A %prec B    // error, different precedence and associativity

Ambiguity Resolution

When a shift-reduce/reduce-reduce error is encountered by AYACC when generating a parser it will first attempt to resolve the problem by comparing

See Also

Production, Token Declaration, Operator Precedence Grammars, Ambiguity Resolution


YACC Symbol Destructor Rule
Symbol Destructor Rule

Syntax

symbol-destructor-rule :

symbol-destructor-list : action ;

symbol-destructor-list :

symbol-destructor-list | symbol-destructor

symbol-destructor

symbol-destructor :

~ grammar-symbol

Tokens

Identifiers

Rules

Grammar Symbol

Remarks

A symbol destructor rule allows cleanup to be performed during error recovery.  Typically, when the parser encounters a syntax error it examines the topmost state on the stack.  If the state has an error token transition, then an error token is shifted onto the stack and the parser enters error recovery.  On the other hand if the state does not have an error token transition, then the state is popped off of the stack and the next one examined.  This continues until either a state with an error token transition is encountered, or all of the states are popped off of the stack.  If this happens then the parse fails and returns with an error code.

Any states that are popped off of the stack whilst looking for an error token transition are lost for ever.  Further more, any information associated with the state in the form of an attribute is also lost.  If for example we are building a node tree, then we may find that we start losing nodes.  These memory losses are not so important if the application runs, does some processing, and then terminates.  However, if the application remains active for some time, then these memory losses could add up.  There are a number of ways around the problem, however symbol destructors offer a simple solution.

For each symbol, either nonterminal or token, in the grammar you can define an action to be called when the symbol is popped off of the stack.  Note that although the stack itself stores states, each state maps to exactly one symbol.  You define a symbol destructor similar to the way you would normally define a rule.  One easily noticeable difference is the use of the tilde ~ character (borrowed from C++).

~A : { /* do something */ } ;

The action can make reference to the attribute associated with the symbol by using $$.  If you had been building a node tree, and you had a function deletenode then the destructor for A could look like:

~A : { deletenode($$); } ;

It is also possible to tie two or more symbol destructors to a single action using the separator |.  The grammar symbols must have the same type, otherwise the type for $$ cannot be determined.  If you had two more symbols B and C, both of the same type as A then you could write:

~A | ~B | ~C : { deletenode($$) } ;

If you do use symbol destructors in your grammar, it is not necessary to specify symbol destructors for all of your grammar symbols.  Only specify them for the symbols that you need to.

It is also possible to define a symbol destructor for an embedded action.  Note that an embedded action leads to the introduction of a (hidden) rule, and by association a nonterminal.  For more information on embedded action symbol destructors see Action Symbol.

It is not possible to define a symbol destructor for a grammar symbol more than once.  If you do so an error will be generated.

See Also

Rules Section, Action Symbol.


YACC Error Recovery
Error Recovery

YACC provides a simple, but reasonably effective, error recovery mechanism.  It does this by allowing you to specify error recovery rules.  These rules are similar to normal rules, except with one difference, and that is they make use of the error token.  This is a special token that is predefined by YACC for the specific purpose of implementing error recovery.  It cannot be redefined, and has a token number of 256.  Note that normal token numbering in a Token Declaration starts at 257, after characters 0 - 255, and the aforementioned error token.  Below is an example of an error recovery rule.  This rule could be used when implementing a C compiler.  Note that an ExpressionStatement (or expression-statement) encompasses the large group of statements which must be terminated by a semicolon ';':

ExpressionStatement: error ';'

Error recovery rules are typically only used when a parser encounters a syntax error.  That is the parser can neither shift a token onto the stack, reduce a rule, or accept the input.

When the parser encounters a syntax error, it first reports that an error has occurred.  It does this by calling the service function yysyntaxerror (single model C parser only, yymsyntaxerror for a multiple model C parser, yyparser::yysyntaxerror for a C++ parser).  Note however that if the parser is still in error recovery, say from an encounter with a previous syntax error, then this function will not be called.  The yysyntaxerror function typically generates the error message "syntax error".  You can however redefine the function to suit your needs.

After, potentially, reporting a syntax error, the parser then attempts to recover from the error.  It does this by first looking at the state on the top of the stack.  If this state has a shift action for the error token, then the token is quietly inserted into the input stream.  The token, or rather the state associated with the error token action, is then shifted onto the stack and the parser enters error recovery.  If however the state does not have an error token shift action, then it is popped off of the stack and the next state examined.  This is repeated until either an error token shift action is encountered, or all states are popped off of the stack.  Note that for all states popped off of the stack, if there is a symbol destructor associated with a state then one will be called for it.  In cases where the stack is emptied, no error recovery takes place, and the parser exits with the error code YYEXIT_FAILURE.  Note that this is commonly what happens when no error rules are specified for a grammar.

The parser remains in error recovery until three consecutive tokens have been shifted onto the stack.  During this time, the parser will typically behave as it normally would.  That is it will shift tokens onto the stack, reduce rules, and it can even accept the input.  When three tokens are eventually shifted onto the stack, the parser is said to have re-synchronised with the input stream, and error recovery is complete.

The important case though is if, or when, a syntax error occurs whilst the parser is still in error recovery.  Note that, as mentioned above, no new syntax error is reported.  In such circumstances one of three things can happen, depending upon how many tokens have been shifted onto the stack since the parser entered error recovery (note that this does not include the error token), and the current lookahead token:

1	If no tokens have been shifted onto the stack since entering error recovery, and the current lookahead token is the $end (done) token, then the parser exits immediately with the error code YYEXIT_FAILURE.  All states are popped off of the stack, and any symbol destructors called to perform cleanup.

2	If no tokens have been shifted onto the stack since entering error recovery, and the current lookahead token is not the $end (done) token, then the lookahead token is simply discarded.  If there is a symbol destructor associated with the token in question, then it will be called to allow any cleanup to be performed.  Whenever a token is discarded the service function yydiscard is called (single model C parser only, yymdiscard for a multiple model C parser, yyparser::yydiscard for a C++ parser). The default implementation in the YACC and Lex library does nothing.  You can redefine this function in your YACC source file if you wish to report when tokens are discarded.

3	If the parser has shifted one or two tokens onto the stack since entering error recovery, then the parser effectively attempts error recovery again.  That is it goes through the sequence of examining and popping states until an error token shift action is found.  This also means that the parser will end up back at the beginning, having to shift three consecutive tokens onto the stack.  The other one or two tokens are lost and do not count.  Note that it is this very step which ensures that the tokens are actually consecutive i.e. there cannot be an intervening discarded token.

As an example of how the parser would behave, consider the ExpressionStatement error recovery rule above, and the following erroneous C statement.  Note that a normal ExpressionStatement covers any C expression complete with terminating semicolon ';':

A = B!!;

The error occurs when the first pling ! is encountered by the parser.  In C, the pling ! character represents the logical not operator.  At this point the parser generates an error message and then starts examining and popping states until it encounters a state with an error token transition.  In this case, it is the one for the ExpressionStatement error recovery rule.  The parser then shifts the error token onto the stack and enters error recovery.  The parser now goes back to the start and looks for an action to perform.  Since the parser is now in a state where it is looking for a semicolon to shift onto the stack, the pling again causes a syntax error.  Because the parser is already in error recovery it does not generate a second error message.  The pling is now discarded by the parser because it matches point 2 above.  The parser now goes back to the start again and looks for an action to perform again.  Because there is no current lookahead token one is fetched by the parser.  This will be the second pling character.  Again this will be discarded like the first.  However, when the semicolon character is read by the parser, an action can be performed, and the semicolon is shifted onto the stack.  The error recovery rule is now complete and can be reduced.  Any action associated with the rule will then be performed.

Note though that this example represents an important point.  Although the error rule has been used to effectively re-synchronise the input, the parser doesn't know about this.  Under its rules it must shift three consecutive tokens onto the stack before it considers re-synchronisation to be complete.  So far it has shifted just one token, the semicolon.  It must shift two more tokens onto the stack, and they must be consecutive, before error recovery is complete.  There is a way out of this dilemma, and that is to use the yyerrok function (single model C parser only, yymerrok for a multiple model C parser, yyparser::yyerrok for a C++ parser).  The function is called from within the action associated with an error recovery rule, and informs the parser that error recovery is complete.  For instance, the ExpressionStatement error recovery rule might look something like:

ExpressionStatement: error ';' { yyerrok(); } ;

It is possible to implement error recovery rules which report when certain tokens are missing from the token stream.  These rules are somewhat different to the above, generalised, type of error recovery rules where symbols can be popped off of the stack, and lookahead tokens discarded.  In missing token error recovery rules, quite the opposite occurs.  Indeed, no symbols are popped off of the stack, and no tokens are discarded.  However, the rules need to be careful with the stack.  An example missing token error recovery rule is shown below.  Note that it makes use of a couple of functions to enable it to do this.  The rule handles a missing right parenthesis for an expression parser:

Expr:

    '(' Expr error

        {

            if (yypopping()) {

                yythrowerror(1);

            }

            else {

                printf("missing ')'\n");

            }

         }

Note that the first thing is a call to the yypopping function (single model C parser only, yympopping for a multiple model C parser, yyparser::yypopping for a C++ parser).  This is an important call.  It returns whether the parser has needed to pop states off of the stack in order to reach this error rule.  If so then a missing token error should not be reported.  After all, if states have been popped off of the stack, it implies that the point in the token steam where a token was potentially missing, has long since passed.  In order to reject this error recovery rule the yythrowerror function is called (single model C parser only, yymthrowerror for a multiple model C parser, yyparser::yythrowerror for a C++ parser).  An argument of 1 means that the parser will skip over an error token shift action (in this case the shift action which leads to this rule), before it will start to look for another.  The error will then, hopefully, be caught by a more general purpose, higher level error recovery rule later on.

Lastly, implementing an error recovery strategy is not an exact science.  It is more like trial and error.  Experiment to find out what strategies work in particular environments.  Also, generate a verbose file, using the -v Option,  to find out exactly where your error recovery rules come into play.  You may find that the parser is not doing exactly what you intended.  It is also worth noting that you can use the -R Option and -N Option to control the generation of default reductions in the parser's tables.  In particular these options modify the behaviour of the generated parser.  However these modifications only become evident when the parser encounters a syntax error.

See Also

Symbol Destructor Rule, Action Symbol


YACC Ambiguity Resolution
Ambiguity Resolution

YACC accepts ambiguous grammars.  For every grammar, YACC will attempt to create a parser for it.  Where grammars are too complex for YACC to handle, then warning messages will be generated indicating where the trouble occurs.  These warnings come in the form of conflicts, and there are three different types - shift-reduce, reduce-reduce and accept-reduce conflicts.  Note that the third form of conflict, the accept-reduce conflict, is extremely rare, and it is unlikely that you will ever see it.  The three different conflicts are explained below, along with how the conflict is resolved in order for YACC to generate a parser.

Note that the conflicts centre on the action table of the parser, and in particular on the addition of reduction actions to it.  The action table handles the shifting of tokens onto the stack, acceptance of input, and the reduction of grammar rules.  This table is distinct from the goto table which handles the shifting of nonterminal symbols onto the stack.  Note that this occurs as a result of a rule being reduced.  The goto table never contains conflicts.

Conflict warning messages are generated as part of the normal error reporting mechanism as well as in the verbose file.  This is a file which can be generated by specifying the -v Option on the command line.  The verbose file contains a list of all the rules in the grammar, as well as a list of all the states which make up the parser, complete with action table and goto table entries.  In order to properly track down and eliminate conflicts it is usually best to generate and examine the verbose file.  There is a summary of all conflicts at the bottom of the file, and each state shows which conflicts, if any, that it has together with the resolution of the conflict.

In some situations conflicts are unavoidable and/or intended.  For instance the C if-else statement example below.  However in most cases, it is desirable to eliminate all conflicts where possible to ensure that the grammar is unambiguous.  In those cases where ambiguities cannot be eliminated, it is necessary to check each conflict in turn, to make sure the parser is doing what is required.  Removing conflicts typically involves restructuring the grammar, however there are grammars which will remain ambiguous no matter how much they are massaged.

One of the limitations of YACC is that it can only generate a parser with one token of lookahead.  That is at any given point in the input it must decide whether to shift a token onto the stack, or reduce a rule, based on the topmost state on the stack, and the current lookahead token.  It is very rare that grammars require more than one token of lookahead.  In those cases, where they do, it is possible to simplify the grammar and push some of the extra processing down to the lexical analyser.  The simplified grammar will then gain new tokens which represent sequences of older tokens.  Note that this method does not lend itself to repetition.  The method does though highlight another way for resolving ambiguous grammars.  Instead of simplifying the grammar and moving the processing down to the lexical analyser, you can instead simplify the grammar and move the processing up to the semantic analysis phase - what goes on inside your actions.

Shift-Reduce Conflict

This conflict occurs when a state wants to both shift a token onto the stack and reduce a rule at the same time.  The conflict is resolved by always shifting the token onto the stack.  The commonly used example of a shift-reduce conflict is the if-else statement in C.  As the language is typically expressed, it is ambiguous as to which if statement an else statement belongs, for instance, consider the example below:

statement :

selection-statement

…

selection-statement :

if ( expression ) statement

if ( expression ) statement else statement

…

if (a)        statement 1

    if (b)    statement 2

        c;

    else      statement 3

        d;

The trouble occurs with the state which corresponds to the point where c has just been reduced to a statement nonterminal and the else token is the current lookahead token.  The parser can either shift the else token onto the stack, so associating statement 3 as the else branch for statement 2, or it can reduce statement 2 as a selection-statement nonterminal so associating statement 3 as the else branch for statement 1.  As already stated, a shift is preferred over a reduce, so in this case statement 3 is taken the else branch for statement 2, which is what we want.  Note that things may not always work out correctly, so always check any shift-reduce conflicts in the generated verbose file.

Reduce-Reduce Conflict

This conflict occurs when a state wants to reduce two different rules at the same time for the specified lookahead token.  The conflict is resolved by always reducing the rule that is listed first in the YACC source file.

Accept-Reduce Conflict

This conflict occurs when a state wants to both accept the input and reduce a rule at the same time for the specified lookahead token ($end).  The conflict is resolved by always accepting the input.  Note that this is an extremely rare conflict, and only presents itself under very obscure conditions.  For this type of conflict the lookahead token is always $end.  This token indicates to the parser that there are no more tokens to process.

Additional conflict resolution methods are also used if the rules and tokens which make up a grammar also have a precedence and associativity.  Additional resolution is only provided for shift-reduce conflicts, however this extra functionality allows the specification of operator precedence grammars.  These are an important class of grammars.  Their chief advantage being that they allow the construction of extremely fast parsers.  While these parsers have fewer states than a normal recursive descent parser, they are by no means smaller, since each state gains a larger number of shift/reduce actions.

For a shift-reduce conflict, if the token to be shifted onto the stack, and the rule to be reduced both have a precedence and associativity, then the conflict is resolved according to the following rules.  No shift-reduce conflict warning message is generated in this case.  Otherwise, the conflict is resolved according to the rule mentioned above concerning shift-reduce conflicts:

1	If the token to be shifted onto the stack has a higher precedence than the rule to be reduced, then the token is shifted onto the stack.  This is the default way that shift-reduce conflicts are normally resolved.

2	If the rule to be reduce has a higher precedence than the token to be shifted onto the stack, then the rule is reduced.

3	If both the token to be shifted onto the stack, and the rule to be reduced have the same precedence, then the conflict is resolved according to the associativity of the token and rule:

1	If the token is left associative, as specified using the %left keyword in a token declaration, then the rule is reduced.

2	If the token is right associative, as specified using the %right keyword in a token declaration, then token is shifted onto the stack.

3	If the token is non-associative, as specified using the %nonassoc keyword in a token declaration, then no action is taken i.e. there is no shift/reduce action for the specified lookahead token.

Note that in all of the above cases, because the rule to be reduced has the same precedence as the token, then by implication it also has the same associativity as well.  For more information see also Token Declaration.

Precedence and associativity are assigned to tokens using a Token Declaration.  That is, the tokens have been declared using one of the %left, %right or %nonassoc keywords.

Precedence and associativity are assigned to rules according to the first token on the right-hand-side of a rule.  That is the rule is assigned the precedence and associativity of that token.  If there are no tokens on the right-hand-side of a rule, then correspondingly, the rule has no precedence and associativity.  There is however a way to specify the precedence and associativity of a rule explicity, and that is to use a Precedence Symbol.  Basically, this will assign the precedence and associativity of the specified token to the rule.  The precedence symbol is useful when, in the case of specifying an operator precedence grammar, a token has more than one meaning.  For instance the minus sign '-' can represent both the subtraction operator, and the unary minus operator.  For more information see also Operator Precedence Grammars.

The use of precedence and associativity is not solely limited for the use of specifying operator precedence grammars.  It can also be used to remove shift-reduce conflicts from grammars.  This is not to say though that all shift-reduce conflicts should be removed in this way.  Far from it.  The example below again concentrates around the C if-else statement shift-reduce conflict on the else keyword.  In this case it is necessary to tell YACC that the shift operation is the required way to resolve the conflict.  There are a number of ways that this can be done, however the solution below relies on the fact that if the precedence for the rule to be reduced, and the token to be shifted onto the stack are the same, then the action performed is based on the associativity of the token.  In this case, since we want to force a shift, then the token is made right associative.  Conversely, if we had wanted to force a reduce, then we could have made the token left associative via the use of the %left keyword.  The first IF statement rule also has to be assigned the same precedence and associativity as the ELSE keyword.  This achieved by using the %prec keyword.  The result of all this is that when YACC comes across the shift-reduce conflict, it can resolve it.  The first IF statement rule, which is being reduced, has the same precedence and associativity as the token to be shifted onto the stack - the ELSE token.  Because this token is right associative, YACC knows that it needs to perform a shift to resolve the conflict.

%right ELSE

...

%%

...

SelectionStatement

    : IF '(' expression ')' Statement %prec ELSE

    | IF '(' expression ')' Statement ELSE Statement

    | ...

         ;

Note that although the second IF statement rule contains the ELSE keyword, it does not have a precedence and associativity.  The first token in the right-hand-side of the rule, which governs such things, is the IF keyword, which does not have a precedence and associativity.

See Also

Token Declaration, Precedence Symbol, Operator Precedence Grammars


YACC Programs Section
Programs Section

Syntax

programs-section :

program-code

Tokens

Program Code

Remarks

The programs section is where any functions used by the parser are typically placed.  It is an optional section and is indicated by the second %% mark in the YACC file.  Note that the Program Code token wraps both the %% mark and the trailing characters until the end-of-file character.

For a list of functions which can be used to operate on the parser, and any variables which can also be modified see the YACC And Lex Library.

See Also

Grammar


Multiple YACC Parsers
Multiple C Parsers

It is possible to have two or more C parsers in the same executable.  To do this you must use the multiple parser model when generating source code.  This model differs significantly from the single (default) parser model.  What it does is group the necessary variables, functions and tables from the single parser model into one single structure called yymparse_t.  Calls to the parser are then typically made with an extra pointer parameter identifying the parser in question.

Other versions of YACC take an alternative approach to implementing multiple parsers.  Instead of having a structure they allow you to change the "yy" prefix at the start of all function and variable names.  For instance you could change the prefix to "zz", and then call zzparse etc.  Naturally there are advantages and disadvantages to both approaches.

The multiple parser model has a different header file "myacc.h" to the single parser model's "yacc.h".  This file holds all declarations and definitions required by the parser.  All multiple parser names, both function and variable, have the prefix "yym" instead of "yy".  The notable exceptions are yydebug, yydebugflush and yydebugstack which apply to both the single and multiple parser models.  All multiple parser functions also take an additional parameter.  This is a pointer to the parser that the function is to be carried out on.  The parameter is always first and is of type yymparse_t *.

In order to distinguish one multiple parser from another it is necessary to give each parser a name, or identifier.  You can do this by using the %name keyword.  For more information see also Name Declaration:

%name myparser

The generated source code then contains a variable myparser of type yymparse_t.  To run the parser you would then call yymparse with the address of myparser as its parameter:

int main(void)

{

    return yymparse(&myparser);
}

If a header file has been generated then a declaration for the parser also appears in it.  An alternative to using %name is to define YYPARSENAME directly:

#define YYPARSENAME myparser

Note that in this case a generated header file will not contain any parser declaration.

When redefining multiple parser service functions be aware that the functions will be used by all multiple parsers.  There are of course ways of redefining these functions for individual parsers.  The yymparse_t structure has a number of pointer to function members, one for each of the service functions.  When a service function is called by the parser it consults one of these members.  In the declarations section of your YACC source file, you can define how each of these members is initialised.  For instance if you had written a new error handler then you would need to define YYMERROR as the name of the new function:

void myerror(yymparse_t *yy, const char *text)

{
    ...

}

#define YYMERROR myerror

The complete list of service functions, and the associated name to define is shown below:

Name	Service Function

YYMDISCARD	yymdiscard

YYMERROR	yymerror

YYMGETTOKEN	yymgettoken

YYMSTACKOVERFLOW	yymstackoverflow

YYMSYNTAXERROR	yymsyntaxerror

If you do not define the name, then the default service function is used.  Note that the replacement function must have the same prototype as the corresponding service function.  Alternatively you can assign service functions to the members at run time.  Each member has the same name as its functional equivalent i.e. there is a yymerror member for the function yymerror etc.

The multiple parser structure yymparse_t has a user data member yymdata.  It is a simple void * type.  You can specify the value of it at compile time by defining YYMPARSEDATA in your YACC source file:

#define YYMPARSEDATA mydata

Alternatively you can assign a value to it at run time.  The default version of yymgettoken makes use of yymdata.  It assumes that it points to a multiple lexical analyser.  The default implementation of yymgettoken looks like:

int yymgettoken(yymparse_t *yy)
{

    yymlex_t *yy = (yymlex_t *) yy->yymdata;

    return yymlex(yy);
}

To make use of this all you need to do is define YYMPARSEDATA to point to the multiple lexical analyser.  For instance, if you had a multiple lexical analyser called mylexer:

#include "mylexer.h"

...

#define YYMPARSEDATA &mylexer

The header file is generated by ALex, and contains the variable declaration for mylexer.  Incidentally, the name of the lexical analyser is specified in a similar way to the parser by using the %name keyword.

If you are moving a parser from the single model to the multiple model it is not necessary to change all of the names from their single to multiple equivalent.  For instance, if you use the function yyabort() in your actions, then the multiple equivalent would be yymabort(yy).  Where yy is the pointer to the multiple parser.  It is possible to include the header file <myconv.h> in your YACC source file to do just this.  A more important example than yyabort is yyerror.  This function is frequently used in YACC source files to output error messages.  In the multiple parser environment, however it is a service function, and is accessed through a pointer to function member in the yymparse_t structure.  To call this function is quite long winded (*yy->yymerror)(yy), again where yy is the pointer to the multiple parser.  The header file <myconv.h> will convert yyerror for you automatically.

Multiple Instance C Parsers

It is also possible to create multiple instances of a C parser.  The creation of such parsers is slightly more involved than a normal C parser.  For a start, the initial parser that is created is a template for other parsers which are created on the stack and/or heap.  Further more, all actions have to use the yy parameter passed to the wrapping yyparseaction function.  This identifies the parser that the action is being performed for, and is of type yymparse_t *.  Similar to multiple parsers it is possible to include a header file <miyconv.h> to convert single parser names to their multiple instance equivalent.  Note that you may find problems when you do this.  All functions assume that there is a yy parameter defined which points to the parser.  This is fine, if the function is being called from the action, however if it is from a sub-function, then trouble may arise.  The creation, and destruction sequence for a multiple instance parser is shown below.  For completeness, the code also includes the creation of a multiple lexical analyser as well:

#include "myparser.h"

#include "mylexer.h"

int parse(void)

{

    int n = 1;

    yymparse_t parser;

    if (yymcreateparse(&parser, &myparser)) {

        yymlex_t lexer;

        if (yymcreatelex(&lexer, &mylexer)) {

            parser.yymdata = &lexer;

            lexer.yymdata = parser.yymlvalptr;

            n = yymparse(&parser);

            yymdestroylex(&lexer)

        }

        yymdestroyparse(&parser);

    }

    return n;

}

The function parse() first defines two local variables.  The first, parser will hold our parser instance.  Similarly the second, lexer will hold our lexical analyser instance.  The functions yymcreateparse and yymcreatelex, then initialise these variables.  This also includes the allocation of any memory for stacks, attributes and buffers.  The following two lines are interesting.  Firstly, the address of the lexical analyser is assigned to parser.yymdata.  This is because the default yymgettoken service function is being used.  Which, as has already been pointed out requires the address of the multiple lexical analyser to be stored in yymdata.  The second line then assigns the address of yylval to lexer.yymdata.  This enables the lexical analyser to pass any attribute information back to the parser with the token.  Now that everything has been set up the parser can be called with yymparse.  Afterwards, the parser and lexical analyser instances are destroyed by calling yymdestroyparse and yymdestroylex.  It is not necessary to call yymparsecleanup, or yymlexcleanup.  This is handled automatically.

The lexical analyser gets access to yylval by casting its user data member yymdata to the required type.  In your Lex source file you can place the following code fragment to do this.  Note that it goes immediately after the first %% mark separating the declarations section from the rules section.  In the generated source code, the code fragment will be placed at the top of the action wrapping yylexaction function:

%%
%{
YYSTYPE *yymlvalptr = (YYSTYPE *) yy->yymdata;

yyassert(yymlvalptr != NULL);

#define yylval (*yymlvalptr)

%}

Note the definition of yylval.  This little trick enables you to switch between single and multiple model lexical analysers relatively easily.

See Also

Name Declaration


YACC Operator Precedence Grammars
Operator Precedence Grammars

YACC accepts operator precedence grammars.  These are a handy class of grammars that can be used to implement expression parsers.  One of the major differences between operator precedence grammars, and other (normal) grammars, is that operator precedence grammars are inherently ambiguous.  That is, on there own, a parser generated by YACC using these rules will contain conflicts, and will usually not evaluate expressions correctly.  The problem is solved by forcing YACC to resolve these conflicts in a controlled fashion.  For a general discussion on ambiguity resolution see also Ambiguity Resolution.

Besides this, there are a couple of other differences between operator precedence grammars and there counterpart unambiguous grammars that are worth noting.  Firstly, as you will see later, an operator precedence grammar is considerably smaller than an unambiguously specified equivalent grammar.  It also has the added advantage that you can easily insert new operators.  Secondly, the resultant parser generated from an operator precedence grammar is considerably faster than that generated by an unambiguous grammar.  In general, each rule in an operator precedence grammar takes a few shift/reduce actions to parse.  In comparison, a parser generated from an unambiguous grammar, although taking the same number of shift actions, can take a considerable number of reduce actions, especially if the grammar contains a large number of precedence levels.  Parsers generated from operator precedence grammars also have fewer states than their counterparts.  However this is not to say that the resultant parser is smaller, since the states contain more actions.

Below is an operator precedence grammar for a simple expression parser.  The grammar supports the addition, subtraction, multiplication and division operators, and grouping using parentheses:

Expression

    : Expression '+' Expression     { $$ = $1 + $3; }

    | Expression '-' Expression     { $$ = $1 - $3; }

    | Expression '*' Expression     { $$ = $1 * $3; }

    | Expression '/' Expression     { $$ = $1 / $3; }

    | '(' Expression ')'            { $$ = $2; }

    | NUMBER

    ;

Note that, on its own the grammar as it stands is ambiguous.  Indeed, if you entered the above grammar into YACC it would complain about a number of shift-reduce conflicts.  For instance consider the following example expression:

1 + 2 * 3

This can be interpreted as either (1 + 2) * 3 or 1 + (2 * 3), according to the above rules.  Obviously, if normal mathematical conventions are being followed, the second form is the correct one.  The trouble is, YACC does not know this and needs to be told explicitly.  This is done by attaching a precedence and associativity to each operator in the grammar.  This is done in the Declarations Section by using a Token Declaration.  Instead of tokens being declared with the %token keyword, they are declared with either %left, %right or %nonassoc.

The %left keyword specifies that the operator has left associativity, and correspondingly the %right keyword specifies that the operator has right associativity.  If we have a generalised expression:

x op y op z

Then depending upon the associativity of the operator op, the expression will be evaluated as:

Associativity	Evaluation Order

Left	(x op y) op z

Right	x op (y op z)

Most binary operators tend to be left associative, although note that the C assignment operators (=, +=, -= etc.) are right associative.  The associativity of an unary operator depends upon whether it is a pre or post-fix operator.  Pre-fix operators, that is those that go before an expression, are right associative, whilst post-fix operators, those that go after an expression, are left associative.

The %nonassoc keyword is a little more involved.  It states that the operators cannot appear next to each other.  There must be an intervening operator.  The example that is always drawn on is the FORTRAN equality operators.  In FORTRAN it is impossible to have:

IF A .EQ. B .EQ. C THEN

Compare this to the C if statement equivalent.  Although this is a valid C statement, it is probably not very useful:

if (a == b == c) {

Precedence is assigned to the operators according to the order in which they are declared.  Those that are declared first have the lowest precedence, and those that are declared last, the highest.  Note though that operators in the same token declaration all have the same precedence level.  Continuing the operator precedence grammar example, the token declarations for the grammar might look something like:

%left '+', '-'  // lowest precedence

%left '*', '/'  // highest precedence

Note that it is not necessary to define a precedence and associativity for the parentheses '(' and ')', and the NUMBER tokens.  In general, unary expressions can be left alone.  Saying that though, if an unary minus expression is added to the above grammar, a problem occurs.  Below is the added unary minus expression rule:

Expression:

    : '-' Expression                { $$ = -$1; }

    ;

The problem is best illustrated by an example.  Consider the following, where x and y represent some expression:

-x * y

You would normally expect this to evaluate as (-x) * y.  However, because the '*' operator has a higher precedence than the '-' operator, what you end up with is -(x * y).  This is not what is wanted.  Instead, it is necessary to make the unary minus operator unique, with a higher precedence than multiplication (and division).  You can do this by declaring a pseudo token, and then assigning the precedence and associativity of the token to the rule in question using a Precedence Symbol.  The declaration of the new pseudo token might look something like:

%right UNARY    // pseudo token

Note that the declaration must follow that for the multiplication and division operators so that it has a higher precedence.  It is also worth noting that the token has been declared right associative.  This is the natural associativity of an unary minus operator.  In the rules section, the follow modification would then be made to the unary minus expression:

Expression

    : '-' Expression %prec UMINUS   { $$ = -$2; }

    ;

Here the %prec keyword is forcing the '-' operator to assume the precedence and associativity of the UMINUS token.  Now when -x * y is parsed it is correctly evaluated as (-x) * y.  The unary minus operator now has a correct higher precedence.  Following is the full, YACC friendly, specification of the example grammar, complete with unary minus expression:

%left '+', '-'  // lowest precedence

%left '*', '/'  // highest precedence

%right UNARY    // pseudo token

%token NUMBER

%%

Expression

    : Expression '+' Expression     { $$ = $1 + $3; }

    | Expression '-' Expression     { $$ = $1 - $3; }

    | Expression '*' Expression     { $$ = $1 * $3; }

    | Expression '/' Expression     { $$ = $1 / $3; }

    | '(' Expression ')'            { $$ = $2; }

    | '-' Expression %prec UMINUS   { $$ = -$2; }

    | NUMBER

    ;

See Also

Ambiguity Resolution


Lex Language

Reference Topics

Introduction

Tokens

Grammar Summary

Grammar

Declarations Section

Declarations Section

Start Declaration

Table Declaration

Macro Declaration

Array Declaration

Option Declaration

Name Declaration

Include Declaration

Defines

Rules Section

Rules Section

Group Rule

Expression Rule

Left Context

Right Context Expression

Union Expression

Concatenation Expression

Repetition Expression

Primary Expression

Ambiguity Resolution

Programs Section

Programs Section

Quick Reference Topics

Multiple C Lexical Analysers

Operator Summary

ASCII Character Set


Lex Introduction
Introduction

Lex is a generalised language for creating pattern matching routines.  The patterns are specified using regular expressions.  In the broadest sense these consist of characters and a set of operators to work on them.  Each regular expression has a corresponding action associated with it.  When a string matching the regular expression is found the corresponding action is performed.

Lex was originally written by M. E. Lesk and E. Schmidt on a UNIX platform.  It is closely tied in with YACC (Yet Another Compiler Compiler).  A parser generating tool.  Since then there have been many flavours of Lex implemented.  Perhaps the most notable being Flex.

The lexical analysers generated by Lex are specified in another programming language.  The original Lex generated output for C and RATFOR, a language which can be automatically translated to FORTRAN.  Currently, ALex generates output for C and C++.  Note that whilst a C lexical analyser can be compiled in a C++ environment, the C++ lexical analyser itself makes use of classes.  There are other versions of Lex which generate output for other languages, including Ada, JAVA and Delphi etc.  The language that Lex generates output for is sometimes termed the target language.  Note that the actions associated with the regular expressions, and any other code which is copied verbatim into the output file must also be in the target language.

One major difference between ALex and Lex, and indeed other versions of Lex is that most of lexical analyser driving code is hidden in a library.  The generated output file contains the driving tables for the lexical analyser, definitions for the buffers, a wrapper function yylexaction which contains the actions, and any other code that is copied from the source file to the output file.  The format of the generated output file looks like:

declarations

buffer definitions

int yylexaction(int action)

{

actions

}

programs

lexical analyser driving tables

There are advantages and disadvantages to using a library to hold the lexical analyser driving code.  Placing the code in a library offers a considerable amount of flexibility, and means that you can tinker with how the lexical analyser itself behaves.  Whether you would want to do this or not is another matter.  Suffice to say that the opportunity is there if it is needed.  The major disadvantage to having the code in a library is the need to actually link the library in with the application you are developing.  This can be awkward if you are unfamiliar with the development environment, or you have to actually rebuild the libraries.  Library rebuilding is not particularly difficult these days, especially with the plethora of compiler development environments.  All that you generally need to do is create the relevant project, in this case a (static) library, add the library source files to it and perform a rebuild operation.


Lex Tokens
Tokens

The tokens recognised by Lex are listed below:

Keywords

Identifiers

Integers

Characters

Character Strings

Character Classes

Action

Separator

Comments

Program Code

Code Block

Code Line

Macro Name

Translation String

Macro Reference

Declaration Code Block

Other Tokens


Lex Keywords
Keywords

A Lex keyword starts with a percent sign % and has one or more trailing letters.  The percent sign is a special character in Lex and is referred to as the Lex escape character.  It is used to introduce a number of tokens.  The currently recognised keywords are:

%include, %exclusive, %name, %start, %table

Compatibility

The case of the letters in a keyword is not important.  For instance %start, %Start and %START all specify the %start keyword.  The following keywords are also recognised by Lex:

%a, %e, %k, %n, %o, %option, %p

There are also a number of synonyms for the Lex keywords:

Keyword	Synonym

%exclusive	%x

%start	%s, %st, %sta, %star

%table	%t

Note that this also includes all upper and lower case letter combinations of the keywords.


Lex Identifiers
Identifiers

A Lex identifier starts with a letter or underscore and has zero or more trailing letters, digits and underscores.  Case is important, so the identifier "AA" is different to "aa".  All characters in an identifier are significant.  The only limiting factor is the size of the line buffer used by the compiler.

Identifiers are used to represent start states.  An identifier is also very similar to a Macro Name.  The only difference being that a macro name starts at the beginning of a line (in the declarations section).  It is possible to declare start states and macros with the same name.  The example below declares the start state letter and the macro letter:

%start letter

letter [a-z][A-Z]

Whether this is a good idea or not is another matter.

See Also

Macro Name


Lex Integers
Integers

There are three types of integers.  Octal, decimal and hexadecimal.  Octal integers start with a 0 and have zero or more trailing octal digits (0 - 7).  A decimal integer starts with any digit except zero (1 - 9) and has zero or more trailing digits (0 - 9).  A hexadecimal integer starts with either 0x or 0X and has one or more trailing hexadecimal digits (0 - 9, a - f, A - F).  Below are some examples of integers:

0       // zero

0713    // octal

99      // decimal

0x7f    // hexadecimal

Integers are used for defining the values in the {…} range operator.  They are stored internally as 32-bit unsigned integers.

See Also

Repetition Expression


Lex Characters
Characters

Any character in a regular expression which does not introduce another token is considered as a regular expression character.  The regular expression character has the ASCII value of the character in question, unless of course the values have been redefined by a Table Declaration.

In order to specify certain characters it is necessary to prefix them with the backslash \ escape character.  For instance:

\+  { /* matches a plus sign */ }

Normally the plus sign + specifies the positive closure operator.  Most characters can be escaped in this way.  However certain escape sequences have special meanings:

Name	Character	Escape

New-line	NL (LF)	\n

Horizontal tab	HT	\t

Vertical tab	VT	\v

Backspace	BS	\b

Carriage return	CR	\r

Form feed	FF	\f

Audible alert	BEL	\a

Backslash	\	\\

Octal number	ooo	\ooo

Hex number	hh	\xhh

The octal escape sequence can have one, two or three octal digits (0 - 7). The hexadecimal escape sequence can have one or more hexadecimal digits (0 - 9, a - f, A - F).  In either case the value of the number must be between 0 and 255.  An escaped character cannot cross a line boundary.  In order to specify the newline character, use the newline escape sequence \n.

In general it is recommended that any character which is not a letter, digit or underscore should be escaped.  This is to protect against future enhancements.  For completeness, the list of characters which introduce other tokens is shown below:

(  )  +  *  ?  {  }  [  ]  .  "  |  ^  <  >  /  $

For compatibility with Lex, the list includes the right bracket ].  In ALex this is treated as a regular expression character.  Remember also that a space (or tab) introduces an action.

See Also

Primary Expression


Lex Character Strings
Character Strings

A character string is a sequence of characters surrounded by double quotes, as in "...".  A character string can consist of one or more characters.  In order to specify certain characters in a character string it is necessary to use the backslash \ escape character.  For instance in order to be able to specify a double quote in a character string it is necessary to escape it \".  Most characters can be escaped in this way, however certain escape sequences have special meanings:

Name	Character	Escape

New-line	NL (LF)	\n

Horizontal tab	HT	\t

Vertical tab	VT	\v

Backspace	BS	\b

Carriage return	CR	\r

Form feed	FF	\f

Audible alert	BEL	\a

Backslash	\	\\

Octal number	ooo	\ooo

Hex number	hh	\xhh

The octal escape sequence can have one, two or three octal digits (0 - 7).  The hexadecimal escape sequence can have one or more hexadecimal digits (0 - 9, a - f, A - F).  In either case the value of the number must be between 0 and 255.  A character string cannot cross a line boundary.  In order to specify the newline character, use the newline escape sequence \n.

See Also

Concatenation Expression


Lex Character Classes
Character Classes

A character class is a sequence of characters surrounded by brackets, as in [...].  Character classes provide a notational shorthand for the Union operator.  See also Union Expression.  The character class [abc] represents the regular expression a|b|c.  The minus sign - is special in character classes.  It indicates a range of characters.  The character class [a-z] represents a regular expression matching a lower case letter.  If the characters denoting the range are in the wrong order then they are automatically swapped around.  If the minus sign cannot be associated with two range defining characters then it is taken to be a normal character.  The character class [+-] represents the characters + and - since there is no character to the right of the minus sign.  It is possible to specify several ranges in a character class.  For instance, the character class [a-zA-Z0-9_] specifies the characters that can follow an initial C identifier character.  The escape character \ can be used to include characters like - and ] in the character class.  Most characters can be escaped in this way, however certain escape sequences have special meanings:

Name	Character	Escape

New-line	NL (LF)	\n

Horizontal tab	HT	\t

Vertical tab	VT	\v

Backspace	BS	\b

Carriage return	CR	\r

Form feed	FF	\f

Audible alert	BEL	\a

Backslash	\	\\

Octal number	ooo	\ooo

Hex number	hh	\xhh

The octal escape sequence can have one, two or three octal digits (0 - 7).  The hexadecimal escape sequence can have one or more hexadecimal digits (0 - 9, a - f, A - F).  In either case the value of the number must be between 0 and 255.  A character class cannot cross a line boundary.  In order to specify the newline character, use the newline escape sequence \n.

As well as specifying the characters in a class, it is also possible to specify those that aren't.  This is done by following  the opening bracket with a circumflex character ^.  The character class [^abc] represents all characters except a, b and c.  The character ^ can be escaped in the normal manner, however it is only necessary if it is the first character in the class.

There is a special character class which is represented by a period ..  It means match any character except newline.  Represented by a normal character class it would appear as [^\n].

It is also possible to use character class expressions in character classes.  Note that they cannot be used outside of character classes.  They are surrounded by [: and :] and specify sets of commonly used characters.  For instance, the character class expression [:alpha:] specifies all alphabetic characters, or letters.  Character class expressions can be used alongside other characters and character ranges inside a character class.  For example, the regular expression below specifies a C identifier:

[[:alpha:]_][[:alpha:]0-9_]*    { /* match a C identifier */ }

Each character class expression typically has a function in the C run-time library which describes the characters within it.  For instance in the case of [:alpha:], the corresponding function would be isalpha.  The list of character class expressions supported by ALex, together with the corresponding C run-time library function which specifies it, and a description is shown below:

Expression	Function	Description

[:alnum:]	isalnum	Letters and decimal digits.

[:alpha:]	isalpha	Letters.

[:blank:]	-	Horizontal tab '\t' (0x09) and space ' ' (0x20).

[:cntrl:]	iscntrl	Characters 0x00 (NUL) to 0x1F (US), and 0x7F (DEL).

[:digit:]	isdigit	Decimal digits.

[:graph:]	isgraph	Printable characters excluding space.  These are the characters 0x21 '!' to 0x7E '~'.

[:lower:]	islower	Lower-case letters.

[:print:]	isprint	Printable characters including space.  These are the characters 0x20 ' ' to 0x7E '~'.

[:punct:]	ispunct	Printable characters excluding space, letters or decimal digits.

[:space:]	isspace	Horizontal tab '\t' (0x09), new-line '\n' (0x0A), vertical tab '\v' (0x0B), form feed '\f' (0x0C), carriage return '\n' (0x0D) and space ' ' (0x20).

[:upper:]	isupper	Upper-case letters.

[:xdigit:]	isxdigit	Hexadecimal digits.

Note that the character class expression [:blank:] does not have a corresponding C run-time library function.  It is not possible to use a character class expression as part of a character class range.  For instance the character class below:

[a-[:digit:]]

will be considered as the letter 'a', the minus sign '-', and the digits '0' to '9'.  No error message is generated.

See Also

Union Expression


Lex Action
Action

An action starts with the first space or tab following a regular expression.  Following can be zero or more trailing white space characters, including the newline character.  If the next character is a | character, then the token is a Separator token.  Otherwise, it is an action proper.

If the character is a left brace {, then all succeeding characters up to the terminating right brace } are copied into the output file.  If the character is not a left brace, then the character, and all succeeding characters up to the terminating semicolon ; are copied into the output file.  The copied code must be written in C (or C++).  No checks are made on the actions to ensure that they are correct.  This is the job of the compiler which will subsequently use the output file.

auto    { return AUTO; }

break   return BREAK;

An action which starts with a left brace { is aware of certain C and C++ constructs.  In particular it is aware that the braces { and } are used for forming compound statements.  As such the action keeps track of the number of left and right braces that it has seen.  Only when the counts are the same, and the closing brace encountered, is the action considered complete.

An action is also aware of certain C and C++ tokens.  These are string constants, character constants and comments (both line and block).  For both string and character constants, the action is aware of the backslash \ escape character.  If any braces { and } or semicolons ; are encountered inside any of these tokens then they are ignored i.e. they are not added to the running count in the case of braces, or treated as a terminating character.

It is possible to use a C and C++ return statement inside an action.  If you do so it must return an int value.  Typically, this is used to pass information from the lexical analyser back to the calling function.  This is what is happening in the auto and break rules above.  The lexical analyser itself uses a return value of EOF (-1) to indicate that there is no more input to process, so it is generally best to avoid using this value.  That is unless you want to indicate that there is no more input.  Furthermore, if the lexical analyser is being called by a YACC generated parser, then a return value of 0 or a negative number indicates the done token to the parser.

The YACC And Lex Library provides a number of functions that can be used with the lexical analyser.  The functions which are commonly used within actions are listed below:

Function	Description

yybegin	Places the lexical analyser in the requested start state.

yyecho	Echoes the matched string.

yyless	Backs up the matched string to the requested length.

yymore	Concatenates the matched string with the next one.

yyreject	Backups up the matched string, and attempts to match a shorter string.

See Also

Separator


Lex Separator
Separator

A separator starts with the first space or tab following a regular expression.  Following can be zero or more other white space characters, including the newline character.  The terminating character is the | character.  Note that if any other character is encountered, then the token would be an Action.  Separators are used for making numerous regular expressions share the same action.  For instance:

[0-7]+             |

[1-9][0-9]*        |

0[Xx][0-9A-Fa-f]+  { yylval.symbol = installinteger(); }

Here we have three regular expressions which match the C octal, decimal and hexadecimal integers.  They are all sharing a common action which installs the integer into a symbol table.  Note that the expressions are missing an optional type i.e. ul, l, u etc.

See Also

Action


Lex Comments
Comments

There are two ways to specify comments in Lex.  One is block based the other line based.  A block comment starts with /* and ends with */.  Any characters in between, including the newline are considered as comment.  The line based comment starts with // and is terminated, naturally with a newline.  Note that comments are not copied to the output file.


Lex Program Code
Program Code

In the rules section, the character sequence %% indicates the start of program code.  Any characters between here and the end-of-file are copied into the output file.

See Also

Programs Section


Lex Code Block
Code Block

A code block is delimited by %{ and %}.  Any characters inside are copied to the relevant point in the output file.

%{

/* comment */

int variable;

void function(void);

%}

A code block is recognised in both the declarations and rules sections.  In the rules section, if the code block occurs before any action, then the characters are copied into the top of the yylexaction function.  Remember that this function is called whenever a regular expression is matched and an action is performed.  Otherwise the characters are copied after the last action that was seen.  Since each action is contained in a case statement, with a following break statement, any code copied after it will not be run.  There are ways around this, like using a goto statement, but this sort of defeats the object.

A code block is aware of certain C and C++ tokens.  These are string constants, character constants and comments (both line and block).  For both string and character constants, a code block is aware of the backslash escape character.  If any %} character sequences are encountered inside any of these tokens then they are ignored.

Compatibility

In Lex, code placed at the start of the rules section is copied into the top of the generated yylex function.   This is different to ALex where it is copied into the top of the yylexaction function.

See Also

Defines


Lex Code Line
Code Line

Any line in the declarations or rules section that starts with a space or tab and is not part of another token is copied into the output file.

    /* this is a comment */

In the rules section, if the line code occurs before any action, then the characters are copied into the top of the yylexaction function.  Remember that this function is called whenever a regular expression is matched and an action is performed.  Otherwise the characters are copied after the last action that was seen.  Since each action is contained in a case statement, with a following break statement, any code copied after it will not be run.  There are ways around this, like using a goto statement, but this sort of defeats the object.

See Also

Defines


Lex Macro Name
Macro Name

A macro name starts with a letter or underscore '_', and has zero or more trailing letters, digits and underscores.  So far, this is very similar to an identifier, however a macro name must start in the first column of a line.  Furthermore a macro name is only recognised in the declarations section of a Lex file.  All characters in a macro name are significant.  Case is also important, so the macro name exponent is different to the macro name Exponent.

Since macro names and identifiers (start states) use different symbol tables, it is possible to declare a macro name and an identifier which are the same.  For instance:

%start digit

digit [0-9]

Here we have declared the start state digit and the macro digit.

See Also

Macro Declaration, Identifiers


Lex Translation String
Translation String

A translation string starts with the first character following a Macro Name, and terminates with a newline character.  Any white space surrounding the translation string is then removed to give the final translation string.  For our purposes white space consists of the space, tab and newline characters.  Note that it is possible for the resultant translation string to be empty.  Translation strings are only used for when defining macros.

See Also

Macro Declaration


Lex Macro Reference
Macro Reference

A macro reference can only occur in a regular expression.  A macro reference is introduced with a left bracket {.  It then has a following letter or underscore '_', followed by zero or more letters, digits or underscores.  This specifies the name of the macro which is being referenced.  A macro with this name should have been previously defined in the declarations section.  Finally, the macro reference is terminated with a right bracket }.

When a macro reference is encountered, the translation string associated with the macro is substituted for the reference.  The resultant (modified) line is then re-scanned at the substitution point.  If any more macro references are encountered, then the relevant substitutions are made.  Note that ALex does not (currently) have any recursive loop detection logic.  However, in most cases if this occurs, the error will be detected when the line buffer overflows.  There are instances however where this will not happen.  Consider the following macro definition and reference:

trouble {trouble}

%%

{trouble}  { /* whoops! */ }

Because the macro reference is being replaced with exactly the same text, the line buffer can never overflow.  ALex will be locked in a loop, forever replacing the same piece of text over and over again.

See Also

Macro Declaration


Other Lex Tokens
Other Tokens

The other tokens recognised by Lex are:

%%  *  ?  +  (  )  ,  {  }  .  $  /  ^  |  <  > <<EOF>>

Note that the <<EOF>> token includes all upper and lower case combinations of the EOF text.  For instance, the strings "<<eof>>", "<<Eof>>" and "<<EOF>>" all specify the <<EOF>> token.


Declaration Code Block

A declaration code block is delimited by brackets { and }.  Any characters inside are copied to the relevant point in the output or include file.  The destination will depend upon the context in which the declaration code block is being used and whether or not an include file is being generated.  A declaration code block is recognised in the declarations section of the Lex source file, and in some places in the rules section.  Syntactically, though it is only used in the declarations section.

A declaration code block is aware of certain C and C++ constructs.  In particular it is aware that the brackets { and } are used for forming compound statements and structures etc.  As such a declaration code block keeps track of the number of left and right brackets that it has seen.  Only when the counts are the same, and the closing bracket encountered, is the declaration code block considered complete.

A declaration code block is also aware of certain C and C++ tokens.  These are string constants, character constants and comments (both line and block).  For both string and character constants, a declaration code block is aware of the backslash \ escape character.  If any brackets { and } are encountered inside any of these tokens then they are ignored i.e. they are not added to the running count, or treated as a terminating character.

See Also

Name Declaration, Include Declaration


Lex Grammar Summary
Grammar Summary

grammar :

declarations-section rules-section [programs-section]

Declarations Section

declarations-section :

[declaration-list]

declaration-list :

declaration-list declaration

declaration

declaration :

start-decl

table-decl

macro-decl

array-decl

option-decl

name-decl

include-decl

start-decl :

start-keyword init-start-list

start-keyword :

%start

%exclusive

init-start-list :

init-start-list init-start

init-start

init-start :

identifier

table-decl :

%table table-item-list %table

table-item-list :

table-item-list table-item

table-item

table-item :

integer character-string

macro-decl :

macro-name translation-string

array-decl :

array-keyword integer

array-keyword :

%a

%e

%k

%n

%o

%p

option-decl :

%option [option-list]

option-list :

option-list identifier

identifier

name-decl :

%name identifier [declaration-code-block] [declaration-code-block]

include-decl :

%include declaration-code-block

Rules Section

rules-section :

%% [rule-list]

rule-list :

rule-list rule

rule

rule :

group-rule

expression-rule

group-rule :

[left-context] { [rule-list] }

expression-rule :

expression separator expression-rule

expression action

left-context :

< start-list > [left-context]

< * > [left-context]

^ [left-context]

start-list :

start-list , start

start

start :

identifier

expression :

[left-context] right-context-expression

right-context-expression :

[union-expression] / union-expression

[union-expression] $

union-expression

<<EOF>>

union-expression :

concatenation-expression

union-expression | concatenation-expression

concatenation-expression :

repetition-expression

concatenation-expression repetition-expression

repetition-expression :

primary-expression

repetition-expression *

repetition-expression +

repetition-expression ?

repetition-expression range

range :

{ integer , [integer] }

{ integer }

primary-expression :

character

character-string

character-class

.

( union-expression )

Programs Section

programs-section :

program-code


Lex Grammar
Grammar

Syntax

grammar :

declarations-section rules-section [programs-section]

Rules

Declarations Section, Rules Section, Programs Section

Remarks

A Lex source file consists of three distinct sections: the declarations section, rules section and progams sections.  All three sections are separated by the character sequence %%, sometimes called a mark.

declarations

%%

rules

%%

programs

The second mark, and the following programs are optional.  Typically any declarations go in the declarations section, the regular expressions and their associated actions in the rules section and any user routines go in the (optional) programs section.

Note that the contents of the declarations and rules sections are optional.  As already stated, the second mark and subsequent programs section is also optional.  This leaves the minimum Lex source file as simply:

%%

The lexical analyser generated from such a file is very simply (if non-existent).  All it will do is copy the input stream to the output stream.  It will never match any regular expressions.

See Also

Grammar Summary, Code Block


Lex Declarations Section
Declarations Section

Syntax

declarations-section :

[declaration-list]

declaration-list :

declaration-list declaration

declaration

declaration :

start-decl

table-decl

macro-decl

array-decl

option-decl

name-decl

include-decl

Rules

Start Declaration, Table Declaration, Macro Declaration, Array Declaration, Option Declaration, Name Declaration, Include Declaration

Remarks

The declarations section is where any information is set-up prior to defining the regular expressions for the lexical analyser.  You can declare functions and define variables which will be used in actions by placing the relevant code in a Code Block, Code Line or Include Declaration.

If the lexical analyser makes use of start states, then these can be defined using a Start Declaration.  If it is necessary to change the value associated with characters (i.e. the target application will not be using the ASCII character set), then you can do this using a Table Declaration.  To simplify regular expressions you can define macros using a Macro Declaration.  If you are building a C++ lexical analyser, or a multiple model, or multiple instance model C lexical analyser, then you can specify the name of the lexical analyser using a Name Declaration.

For compatibility purposes the Array Declaration and Option Declaration declarations are provided.  These support the %option and %a, %e, %k, %n, %o, %p keywords used by other versions of Lex.

See Also

Grammar, Code Block, Code Line


Lex Start Declaration
Start Declaration

Syntax

start-decl :

start-keyword init-start-list

start-keyword :

%start

%exclusive

init-start-list :

init-start-list init-start

init-start

init-start :

identifier

Tokens

Identifiers

Remarks

This declaration declares a lexical analyser start state.  There are two types of start state - exclusive start states and non-exclusive start states.  Exclusive start states are declared using the %exclusive keyword.  Correspondingly, non-exclusive start states are declared using the %start keyword.  Start states are used for implementing left context in regular expressions.  For more information on how they are used in regular expressions see also Left Context.  The example below declares the two (non-exclusive) start states AA and BB, and the exclusive start state CC:

%start AA BB

%exclusive CC

It is not possible to re-declare a start state with a different exclusivity.  For instance, if the start state AA was declared as being exclusive (with the %exclusive keyword), it is not possible to subsequently re-declare it as being non-exclusive, or vice-versa.

The start state INITIAL is pre-defined by Lex.  This is the default start state that the lexical analyser generated by Lex starts up in.  The start state INITIAL is a non-exclusive start state.  It is not possible, as implied above, to re-declare it as an exclusive start state.

The identifiers specified in a start state declaration are copied character for character into the output file as part of a #define statement.  Note that the start state INITIAL is also output as a #define statement.  Be careful of the naming of start states.  For instance, the start state declaration:

%start NULL

generates, assuming it is the first start state declaration, the code:

#define NULL 2

This is potentially disastrous, since the identifier NULL is already defined by the standard C library.  Note that the Lex header files lex.h and mlex.h include the standard C library header file stdio.h by default.

The current start state for the lexical analyser can be changed by calling the function yybegin.  This is typically done from within an Action.  The start state to change to (it's #define equivalent) is passed as a parameter.  The value 0 can be used as an alternative to specifying INITIAL.

Compatibility

The %start keyword has a number of synonyms: %s, %st, %sta, %star.  The %exclusive keyword has the synonym %x.  Note that this also includes all upper and lower case combinations of the keywords.

See Also

Declarations Section, Left Context, Action


Lex Table Declaration
Table Declaration

Syntax

table-decl :

%table table-item-list %table

table-item-list :

table-item-list table-item

table-item

table-item :

integer character-string

Tokens

Integers, Character Strings

Remarks

This declaration allows the value associated with characters in a regular expression to be changed.  This applies to all characters including those defined in character strings and character classes.  By default the value of a character in a regular expression is the same as its ASCII value.  So for example the character 'A' has the value 65, its ASCII value.

Under normal circumstances this is all well and good, however situations may arise where the character 'A' does not map onto the value 65.  For instance, consider the problem where the code for a lexical analyser is to be compiled on a machine which uses the EBCDIC character set.  The Extended Binary-Coded Decimal Interchange Code.  On such a platform the character 'A' has the value 193 (decimal).

A table declaration can be used to enable a lexical analyser to be written easily for an alternate character set.  The mappings for an EBCDIC character set might look something like:

%table

193     "A"

194     "B"

195     "C"

...

%table

Each table entry consists of an integer followed by a character string.  In the example above, each character is mapped onto a unique integer.  It is however possible to map several characters to the same integer.  Whether this is sensible or not is another matter.  It is simply possible.  For example:

%table

65 "ABC"

97 "a"

97 "b"

96 "c"

%table

Whether this is sensible or not is another matter.  It is simply possible.

If a value is not defined for a character in a table declaration, then it assumes its default ASCII value.  It is possible to have more than one table declaration in a Lex source file.

Compatibility

The %table keyword has the keyword %t as a synonym.  In fact in other versions of Lex, %table is not recognised as a keyword at all.  Note that in all cases, all upper and lower case letter combinations of the two keywords can be used i.e. %Table, %T etc.

Also in other versions of Lex, entries in the table declaration have to be on their own line.  Further more, the character string following the integer is not delimited by quotes.  In fact it is not considered as a character string at all.  Hence the EBCDIC example above, with keyword replacement, might appear as:

%t

193     A

194     B

195     C

...

%t

Although this incompatibility remains, table declarations are rarely used in practice.  The facility is simply provided if it is required.

See Also

Declarations Section, Character Classes, Characters


Lex Macro Declaration
Macro Declaration

Syntax

macro-decl :

macro-name translation-string

Tokens

Macro Name, Translation String

Remarks

This declaration declares a macro. A macro declaration consists of a macro name followed by a translation string.  For instance the macro digit below defines the digits 0-9:

digit   [0-9]

It is not possible to re-declare a macro, even if the translation strings are the same.  Macros can only be referenced from within regular expressions.  They are used for simplifying the implementation of regular expressions.

See Also

Declarations Section, Macro Reference


Lex Array Declaration
Array Declaration

Syntax

array-decl :

array-keyword integer

array-keyword :

%a

%e

%k

%n

%o

%p

Tokens

Keywords, Integers

Remarks

This declaration is implemented for compatibility purposes only.  The declaration is used to specify the size of internal arrays.  The keyword specifies the array type, and the integer its size.  In other versions of Lex, these are used to store information when calculating the tables for the lexical analyser.  Because ALex uses a more dynamic approach it is not necessary to do this.  The array which each keyword refers to is described briefly below:

Keyword	Description

%a	Positions.

%e	States.

%k	Tree nodes.

%n	Transitions.

%o	Packed character classes.

%p	Output array size.

See Also

Declarations Section, Option Declaration


Lex Option Declaration
Option Declaration

Syntax

option-decl :

%option [option-list]

option-list :

option-list identifier

identifier

Tokens

Keywords, Identifiers

Remarks

This declaration is implemented for compatibility purposes only.  The declaration is used to specify what functions should not be generated for the lexical analyser.  In other versions of Lex, all functions required by the lexical analyser are placed in the generated output file.  Therefore, if you want to redefine a lexical analyser function, it is necessary to tell Lex not to generate the default implementation.  For instance, the following sequence would be used to redefine the function yywrap:

%option yywrap

int yywrap(void)
{

    ...
}

Because all of the functions required by the ALex generated lexical analyser reside in the YACC and Lex library, it is no longer necessary to do this.  All you have to do is simply redefine the function in your source code.  At the linking phase, when the linker encounters two symbols with the same name it will take the symbol defined in your source code as a default.  The second symbol, in the YACC and Lex library, will simply be ignored.

See Also

Declarations Section, Array Declaration


Lex Name Declaration
Name Declaration

Syntax

name-decl :

%name identifier [declaration-code-block] [declaration-code-block]

Tokens

Identifiers, Declaration Code Block

Remarks

This declaration declares the name of the lexical analyser.  It is not necessary to do this if you are generating a single model C lexical analyser.  However if you are generating a multiple, or multiple instance model C lexical analyser, or a C++ lexical analyser then you will need to do so.  The name declaration below declares the name of the lexical analyser as mylexer:

%name mylexer

For a multiple or multiple instance model C lexical analyser, the name specifies the name of a yymlex_t structure generated in the output file.  If you have generated an include file as well, then a corresponding declaration for the structure will appear within it.  As an alternative to using a name declaration, you can duplicate its effect by defining YYLEXNAME instead.  The two Include Declarations below replicate the effects of using a name declaration.  The first is for a multiple model C lexical analyser, and the second is for a multiple instance model C lexical analyser:

%include {

extern yymlex_t YYNEAR name;

#ifndef YYLEXNAME

#define YYLEXNAME name

#endif

}

%include {

extern YYCONST yymlex_t YYNEAR name;

#ifndef YYLEXNAME

#define YYLEXNAME name

#endif
}

Where name is the name of the lexical analyser.  Note that the only difference between the two include declarations is the presence of the YYCONST modifier for multiple instance model lexical analysers.  The structure generated for a multiple instance model lexical analyser is used as a template for creating other lexical analysers, and as such is never modified.

For a C++ lexical analyser, a declaration name specifies the name of a new C++ class.  This new class will be derived from one of the existing YACC and Lex library classes.  For instance, if you have generated a default (fast) lexical analyser, then the new class will be derived from yyflexer.  Otherwise, if you have generated a compact lexical analyser it will be derived from yyclexer.

If you generated an include file for your lexical analyser, then the definition for the new class will be placed within it.  Otherwise it will be placed within the output file.  A default constructor is always generated for the new class.  The definition of which is always placed in the output file.

It is possible to place additional member functions and member variables in the new class.  You can do this by placing a Declaration Code Block after the name declaration.  The contents of the declaration code block will be copied, verbatim, into the class definition.  By default, the members will be declared as public.  You can of course change this by placing the relevant private / protected declarations in the class.

It is also possible to place additional code in the default constructor for the new class.  In this case, it is the presence of a second declaration code block which provides the source.  The contents of the code block will be copied, verbatim, into the constructor.  Note that the default constructor calls the member function yytables.  This function initialises the table array member variables for the class with their required values.  If you add other constructors to the class, be sure to call this function, otherwise the lexical analyser will not work.  For instance, the name declaration below will place an int member variable in the class mylexer, and initialise it to zero in the constructor:

%name mylexer

// class definition

{

    int m_count;    // add an m_count member variable...
}

// constructor

{

    m_count = 0;    // and initialise it to zero
}

Similar to multiple model and multiple instance model C lexical analysers, it is also possible to define YYLEXNAME as the name of the new C++ class.  However, in order to do this you will also need to generate a class definition, and a constructor.  The include declaration and Code Block below replicate the effects of using a name declaration:

%include {

class YYFAR name : public yylexer {

public:

    name();

protected:

    void yytables();

    virtual int yyaction(int action);

public:

    other class definition code…

};

#ifndef YYLEXNAME

#define YYLEXNAME name

#endif

}

%{

YYLEXNAME::YYLEXNAME()

{

    yytables();

    other constructor code…

}

%}

Where name is the name of the lexical analyser, and yylexer is the base class of the lexical analyser.  This will either be yyflexer or yyclexer, depending upon whether you have generated a default (fast) lexical analyser or a compact lexical analyser.  The member functions yytables and yyaction are automatically generated by ALex.  Obviously, it is much easier to use a name declaration, rather than typing in the new class by hand.

It is only possible to specify one name declaration in a Lex file.  If you attempt to use more than one name declaration, then an error will be generated.

If you specify a name declaration for a single model C lexical analyser, then it is ignored.  If any declaration code blocks are placed after the name declaration, then the first block is copied into the generated include file, or the output file if no include file is generated.  The second block, if any, is always copied into the output file.

If a name declaration for a multiple or multiple instance model C lexical analyser has any following declaration code blocks, then they are copied into the generated include and output files as specified for single model C lexical analyser's above.

See Also

Declarations Section, Include Declaration, Code Block, Defines, yymlex_t, yyflexer, yyclexer


Include Declaration

Syntax

include-decl :

%include declaration-code-block

Tokens

Declaration Code Block

Remarks

An include declaration places the code specified in the Declaration Code Block into the generated include file.  If an include file is not being generated, then the code is placed in the output file instead.  For example, the include declaration below places a declaration for the function myfun in the generated include file:

%include {

    void myfun(void);
}

In order to generate an include file it is necessary to specify the -i option when running the ALex tool.  You can do this from Parser Generator by selecting the Generate Include File option in the ALex Options dialog box.  You can reach this dialog box by first choosing the Project command from the Options menu, and then clicking the ALex button in the Project Options dialog box.

See Also

Code Block, Code Line


Lex Defines
Defines

It is possible to define a number of symbols in the output file which will affect how the lexical analyser behaves.  This is done using the C preprocessor #define statement:

#define name value

Typically you will place such definitions in a Code Block in the declarations section of your Lex source file.  It is possible to place some definitions elsewhere, however it is recommended that you place them all in the declarations section.

A list of the symbols that can be defined, together with their descriptions is shown below.  Note that they have been split up into a number of different groups.  The first is for all lexical analysers, the second for C++ lexical analysers only, and the last two for C multiple model lexical analysers.

All Lexical Analysers

Name	Description

YYTEXT_SIZE	Defines the size of the initial (static) text buffer. This also includes the accompanying state buffer.  If you do not define YYTEXT_SIZE, then it is assigned an initial value of 100.  The specified size does not include the NUL terminating character.  So in the default case the text buffer would 101 characters in size.  The state buffer on the other hand would be 100 characters.  After all it does not require a NUL terminating character.

YYUNPUT_SIZE	Defines the size of the initial (static) unput buffer.  If you do not define YYUNPUT_SIZE then it assumes the same value as that defined for YYTEXT_SIZE.

C++ Lexical Analysers

Name	Description

YYLEXNAME	Defines the name of the lexical analyser.  You must define this symbol if you are building a C++ lexical analyser.  It is however much more preferable to use a Name Declaration to this for you.  With a C++ lexical analyser it is not just the name of the lexical analyser that you have to define, but a whole class on which to hang the generated lexical analyser.  If you wanted to do things long hand then you would need to generate the following code (or something similar) in your output and include files:

Include File

class YYFAR name : public yylexer {

public:

    name();

protected:

    void yytables();

    virtual int yyaction(int action);

public:

};

#ifndef YYLEXNAME

#define YYLEXNAME name

#endif

Output File

YYLEXNAME::YYLEXNAME()

{

    yytables();

}

Where name is the name of the lexical analyser, and yylexer is the base class of the lexical analyser.  The base class depends upon whether you have built a default (fast) lexical analyser or a compact lexical analyser.  If you have built a fast lexical analyser, then the base class should be yyflexer.  If you have built a compact lexical analyser then the base class should be yyclexer.  Definitions for the yytables and yyaction member functions are generated automatically by ALex.

Multiple C Lexical Analysers

Name	Description

YYLEXNAME	Defines the name of the multiple lexical analyser.  The name must be a valid C identifier.  It is used to define a variable of type yymlex_t, the lexical analyser structure.  As an alternative to defining YYLEXNAME directly, you can use a Name Declaration (it does pretty much the same thing).  It does however have the advantage of placing a variable declaration in the lexical analyser header file if it is generated.

YYMLEXDATA	Defines the custom data to associate with the lexical analyser.  This value is assigned to the yymdata member of the yymlex_t structure.  Note that the yymdata member is of type void *.  If YYMLEXDATA is undefined then it assumes the value NULL.

Multiple C Lexical Analysers (Service Functions)

Service functions are called by the lexical analyser when they need to perform a task i.e. they want to get the next un-buffered character from the input or they want to output an unmatched character etc.  They are implemented in the yymlex_t structure as a set of pointers to functions.  This means that it is possible to easily plug in a new, replacement function, if required.

Name	Description

YYMGETCHAR	Defines the pointer to the unbuffered character input routine.  This value is assigned to the yymgetchar member of the yymlex_t structure.  The yymgetchar member is of type int (*)(yymlex_t *yy).  If YYMGETCHAR is undefined then it assumes the value yymgetchar.  That is it points to the function yymgetchar.

YYMINPUT	Defines the pointer to the buffered character input routine.  This value is assigned to the yyminput member of the yymlex_t structure.  The yyminput member is of type int (*)(yymlex_t *yy).  If YYMINPUT is undefined then it assumes the value yyminput.  That is it points to the function yyminput.

YYMOUTPUT	Defines the pointer to the character output routine.  This value is assigned to the yymoutput member of the yymlex_t structure.  The yymoutput member is of type void (*)(yymlex_t *yy, int ch).  If YYMOUTPUT is undefined then it assumes the value yymoutput.  That is it points to the function yymoutput.

YYMTEXTOVERFLOW	Defines the pointer to the text (and/or state) buffer overflow routine.  This value is assigned to the yymtextoverflow member of the yymlex_t structure.  The yymtextoverflow member is of type void (*)(yymlex_t *yy).  If YYMTEXTOVERFLOW is undefined then it assumes the value yymtextoverflow.  That is it points to the function yymtextoverflow.

YYMUNPUT	Defines the pointer to the character unput (backup) routine.  This value is assigned to the yymunput member of the yymlex_t structure.  The yymunput member is of type void (*)(yymlex_t *yy, int ch).  If YYMUNPUT is undefined then it assumes the value yymunput.  That is it points to the function yymunput.

YYMUNPUTOVERFLOW	Defines the pointer to the unput buffer overflow routine.  This value is assigned to the yymunputoverflow member of the yymlex_t structure.  The yymunputoverflow member is of type void (*)(yymlex_t *yy).  If YYMUNPUTOVERFLOW is undefined then it assumes the value yymunputoverflow.  That is it points to the function yymunputoverflow.

YYMWRAP	Defines the pointer to the routine called when all input has been processed.  The routine specifies whether there is more to come, after of course preparing the new input if there is any.  This value is assigned to the yymwrap member of the yymlex_t structure.  The yymwrap member is of type int (*)(yymlex_t *yy).  If YYMWRAP is undefined then it assumes the value yymwrap.  That is it points to the function yymwrap.

See Also

Declarations Section, Code Block, yymlex_t


Lex Rules Section
Rules Section

Syntax

rules-section :

%% [rule-list]

rule-list :

rule-list rule

rule

rule :

group-rule

expression-rule

Rules

Group Rule, Expressionm Rule

Remarks

The rules section is where the rules for the lexical analyser are defined.  There are two main types of rule, expression rules and group rules.  An expression rule represents the basic rule.  That is a regular expression with an associated action.  A group rule allows left context to be associated a number of other rules, either group or expression rules.  The rules section is introduced with the first %% mark in the Lex source file.

If a Code Block is placed before the first rule in the rules section, then the enclosed text in the code block is copied into the top of the generated yylexaction, yymlexaction or yylexer::yyaction function.  This function is called whenever the lexical analyser needs to execute an action.  Otherwise the text in the code block is copied after the previous action in the generated output file.  However, note that these actions have a trailing break statement, so the code will never be executed.

A code block placed at the start of the rules section has at least one important use, and that is to gain access to the yylval variable.  This is the variable used by YACC generated parsers to hold the attribute for the current lookahead token.  It is only necessary to do this for multiple instance model C parsers and C++ parsers.  For single model and multiple model C parsers, yylval is defined as a global variable, although for multiple model C parsers it is statically defined in one file.

The following code block will allow a multiple instance model C lexical analyser to gain access to the yylval variable for a corresponding multiple model C parser.  Note that the code assumes that the yymdata member of the yymlexer structure points to the yylval variable.  Typically, when you create the lexical analyser, you initialise the member as well.  It is assigned with the yymlvalptr member of the corresponding yymparser structure.  The type YYSTYPE should be defined by the parser in either the generated include or output file:

%{

YYSTYPE *yymlvalptr = (YYSTYPE *) yy->yymdata;

yyassert(yymlvalptr != NULL);

#define yylval (*yymlvalptr)

%}

The following code block will allow a C++ lexical analyser to gain access to the yylval variable for a corresponding C++ parser.  The yyparserptr member is assumed to point to a valid C++ parser.  This can be initialised in the call to the yylexer::yycreate member function when the lexical analyser is created.  The type YYSTYPE should be defined by the parser in either the generated include or output file.

%{
YYSTYPE& yylval = *(YYSTYPE*)yyparserptr->yylvalptr;

%}

There do not have to be any rules in the rules section in order for Lex to create a lexical analyser.  In cases where there are no rules, the generated lexical analyser will simply copy characters from the input stream to the output stream.

See Also

Grammar, Code Block


Lex Group Rule
Group Rule

Syntax

group-rule :

[left-context] { [rule-list] }

Rules

Left Context, Rule List

Remarks

A group rule provides a convenient and easy way of associating left context with a number of regular expressions.  For example a set of regular expressions to recognise a C block comment would look something like:

<COMMENT>.|\n   { /* keep going */ }

<COMMENT>"*/"   { yybegin(0); }

"/*"            { yybegin(COMMENT); }

Where COMMENT is a start state that has been declared earlier.  The alternative, shorter representation using a group rule would look something like:

<COMMENT>{
.|\n            { /* keep going */ }

"*/"            { yybegin(0); }

}

"/*"            { yybegin(COMMENT); }

Note that it is possible to nest group rules.  The effects of nested group rules are cumulative.  For instance, the regular expressions below match the string "a" when the lexical analyser is in start state A, and "b" when the lexical analyser is in start states A or B and the last character read was a newline character.

<A>{

"a"         { /* match "a" */ }

^<B>{

"b"         { /* match "b" */ }

}
}

Note that is not possible to "tab" the regular expressions to give a more meaningful representation.  If you were to do this, then the lines would be considered as Code Lines and simply copied into the generated output file.

See Also

Rules Section


Lex Expression Rule
Expression Rule

Syntax

expression-rule :

expression separator expression-rule

expression action

expression :

[left-context] right-context-expression

Rules

Left Context, Right Context Expression

Tokens

Separator, Action

Remarks

An expression rule consists of a regular expression and an action.  The regular expression always starts in the first column of a line in the rules section.  A white space character (space or tab but excluding the newline character) indicates the end of a regular expression.  In fact it indicates the introduction of an Action or Separator token.

It is possible to split a regular expression across two or more lines.  There must be no white space characters (space or tab) preceding the newline character.  Note that the split regular expression is automatically concatenated by the compiler.  Individual tokens themselves cannot be split up this way.  For instance, Character Strings cannot cross a line boundary.

A Macro Reference can be made from within a regular expression.  They cannot be made from anywhere else.  They are useful for making regular expressions more readable.  Macros must first be declared with a Macro Declaration before they can be referenced.

When a rule is matched, the corresponding action is performed.  For example, the regular expression below defines the (decimal) integer constant available in the C programming language.  When the integer constant is matched, the function installinteger() is called.  This installs the token into the symbol table and returns a pointer to the symbol.  This is then assigned to yylval for use by the parser:

[1-9][0-9]*     { yylval.symbol = installinteger(); return INT; }

Note that the lexical analyser will always try and match the longest possible string that it can.  For instance, the above regular expression could partition the input "123" a number of different ways, such as:

i	"1", "2", "3"

ii	"1", "23"

iii	"12", "3"

iv	"123"

However it will always partition it as iv, since "123" is the longest possible string.

It is possible to have more than one regular expression share the same action.  This is possible by using the Separator token.  For example, the regular expressions below define the three integer constants available in the C programming language.  That is the octal, decimal and hexadecimal integer constants:

0[0-7]*             |

[1-9][0-9]*         |

0[xX][0-9a-fA-F]+   { yylval.symbol = installint(); return INT; }

As before the function installinteger() is used to install the token into the symbol table.  Note that in all of the above examples the optional integer constant suffix has been omitted.  This states whether the integer is unsigned (u or U) and/or a long integer (l or L).  This is best implemented using a macro.  The example below demonstrates decimal integer constants only:

suffix  ([uU][lL]?|[lL][uU]?)

%%

[1-9][0-9]*{suffix}?    { yylval.symbol = installint(); return INT; }

If no regular expressions match an input string, then the first character in the string is discarded and the input string is re-examined.  The discarded character is passed to the function yyoutput.  By default this outputs the character to the stream yyout.  By default yyout is assigned to the standard output stream stdout  In other words any unmatched characters are copied to stdout.  Note that it is possible to redefine yyoutput and/or assign other values to yyin depending upon how you want the lexical analyser to behave.

It is possible (and indeed desirable) to specify ambiguous regular expressions in rules.  If two different regular expressions match the same string, then the regular expression listed first in the rules section takes precedence i.e. it will match the string.  No error is caused by this.  Note however that this can lead to situations where a regular expression can never be matched.  In such circumstances a warning will be given instead.  An example best illustrates its use.  Below are rules for recognising C identifiers and the first four (alphabetically listed) keywords:

"auto"                  { return AUTO; }

"break"                 { return BREAK; }

"case"                  { return CASE; }

"char"                  { return CHAR; }

[A-Za-z_][A-Za-z0-9_]*  { yylval.symbol = installid(); return ID; }

The string "auto" is matched by both the regular expression for the keyword auto and the regular expression for an identifier.  Because the regular expression for the keyword is listed first the keyword is matched.  The same applies for all the other keywords.  If however, the regular expression for the identifier had been placed at the top, then no keywords would be matched, and corresponding warning messages issued.

See Also

Rules Section, Ambiguity Resolution


Lex Left Context
Left Context

Syntax

left-context :

< start-list > [left-context]

< * > [left-context]

^ [left-context]

start-list :

start-list , start

start

start :

identifier

Tokens

Identifiers

Remarks

Left context can be specified for an Expression Rule or a Group Rule.  This means that the expression, or group of expressions, will only be matched when they are preceded by a certain sequence of strings.  The simplest form of left context that can be specified for an expression is whether the matched string is preceded by a newline character or not.  This is done by prefixing the expression with the ^ operator, called the leading newline operator.  The example expression below recognises assembler labels.  Note that assembler labels typically start in column 1 of a line i.e. after a newline character:

^[a-zA-Z_][a-zA-Z0-9_]*     { return LABEL; }

Note that the leading newline character is not counted as part of the string.  It is context which is used to determine whether the regular expression is matched or not.  The first character read by a lexical analyser does not have a preceding character.  In this case the preceding character is assumed to be a newline character. In other words, the first character read by the lexical analyser is assumed to start on a new line.  One thing to careful of with the ^ operator is the ordering of regular expressions.  Consider the two regular expressions below.  One to recognise assembler labels as above, and the other to recognise other identifiers:

[a-zA-Z_][a-zA-Z0-9_]*      { return ID; }

^[a-zA-Z_][a-zA-Z0-9_]*     { return LABEL; }

You would think this would be Okay, however it isn't.  The second regular expression is never matched, and indeed the compiler will inform you of this.  The reason is that the first expression is a generalised expression and will matched all identifiers whether they are preceded by a newline character or not.  Remember that the preceding newline character is not treated as part of the matched string, and will not affect its length.  The solution to the problem, Is of course, to reverse the ordering of the regular expressions.

In order to implement more complex left context for regular expressions it is necessary to resort to start states.  Any start states which are going to be used by regular expressions must first be declared with a Start Declaration.  When a start state is associated with a regular expression it means that the regular expression can only be recognised when the lexical analyser is in the specified state.  Note that it is possible, and indeed desirable, to be able to associate more than one start state with a regular expression.  The example below declares two start states AA and BB.  The string "a" will only be recognised when the lexical analyser is in start state AA.  The string "b" on the other hand will be recognised when the lexical analyser is in both start states AA and BB:

%start AA

%start BB

%%

<A>a        { /* match a */ }

<AA,BB>b    { /* match b */ }

When no start states are associated with a regular expression, then the regular expression will be recognised when the lexical analyser is in any (inclusive) start state.  An inclusive start state is a start state that has been declared with the %start keyword.  In order to be able to recognise a regular expression when the lexical analyser is in any start state, inclusive as well as exclusive, it is necessary to prefix the regular expression with <*>.  An exclusive start state is a start state that has been declared with the %exclusive keyword.  The example below again declares two start states.  However in this case start state BB is now an exclusive start state.  When the lexical analyser is in neither of these start states, the strings "a" and "b" can be matched.  When the lexical analyser is in start state AA, the strings "a", "b" and "c" can be matched.  However, when the lexical analyser is in start state BB only strings "b" and "d" can be matched:

%start AA

%exclusive BB

%%

a           { /* match a */ }

<*>b        { /* match b */ }

<AA>c       { /* match c */}

<BB>d       { /* match d */ }

The lexical analyser always starts off in the INITIAL start state.  This is an inclusive start state and is automatically defined by Lex.  In the example above, this is the start state the lexical analyser is in when it is in neither AA or BB.  In order to change start state it is necessary to use the function yybegin.  This takes one parameter which specifies the start state to change to.  The function yybegin is typically called from within an Action associated with a regular expression.  As an alternative to calling yybegin(INITIAL), it is also possible to use the much shorter yybegin(0).  The example below is a small program which counts the number of words in a file.  A word is assumed to consist of letters and nothing else.  The start state WORD is entered whenever a letter is encountered.  The lexical analyser remains in this start state, eating characters, until it encounters a character which is not a letter.  At this point it then returns to the INITIAL default start state.  Note also the use of the function yywrap.  This is called when the lexical analyser has run out of input.  Here we are using it to print out the number of words that were counted.  We return 1 to indicate that there is indeed no more input to follow:

%{

int wc = 0;

%}

%start WORD

%%

<WORD>[a-zA-Z]  { /* gobble up */ }

<WORD>.|\n      { yybegin(0); }

[a-zA-Z]        { wc++; yybegin(WORD); }

.|\n            { /* gobble up */ }

%%

int yywrap(void)

{

    printf("word count: %d\n", wc)

    return 1;

}

This program can be written much simply without the use of start states, however the above example has one distinct advantage, and that is it only ever matches strings of one character in length.  This type of construct has applications elsewhere, for instance consider the recognition of C comments /*…*/.  Note also that the ordering of the regular expressions in the example above is crucial.  If they were arranged any differently then the program would not work correctly.  If the WORD start state was declared as being exclusive, then it would be possible to move the WORD regular expressions down.

It is possible to specify the leading newline operator ^ in a regular expression and to associate start states with the same regular expression.  The affects are cumulative i.e. the regular expression will only be matched when the preceding character was a newline character, and the lexical analyser is in one of the specified start states.

See Also

Rules Section, Group Rule, Expression Rule


Lex Right Context Expression
Right Context Expression

Syntax

right-context-expression :

[union-expression] / union-expression

[union-expression] $

union-expression

<<EOF>>

Rules

Union Expression

Remarks

This expression allows you to specify right context for a regular expression.  The / operator, referred to as the right context operator, is the main way of specifying right context.  In many aspects, the operator can be thought of as being the same as concatenation.  The principle difference occurs when the regular expression is matched.  In such instances, the string which matches the expression on the right hand side of the operator is backed up onto the unput buffer.  The example below matches the string "abcd".  When such a match occurs, the characters 'c' and 'd' are backed up:

ab/cd   { /* a simple example */ }

You can implement right context expressions without using the / operator.  Instead the backup can be performed in the action by calling the function yyless.  The example below illustrates this:

abcd    { yyless(2); }

Note that the argument passed to the function is the new length of the matched string ("ab"), and not the number of characters to backup.  The choice as to whether to use the operator or the function is for you to decide.  The operator states exactly what is going to happen, however the function yyless is slightly more efficient.

The $ operator, referred to as the trailing newline operator, only matches an expression if it has a tailing newline character.  Similar to the right context operator, the newline character is backed up onto the unput buffer if the expression is matched.  The $ operator is really a shortening of a longer right context operator equivalent:

abcd$       { /* shorter */ }

abcd/\n     { /* longer */ }

It is only possible to have one / or $ operator in a regular expression.  Note that the grammar for the right context expression is not recursive.  It simply makes reference to union expressions.  Further, the grouping operator pair ( ) in a Primary Expression also makes reference to a union expression.

The expression on the left hand side of the / and $ operators is optional.  In other words it is possible for an expression to backup completely onto the unput buffer.  Be very careful of this.  It is extremely dangerous, and can lead to your lexical analyser looping indefinitely if it encounters a string which matches one of these expressions.  Typically you will do something in the action associated with the expression to ensure that this does not happen i.e. change start state:

\abcd   { /* trouble */ }

Note that even with an expression on the left hand side of the / and $ operators,  the matched string can still backup completely.  Consider the expression below:

a?\bcd  { /* trouble */ }

Here, the strings "abcd" and "bcd" are matched by the regular expression.  However in the second case, all three characters 'b', 'c' and 'd' will be backed up.

Because of the potential trouble that can be caused by expressions which backup completely, ALex always issues a warning when it encounters one.  They have been categorised into two types.  Those that will always backup completely, and those that can backup completely.

The <<EOF>> operator is a special case and allows you to specify the end-of-file marker as trailing context.  Note that there is no preceding union expression in the above syntax.  Rules of this type are only matched when all input for the lexical analyser has been exhausted.  They are typically used to trap certain conditions at the end of processing.  An example is the recognition of C style block comments below.  Note that the end-of-file marker is only matched when the lexical analyser is in the COMMENT start state:

%x COMMENT

%%

<COMMENT>.|\n       { /* do nothing */ }

<COMMENT>"*/"       { yybegin(0); }

<COMMENT><<EOF>>    { printf("unexpected eof in comment\n"); }

"/*"                { yybegin(COMMENT); }

Beware of returning a value from actions associated with trailing end-of-file expressions.  This will mean that the lexical analyser will not call yywrap, as it normally does, at the end of processing the input.

Note

Right context expressions of the form below cause trouble in ALex (they cause trouble in most, if not all versions of Lex).  Note however that these are very unlikely expressions:

a*/a*   { /* trouble */ }

a*/a+   { /* trouble */ }

In both cases above, no characters are backed up if a string consisting of a's is encountered by the lexical analyser.  In the first case, this is OK, however in the second case you would expect at least one character to be backed up.  This is not the case.  The trouble is caused by how the compiler lays out states for the regular expressions and how the resultant lexical analyser works.

When the lexical analyser performs backup, it first of all examines the matched string to see whether it is matched by the left hand side of the expression.  If not, then the last character in the string is backed up, and the string is then re-examined.  This is repeated until either a match occurs, or the string is completely backed up.  From this description it is quite obvious why a string of a's is never backed up in the examples above.  They are always matched by the left hand sides of their expressions.  The example below is a bit more complicated:

(ab)*/abc   { /* trouble */ }

In this case only the 'c' will be backed up if the expression is matched.  If you want to implement expressions like this in Lex, then you will need to resort to the yyless function.

See Also

Left Context


Lex Union Expression
Union Expression

Syntax

union-expression :

concatenation-expression

union-expression | concatenation-expression

Rules

Concatenation Expression

Remarks

This expression allows you to match alternate expressions in a regular expression.  The | character can be referred to as the union operator.  The expression below matches a binary digit, which can be either a 0 or a 1:

0|1     { /* binary digit */ }

See Also

Right Context Expression


Lex Concatenation Expression
Concatenation Expression

Syntax

concatenation-expression :

repetition-expression

concatenation-expression repetition-expression

Rules

Repetition Expression

Remarks

This expression allows you to match one expression after another in a regular expression.  This is also referred to as regular expression concatenation.  Note that there is no operator associated with concatenation, rather it is implied.  The expression below matches the C keyword if.  It is formed by concatenating the regular expression i with the regular expression f:

if  { /* match if */ }

See Also

Union Expression


Lex Repetition Expression
Repetition Expression

Syntax

repetition-expression :

primary-expression

repetition-expression *

repetition-expression +

repetition-expression ?

repetition-expression range

range :

{ integer , [integer] }

{ integer }

Rules

Primary Expression

Tokens

Integers

Remarks

This expression allows you to specify how many times a regular expression can be matched.  There are a number of different operators, which provide some flexibility.

The Kleene Closure operator, represented by an asterisk *, matches a regular expression zero or more times.  The regular expression below matches a C identifier.  This consists of a letter or underscore, followed by zero or more letters, digits or underscores:

[a-zA-Z_][a-zA-Z0-9_]*  { /* identifier */ }

The Positive Closure operator, represented by a plus sign +, matches a regular expression one or more times.  The regular expression below matches a simple decimal integer.

[0-9]+  { /* integer */ }

The Optional operator, represented by a question mark ?, matches a regular expression zero or one times.  The regular expression below matches the word "car", or its plural "cars".  Note that because the Optional operator has a higher precedence than concatenation, the optional operator applies to the s, and not the entire word:

cars?   { /* plural? */ }

The Range operator, introduced by a left bracket {, provides a number of different ways to specify how many times a regular expression is matched.  If the range operator consists of two integers separated by a comma, then the first integer specifies the minimum number of times that the regular expression can be matched, and the second integer the maximum number of times that the regular expression can be matched.  The minimum value must always be less than or equal to the maximum value.  It is possible to use a value of zero.  Indeed you can have {0,0}, which will effectively remove the expression from the regular expression.  The regular expression below matches the character 'a' 3-5 times:

a{3,5}  { /* aaa, aaaa, aaaaa */ }

It is worth noting that the range operator sequence {0,1} is equivalent to the Optional operator ?.  If the second integer is omitted, then the remaining first integer specifies that the regular expression can be matched this many times or more.  For instance, the regular expression below matches the character 'a' 3 or more times:

a{3,}   { /* aaa, aaaa, aaaaa, aaaaaa etc. */ }

Note that the range operator sequence {0,} is equivalent to the Kleene Closure operator *, and the range operator sequence {1,} is equivalent to the Positive Closure operator +.  If both the comma and second integer are omitted, then the remaining integer specifies that the regular expression can only be matched this many times exactly.  The regular expression below matches the character 'a' three times:

a{3}    { /* aaa */ }

Beware of specifying large minimum and/or maximum values.  Every time the maximum value specified is increased by one, another state is added to the resultant DFA.  If there are a large number of transitions between the states, then the size of the lexical analyser will grow quite quickly.

See Also

Concatenation Expression


Lex Primary Expression
Primary Expression

Syntax

primary-expression :

character

character-string

character-class

.

( union-expression )

Rules

Union Expression

Tokens

Characters, Character Strings, Character Classes

Remarks

This is the basic part of a regular expression.  It consists of characters, character strings, character classes and grouped expressions.

The character . is a notational shorthand for the character class [^\n].  This matches every character except the newline character.  Note that this also includes the NUL (0) character.

It is possible to group regular expressions using parentheses ( and ).  Consider the two regular expressions below:

ab|cd       { /* ab or cd */ }

a(b|c)d     { /* abd or acd */ }

In the first example, since concatenation has higher precedence than the union operator, either "ab" is matched or "cd" is matched.  In the second example, by grouping b|c, we can now change the meaning of the regular expression so that it will always start with an 'a', end with a 'd', and have either 'b' or 'c' in the middle.

See Also

Repetition Expression


Lex Ambiguity Resolution
Ambiguity Resolution

Lex accepts ambiguous rules.  When presented with a set of rules, Lex will always attempt to create a lexical analyser from them.  If cases occur where rules are never matched, then these are reported.  The two rules that Lex uses to resolve ambiguities are relatively simple and are shown below.  Note that the way ambiguities are resolved is really a result of how the states are laid out for the lexical analyser by Lex, plus the way the states are interpreted when processing input:

1	At a partition boundary, Lex always attempts to match the longest string possible.  For instance, consider the two regular expressions:

a

aaa

These two expressions could partition the input "aaaaa" a number of different ways, as shown below:

i	a, a, a, a, a

ii	aaa, a, a

iii	a, aaa, a

iv	a, a, aaa

However, with the application of the first rule, it is quite obvious that the input stream will be partitioned as (ii) above.  Note that Lex is assumed to start at a partition boundary when no input has yet been read.  The rule does have its disadvantages, and can lead to a large amount of lookahead if you are not careful.  This is especially true with [^…]* and .* style regular expressions.  In the latter case, the lexical analyser will have to scan the input until a newline character is encountered.

2	If two or more rules match the same string, then it is the rule that is listed first in the Lex file that matches it.  For instance, consider the two regular expressions blow.  One is for the C keyword if, and the other is for a C identifier.  A C identifier is said to consist of a letter or underscore character, followed by zero or more letters, digits and underscore characters:

[a-zA-Z_][a-zA-Z0-9_]*

if

In the case where the input "if" is considered, both regular expressions will recognise it, however it is the regular expression for the C identifier that will match it.  Indeed, in the current case, the keyword if will never be recognised, and Lex will issue a warning message to indicate so.  This is a common mistake that can be made when writing Lex rules.  The answer to the problem is, of course, to swap the two regular expressions around.

There is, however a case where Lex can potentially match all regular expressions, and this is when the -r Option has been specified on the command line.  In such cases all regular expressions can be matched, and no warning messages are given to say otherwise.  In those cases where more than one regular expression matches a string, it is necessary for a call to yyreject, yymreject or yylexer::yyreject to be made in the action for the first regular expression.  This will then force the lexical analyser to reject the currently matched regular expression and look for a new one.

One final thing that needs to be mentioned concerns the right context operator /.  It is possible using this operator to specify ambiguous regular expressions.  The ambiguity does not concern what regular expressions are matched, since this is governed by the two rules above.  Instead it concerns how many characters are backed up when the regular expression is matched.  For instance, consider the regular expression:

a*/a*

This regular expression will match any number of a's.  However, when a string of a's is matched, determining how many of them are backed up poses a problem.  From the above regular expression you could conceivably backup 0, 1, 2 etc. or all of the a's.  The specification is ambiguous.  It is though accepted by Lex.  The problem is solved in the lexical analyser by backing up the minimum number of characters required.  So in this case no characters will be backed up, since a string of a's is matched by the regular expression on the left hand side of the right context operator /.  Be aware though that Lex will issue a warning for the above regular expression saying that it can backup completely.  Although this is technically true, the way the lexical analyser is implemented ensures that this will never happen.  For more information see also Right Context Expression.


Lex Programs Section
Programs Section

Syntax

programs-section :

program-code

Tokens

Program Code

Remarks

The programs section is where any functions used by the lexical analyser are typically placed.  It is an optional section and is indicated by the second %% mark in the Lex file.  Note that the Program Code token wraps both the %% mark and the trailing characters until the end-of-file character.

For a list of functions which can be used to operate on the parsers, and any variables which can also be modified see the YACC And Lex Library.

See Also

Grammar


Mutiple Lexical Analysers
Multiple C Lexical Analysers

It is possible to have two or more C lexical analysers in the same executable.  To do this you must use the multiple lexical analyser model when generating source code.  This model differs significantly from the single (default) lexical analyser model.  What it does is group the necessary variables, functions and tables from the single lexical analyser model into one single structure called yymlex_t.  Calls to the lexical analyser are then typically made with an extra pointer parameter identifying the lexical analyser in question.

Other versions of Lex take an alternative approach to implementing multiple lexical analysers.  Instead of having a structure they allow you to change the "yy" prefix at the start of all function and variable names.  For instance you could change the prefix to "zz", and then call zzlex etc.  Naturally there are advantages and disadvantages to both approaches.

The multiple lexical analyser model has a different header file "mlex.h" to the single lexical analyser model's "lex.h".  This file holds all declarations and definitions required by the lexical analyser.  All multiple lexical analyser names, both function and variable, have the prefix "yym" instead of "yy".  The notable exceptions are yydebug and yydebugflush which apply to both the single and multiple lexical analyser models.  All multiple lexical analyser functions also take an additional parameter.  This is a pointer to the lexical analyser that the function is to be carried out on.  The parameter is always first and is of type yymlex_t *.

In order to distinguish one multiple lexical analyser from another it is necessary to give each lexical analyser a name, or identifier.  You can do this by using the %name keyword.  For more information see also Name Declaration:

%name mylexer

The generated source code then contains a variable mylexer of type yymlex_t.  To run the lexical analyser you would then call yymlex with the address of mylexer as its parameter:

int main(void)

{

    return yymlex(&mylexer);
}

If a header file has been generated then a declaration for the lexical analyser also appears in it.  An alternative to using %name is to define YYLEXNAME directly:

#define YYLEXNAME mylexer

Note that in this case a generated header file will not contain any lexical analyser declaration.

When redefining multiple lexical analyser service functions be aware that the functions will be used by all multiple lexical analysers.  There are of course ways of redefining these functions for individual lexical analysers.  The yymlex_t structure has a number of pointer to function members, one for each of the service functions.  When a service function is called by the lexical analyser it consults one of these members.  In the declarations section of your Lex source file, you can define how each of these members is initialised.  For instance if you had written a new character input routine then you would need to define YYMGETCHAR as the name of the new function:

int mygetchar(yymlex_t *yy)

{
    ...

}

#define YYMGETCHAR mygetchar

The complete list of service functions, and the associated name to define is shown below:

Name	Service Function

YYMGETCHAR	yymgetchar

YYMINPUT	yyminput

YYMOUTPUT	yymoutput

YYMTEXTOVERFLOW	yymtextoverflow

YYMUNPUT	yymunput

YYMUNPUTOVERFLOW	yymunputoverflow

YYMWRAP	yymwrap

If you do not define the name, then the default service function is used.  Note that the replacement function must have the same prototype as the corresponding service function.  Alternatively you can assign service functions to the members at run time.  Each member has the same name as its functional equivalent i.e. there is a yymgetchar member for the function yymgetchar etc.

The multiple parser structure yymlex_t has a user data member yymdata.  It is a simple void * type.  You can specify the value of it at compile time by defining YYMLEXDATA in your Lex source file:

#define YYMLEXDATA mydata

Alternatively you can assign a value to it at run time.

If you are moving a lexical analyser from the single model to the multiple model it is not necessary to change all of the names from their single to multiple equivalent.  For instance, if you use the function yyecho() in your actions, then the multiple equivalent would be yymecho(yy).  Where yy is the pointer to the multiple parser.  It is possible to include the header file <mlconv.h> in your Lex source file to do just this.

Multiple Instance C Lexical Analysers

It is also possible to create multiple instances of a C lexical analyser.  The creation of such lexical analysers is slightly more involved than a normal C lexical analyser.  For a start, the initial lexical analyser that is created is a template for other lexical analysers which are created on the stack and/or heap.  Further more, all actions have to use the yy parameter passed to the wrapping yylexaction function.  This identifies the lexical analyser that the action is being performed for, and is of type yymlex_t *.  Similar to multiple lexical analysers it is possible to include a header file <milconv.h> to convert single lexical analyser names to their multiple instance equivalent.  Note that you may find problems when you do this.  All functions assume that there is a yy parameter defined which points to the lexical analyser.  This is fine, if the function is being called from the action, however if it is from a sub-function, then trouble may arise.  The creation, and destruction sequence for a multiple instance lexical analyser is shown below.

#include "mylexer.h"

int analyse(void)

{

    int n = 0;

    yymlex_t lexer;

    if (yymcreatelex(&lexer, &mylexer)) {

        n = yymlex(&lexer);

        yymdestroylex(&lexer)

    }

    return n;

}

The function analyse() first defines the local variable lexer.  This will hold the lexical analyser instance.  The function yymcreatelex is then called to initialise it.  This also includes the allocation of any memory for buffers.  Now that everything has been set up the lexical analyser can be called with yymlex.  Afterwards, the lexical analyser instance is destroyed by calling yymdestroylex.  It is not necessary to call yymlexcleanup.  This is handled automatically.

See Also

Name Declaration


Lex Operator Precedence Table
Operator Summary

The table below shows a summary of the operators that can be used to form regular expressions.  Each operator is listed with its precedence and a brief description.  The fixed width font denotes characters as they would appear in a Lex source file.  Italics denote other syntactic units.  Braces […] mean that the enclosed item is optional.

Operator	Precedence	Description

(a)	0 (highest)	Grouping Operator.  Allows default operator precedence to be overridden.  Note that you can only group operators of precedence 3 or higher.  See also Primary Expression.

a*	1	Kleene Closure Operator.  Matches the regular expression a zero or more times.  See also Repetition Expression.

a+	1	Positive Closure Operator.  Matches the regular expression a one or more times.  See also Repetition Expression.

a?	1	Optional Operator.  Matches the regular expression a zero or one times.  See also Repetition Expression.

a{m[,[n]]}	1	Range Operator.  Matches the regular expression a from m to n times inclusive.  If n is omitted then a is matched m or more times.  If both the comma and n are omitted then a is matched exactly m times.  See also Repetition Expression.

ab	2	Concatenation Operator.  Matches the regular expression a followed by the regular expression b.  Note that the operator is implied.  See also Concatenation Expression.

a|b	3	Union Operator.  Matches either the regular expression a or the regular expression b.  See also Union Expression.

 [a]/b	4	Right Context Operator.  Matches the regular expression a followed by the regular expression b. When the expression is matched, b is backed up.  Note that the regular expression a is optional.  There can be only one (if any) right context, trailing newline or trailing end-of-file operator in a regular expression.  See also Right Context Expression.

 [a]$	4	Trailing Newline Operator.  Matches the regular expression a followed by a newline character '\n'.  When the expression is matched, the newline character is backed up.  Note that the regular expression a is optional.  There can be only one (if any) trailing newline, right context or trailing end-of-file operator in a regular expression.  See also Right Context Expression.

<<EOF>>	4	Tailing End-Of-File Operator.  Matches the end-of-file marker.  When the expression is matched, the end-of-file marker is (effectively) backed up (if you think of it as a character).  There can be only one (if any) trailing end-of-file, right context or trailing newline operator in a regular expression.  See also Right Context Expression.

<s>a	5	Left Context Operator.  Matches the regular expression a only when the lexical analyser is in the start state s.  See also Left Context.

<*>a	5	Left Context Operator.  Matches the regular expression a when the lexical analyser is in all start states, both exclusive and non-exclusive.  See also Left Context.

^a	5 (lowest)	Leading Newline Operator.   Matches the regular expression a only when the preceding lexical analyser character was a newline character '\n'.  See also Left Context.

Where a and b are regular expressions, m and n are integers and s is one or more start states separated by commas.

The basic units of a regular expression are Characters, Character Strings and Character Classes.  See also Primary Expression.


ASCII Character Set

The American Standard Code for Information Interchange character set.

Character	Hex	Decimal	Character	Hex	Decimal

NUL	00	0	@	40	64

SOH	01	1	A	41	65

STX	02	2	B	42	66

ETX	03	3	C	43	67

EOT	04	4	D	44	68

ENQ	05	5	E	45	69

ACK	06	6	F	46	70

BEL	07	7	G	47	71

BS	08	8	H	48	72

TAB	09	9	I	49	73

LF	0A	10	J	4A	74

VT	0B	11	K	4B	75

FF	0C	12	L	4C	76

CR	0D	13	M	4D	77

SO	0E	14	N	4E	78

SI	0F	15	O	4F	79

DLE	10	16	P	50	80

DC1	11	17	Q	51	81

DC2	12	18	R	52	82

DC3	13	19	S	53	83

DC4	14	20	T	54	84

NAK	15	21	U	55	85

SYN	16	22	V	56	86

ETB	17	23	W	57	87

CAN	18	24	X	58	88

EM	19	25	Y	59	89

SUB	1A	26	Z	5A	90

ESC	1B	27	[	5B	91

FS	1C	28	\	5C	92

GS	1D	29	]	5D	93

RS	1E	30	^	5E	94

US	1F	31	_	5F	95

SPACE	20	32	`	60	96

!	21	33	a	61	97

"	22	34	b	62	98

#	23	35	c	63	99

$	24	36	d	64	100

%	25	37	e	65	101

&	26	38	f	66	102

'	27	39	g	67	103

(	28	40	h	68	104

)	29	41	I	69	105

*	2A	42	j	6A	106

+	2B	43	k	6B	107

,	2C	44	l	6C	108

-	2D	45	m	6D	109

.	2E	46	n	6E	110

/	2F	47	o	6F	111

0	30	48	p	70	112

1	31	49	q	71	113

2	32	50	r	72	114

3	33	51	s	73	115

4	34	52	t	74	116

5	35	53	u	75	117

6	36	54	v	76	118

7	37	55	w	77	119

8	38	56	x	78	120

9	39	57	y	79	121

:	3A	58	z	7A	122

;	3B	59	{	7B	123

<	3C	60	|	7C	124

=	3D	61	}	7D	125

>	3E	62	~	7E	126

?	3F	63	DEL	7F	127


Build Tools

Operation

AYACC

ALex

Errors

AYACC

ALex


AYACC Operation

The command line format for AYACC is:

AYACC {options} input-file {options}

If no options or file name is given, then a brief summary of the command line format for AYACC is displayed.

Input File

Specifies the name of the YACC source file.  YACC files generally have an extension of .y.

Option	Description

-? Option	Display help.

-$ Option	Suppress display of sign-on banner.

-E Option	No introduction of error actions for default states.

-G Option	No default goto states.

-I Option	No introduction of error gotos for default states.

-N Option	No default reductions if shift on error token.

-R Option	Specifies default reduction options.

-S Option	No default action states.

-T Option	Specifies the target language of the generated parser.

-c Option	Use compatible file names y_tab and y_output.

-d Option	Generate include file.

-e Option	Output file name extension (.c default for C, .cpp for C++).

-f Option	Do not treat #line directive file names as strings

-l Option	Do not produce #line directives in output file.

-m Option	Specifies the parser model.

-o Option	Output file name.

-p Option	Specifies the parser to generate.

-s Option	Use compact tables.

-v Option	Generate verbose output.

-x Option	Include file name extension (.h default).

-z Option	Use short file names.

Note that the options are case sensitive, so -C is not the same as -c.  In fact it is an illegal option, and as such will generate an error.


AYACC -? Option

Syntax	-?

Remarks

Displays a brief description of the command line format expected by AYACC, plus a brief description of each option that it can take.  This option is ignored if there are any other options and/or input file specified on the command line.


AYACC -$ Option

Syntax	-$

Remarks

Suppresses the display of the sign-on banner when the tool is run.  The sign-on banner consists of a version and copyright message.  Also included in the sign-on banner is a summary message output at the end of the compilation.  This gives the total number of errors and warnings found in the compilation.  Note that if a fatal error is encountered, then this message is never given.

The -$ option can appear anywhere on the command line.  It does not need to go at the start.  AYACC does an initial scan of the command line looking for this option alone.


AYACC -E Option

Syntax	-E

Remarks

This option specifies that error actions will not be introduced when generating default states.  This option is closely associated with the -S Option.  Furthermore if the -S option is specified, then the -E option will have no effect on the generated action table.  For more information about default action states see -S Option.

See Also

-S Option


AYACC -G Option

Syntax	-G

Remarks

This option specifies that no default states will be introduced when compressing the parser's goto table.  Ordinarily, if one state, say state A, contains gotos that are present in another state, say state B, then it is possible to make state B the default state of state A.  Gotos that are common between both states are then removed from state A.

A state B will become a default state of state A if it matches the following criterion:  Firstly, the two values L and G are calculated for states A and B:

L	The number of lost gotos.  This is the number of gotos which are the same between states A and B.  These are the gotos that are removed from state A if state B is made a default state.

G	The number of gained gotos.  This is the number of error gotos which are added to state A to deal with unwanted gotos in state B.

If L is greater than G, then T = L - G represents the total number of lost gotos.  If there is no other state B with a larger value of T, then state B is made the default state of state A.

If the -I Option is specified then there is an additional criterion placed on G, and that is it must always be zero.  In effect this means that state B always contains gotos for the same nonterminals that are in state A.

See Also

-I Option


AYACC -I Option

Syntax	-I

Remarks

This option specifies that error gotos will not be introduced when generating default states.  This option is closely associated with the -G Option.  Furthermore if the -G option is specified, then the -I option will have no effect on the generated goto table.  For more information about default goto states see -G Option.

See Also

-G Option


AYACC -N Option

Syntax	-N

Remarks

This option specifies that no default reduction will be introduced to a state if there is an error token shift action in the specified state.  This option is intended for supporting error recovery.  It makes sure that a default reduction is not made when an error recovery production is potentially available.  This option is closely associated with the -R Option.

Note that this option changes the resultant parser that is generated.  The effects of which can be seen in the verbose file.  You can generate one of these files by specifying the -v Option.

See Also

-R Option


AYACC -R Option

Syntax	-R[n|o|e|f|d]

Remarks

This is a table compression option.  It specifies whether default reductions are going to be used when generating the parser's tables, and if so what configuration is going to be used.  Default reductions offer a relatively inexpensive way of compressing the tables, and provide considerable advantages when implementing error recovery in grammars.  The list of default reduction options, together with simple descriptions is shown below:

Option	Description

-Rn	No default reductions.

-Ro	Default reductions if no other actions.

-Re	Default reductions for error token reductions only.

-Rf	Default reductions favouring error token reductions.

-Rd	Default reductions (default).

The -Rn option specifies that no default reductions will be used in the generated parser's tables.  The generated parser will follow exactly, baring any shift-reduce or reduce-reduce conflicts, the grammar specified in the YACC source file.

The -Ro option specifies that a default reduction will be introduced only if there are reduce actions in the specified state, and the actions are all calling for the reduction of the same rule.  For example, a state which reduces rule A on lookahead tokens a, b, c satisfies this criteria.  If there were shift actions, or reductions of other rules, then the criteria would fail.  This option is mainly intended to provide compatibility with other version of YACC, such as Berkeley YACC (or BYACC).

The -Re option specifies that a default reduction will be introduced if there is reduce action associated with the error token in the specified state.  For example, a state which reduces rule A on lookahead token error satisfies this criteria.  Note that the state can have any number of other actions.  This option is useful when implementing error recovery. It makes the minimum number of modifications to the generated tables, whilst ensuring that if an error is encountered, the parser will reduce actions where necessary to reach the required error recovery rule.  See also the -N Option.

The -Rf option specifies that a default reduction will be introduced if there is a reduce action in the specified state.  If there is a reduce action associated with the error token, then this will be made default reduction (it is favouring error token reductions).  Otherwise, the most common reduction will be made the default reduction.  As for the -Re option, this option is useful when implementing error recovery.  It means that you get the table compression advantages of using default reductions, whilst ensuring that if an error is encountered, the parser will reduce actions where necessary to reach the required error recovery rule.  See also the -N Option.

The -Rd option is the default option.  It specifies that a default reduction will be introduced if there is a reduce action in the specified state.  If two or more rules are being reduced, then the most common reduction is chosen.  See also the -N Option.

See Also

-N Option


AYACC -S Option

Syntax	-S

Remarks

This option specifies that no default states will be introduced when compressing the parser's action table.  Ordinarily, if one state, say state A, contains actions that are present in another state, say state B, then it is possible to make state B the default state of state A.  Actions that are common between both states are then removed from state A.

A state B will become a default state of state A if it matches the following criterion:  Firstly, the default reductions for both states A and B must be same. If both states A and B do not have a default reduction, then this is also counted as being the same.  Secondly, the two values L and G are calculated for states A and B:

L	The number of lost actions.  This is the number of actions which are the same between states A and B.  These are the actions that are removed from state A if state B is made a default state.

G	The number of gained actions.  This is the number of error actions which are added to state A to deal with unwanted actions in state B.

If L is greater than G, then T = L - G represents the total number of lost actions.  If there is no other state B with a larger value of T, then state B is made the default state of state A.

If the -E Option is specified then there is an additional criterion placed on G, and that is it must always be zero.  In effect this means that state B always contains actions for the same tokens that are in state A.

See Also

-E Option


AYACC -T Option

Syntax	-T[c|cpp]

Remarks

This option specifies the target language for the generated parser.  Currently, there are two languages that a parser can be generated for.  These are C and C++.  The -T options, and their associated target languages are listed below:

Option	Target Language

-Tc	C (default).

-Tcpp	C++.

The default target language, if no -T option is specified, is C.

See Also

-m Option


AYACC -c Option

Syntax	-c

Remarks

This option specifies that compatible file names should be used.  This applies to the generated output, include and verbose files.  The compatible file name for the output file is y_tab with an extension.  The extension is either specified by the -e Option or the -T Option.  The -e option taking priority over the -T option.  If neither of these options are specified then the extension is .c.  Note that this corresponds to the default case for the -T option.

An include file is generated if the -d Option has been specified.  Only if it is used without a trailing file name will the -c option have an effect on the include file name.  In such cases, the file name for the include file is y_tab with an extension.  The extension can be specified by the -x Option.  Otherwise, the default extension .h is used.

A verbose file is generated if the -v Option has been specified.  Only if it is used without a trailing file name will the -c option have an effect on the verbose file name.  In such cases, the verbose file will be named y_output.  Note that it does not have an extension.

The compatible file names contain an underscore, where under UNIX they would have contained a period '.'.  This is to provide compatibility with MSDOS and Windows 3.1.  In such operating systems a period can only be used to separate the file name from its extension.  Under more advanced operating systems however, like Windows 95 and Windows NT, this is no longer the case and periods can be used anywhere within a file name, similar to UNIX.

See Also

-o Option, -d Option -d, -v Option, -e Option, -x Option


AYACC -d Option

Syntax	-d [file-name]

Remarks

This option specifies that an include file should be generated for the parser.  The include file contains the following items:

1	Any tokens specified by a Token Declaration.  These are introduced by the %token, %left, %right and %nonassoc keywords.

2	Any code placed in an Include Declaration.  These are introduced by the %include keyword, and have a following Declaration Code Block.

3	The attribute stack type defined by a Union Declaration.  These are introduced by the %union keyword, and have a following declaration code block.

4	The name of the parser specified by a Name Declaration,  These are introduced by the %name keyword.  This only applies to multiple model and multiple instance model C parsers and C++ parsers.  If you are generating a C++ parser, then a new class definition will be placed in the include file as well.

If an include file is not generated, then the information above is placed in the output file instead.

If the -d option is specified without a file name, then the generated include file name is based upon the output file name, except with a different extension.  By default, the extension is .h, however this can be changed by using the -x Option.

If the -d option is specified with a file name then it cancels the effects of the -c Option, with respect to the include file name, and the -x Option.  It is not possible to specify the -d option, with a trailing file name, more than once on the command line.  If you do so then an error will be generated.

See Also

-o Option, -x Option


AYACC -e Option

Syntax	-eextension

Remarks

This option specifies the file name extension for the output file.  If the first character of the extension is a period '.' then it is ignored i.e. it is treated as being the period between the file name and the extension.  If this option is omitted then a default extension is used.  This depends upon the target language for which the parser is being generated.  The list of target languages and associated default extensions are listed below:

Target Language	Default Extension

C (default)	.c

C++	.cpp

The target language is specified by the -T Option.

Providing no errors occur, an output file is always generated.  If the -o Option and the -c Option are not specified then the output file name is based upon the input file name, except with a different extension.  The extension is either the default extension, or the one specified by the -e option.  If the -c option is specified then the extension is instead added to the file name y_tab.  If the -o option is specified then the -e option is ignored.

If the -e option is specified more than once on the command line, then it is the last option specified which takes priority.  No error message is generated.

See Also

-o Option, -c Option, -x Option


AYACC -f Option

Syntax	-f

Remarks

This options specifies that #line directive file names are not treated like strings.  Instead they are treated as file names, similar to those used in #include directives.  The Microsoft Visual C++ compiler treats #line directive file names as strings.  The Borland C++ compiler on the other hand treats #line directive file names as proper file names.  The important point about strings is that they have an escape character '\'.  Unfortunately, in Windows this is the directory partitioning character.  Taking an example, the string "c:\\myproj\\mylexer.l" represents the file name c:\myproj\mylexer.l.

See Also

-l Option


AYACC -l Option

Syntax	-l

Remarks

This option prevents #line directives from being generated in the output file.  By default, #line directives are generated.  These give the C/C++ compiler information to map back from the output file to the Lex source file.  Sometimes, particularly during debugging, it is advantageous for these mappings not to take place.

See Also

Option -f


AYACC -m Option

Syntax	-m[s|m|i]

Remarks

This option specifies the parser model to be used.  This option only applies to C parsers.  A C++ parser is always considered as a multiple instance parser model.  There are three types of model, and they are described below:

1	Single parser model (-ms)

2	Multiple parser model (-mm)

3	Multiple instance parser model (-mi)

The single parser model means that you can only have one parser in your application.  This is the default model type for a C parser.  Trying to compile two single model parsers together in the same application will result in multiple symbol definition errors.

The multiple parser model is a step up from the single model.  You can now have more than one parser in the same application.  There are draw backs, however they are slight.  For instance there is an extra pointer de-referencing overhead caused by the introduction of a parser structure.  Furthermore actions can be slightly more complicated, especially with parser action functions like yyaccept etc.  These have a multiple parser equivalent.  In the case given, yyaccept would become yymaccept and take an extra parameter which points to the parser structure.  Note though that it is possible to include a header file MYCONV.H, which effectively converts yyaccept to yymaccept for you.

The multiple instance parser model is again a further refinement.  It allows multiple instances of a parser to be constructed.  Parsers of this type will always be constructed on the stack/heap.  They require the use of special creation and destruction routines yymcreateparse and yymdestroyparse.  As with the multiple parser model there is an include file MIYCONV.H, which can be used to convert single parser functions to multiple instance ones.

See Also

-T Option, Include Files


AYACC -o Option

Syntax	-o file-name

Remarks

This option specifies the file name for the generated output file.  This option can also affect the file name of the generated include file.  If you have specified the -d Option without a trailing file name, then the include file name is based on the output file name except with a different extension.  The extension applied to the include file name is either .h, the default extension, or an extension specified by the -x option.

This option cancels the effects of the -c Option, with respect to output and include file names, and the -e Option.  It is not possible to specify the -o option more than once on the command line.  If you do so then an error will be generated.

See Also

-d Option, -x Option


AYACC -p Option

Syntax	-p[s|la|c]

Remarks

This option specifies the type of parser that is created.  There are three types which are described below in descending order of power:

1	Simple LR or SLR (-ps)

2	Lookahead LR or LALR (-pla)

3	Canonical LR or CLR (-pc)

The LR part in the names mean that the parser traverses the input token sequence left to right, and forms a right-most derivation of the grammar.  Also note that SLR is an abbreviation of SLR(1).  The one indicates that the parser uses one token of lookahead.  The same can also be said of LALR and CLR.  That is that they are LALR(1) and CLR(1) parsers.

The Simple LR (SLR) is the least powerful parser that can be created by AYACC.  It is also the one that can be created the fastest, however with modern processor speeds this is not really relevant.  It is included for completeness.

The Lookahead LR (LALR) parser is somewhere between the SLR and CLR parsers in terms of power.  It has the same number of states as an SLR parser, however it makes use of lookahead information similar to a CLR parser.  This results in it having, potentially, fewer shift/reduce and reduce/reduce conflicts for a given grammar than an SLR parser.  This is the default parser that is created by AYACC.

The Canonical LR (CLR) parser is the most powerful parser that can be created by AYACC.  It has considerably more states than an SLR and LALR parser.  Only use in extremis.  If an LALR parser is generating large numbers of shift/reduce and reduce/reduce errors, it may be worth examining the grammar before switching to a CLR parser.  Although this is the most powerful parser that can be generated (by AYACC), it does not mean that it can create a parser for any grammar.  Quite the contrary.

See Also

-s Option


AYACC -s Option

Syntax	-s

Remarks

This option specifies that the generated parser should use a compact table representation.  By default a fast table representation is used.  A fast table is laid out as a number of large (fast) lookup arrays.  Taking the lookahead token action array as an example, each state has an index into the array.  The value of the current lookahead token is then added onto the index and the array element at that position then determines the action to perform.  As already stated these arrays are large.  There is also the possibility of unused holes appearing in the arrays.

A compact table on the other hand dispenses with large lookup arrays.  Instead it uses lists.  This results in a smaller, although slower parser.  Taking the lookahead token actions again, each state has a list of actions which can be performed.  The list is then traversed, trying to match an action with the current lookahead token.  If a match is found then the corresponding action is performed, otherwise a syntax error occurs.

There are different functions in the YACC and Lex library for dealing with parsers that have been built with either fast or compact tables.  The list of functions which are table specific are shown below.  Note that they are all C functions:

Fast Table Function	Compact Table Function

yydestructclearin	yycdestructclearin

yyparse	yycparse

yywipe	yycwipe

yywork	yycwork

yymdestructclearin	yymcdestructclearin

yymparse	yymcparse

yymwork	yymcwork

yymwipe	yymcwipe

See Also

-p Option


AYACC -v Option

Syntax	-v [file-name]

Remarks

This option specifies that a verbose file should be generated.  This describes every state and transition in the parser.  It also lists any shift/reduce and reduce/reduce conflicts which may have occurred.

If the -v option is specified without a file name, then the verbose file name can be one of two values.  If the -c Option is specified, then the verbose file name is y_output.  Otherwise, the verbose file name is based upon the output file name, except with the extension .v.

It is not possible to specify the -v option with a file name on the command line more than once.  If you do so then an error will be generated.

See Also

-c Option, -o Option


AYACC -x Option

Syntax	-xextension

Remarks

This option specifies the file name extension for the include file.  If the first character of the extension is a period '.' then it is ignored i.e. it is treated as being the period between the file name and the extension.  If this option is omitted then a default extension is used.  For the moment, the default extension is always assumed to be .h.

An include file is only generated if the -d Option is specified.  If the -d option is specified without a file name then the include file name is based upon the output file name, except with a different extension.  The extension is either the default extension .h, or the one specified by the -x option.  The include file name extension also applies to the output file name specified by the -c Option.  If the -d option is specified with a file name then the -x option is ignored.

If the -x option is specified more than once on the command line, then it is the last option specified which takes priority.  No error message is generated.

See Also

-o Option, -c Option, -e Option


AYACC -z Option

Syntax	-z

Remarks

This option specifies that short file names will be used in the generated output and include files.  In particular this option affects any #include and #line directives that are output to the files.  This option is primarily intended to provide support for 16-bit compilers, or those that cannot handle long file names.  Short file names are those that are limited to 8 file name characters and 3 file extension characters.  These file names are also sometimes called 8.3 file names.


ALex Operation

The command line format for ALex is:

ALex {options} input-file {options}

If no options or file name is given, then a brief summary of the command line format for ALex is displayed.

Input File

Specifies the name of the Lex source file.  Lex files generally have an extension of .l.

Option	Description

-? Option	Display help.

-$ Option	Suppress display of sign-on banner.

-E Option	No introduction of error transitions for default states.

-R Option	No default transitions.

-S Option	No default states.

-T Option	Specifies the target language of the generated lexical analyser.

-c Option	Use compatible file name lex_yy.

-d Option	No DFA state minimisation.

-e Option	Output file name extension (.c default for c, .cpp for C++).

-f Option	Do not treat #line directive file names as strings

-i Option	Generate include file.

-l Option	Do not produce #line directives in output file.

-m Option	Specifies the lexical analyser model.

-o Option	Output file name.

-r Option	Full reject capability.

-s Option	Use compact tables.

-t Option	No table compression. (Defunct option, although still supported.  Equivalent to the -E, -R and -S options combined.)

-u Option	Generate case insensitive lexical analyser.

-x Option	Include file name extension (.h default).

-z Option	Use short file names.

Note that the options are case sensitive, so -C is not the same as -c.  In fact it is an illegal option, and as such will generate an error.


ALex -? Option

Syntax	-?

Remarks

Displays a brief description of the command line format expected by ALex, plus a brief description of each option that it can take.  This option is ignored if there are any other options and/or input file specified on the command line.


ALex -$ Option

Syntax	-$

Remarks

Suppresses the display of the sign-on banner when the tool is run.  The sign-on banner consists of a version and copyright message.  Also included in the sign-on banner is a summary message output at the end of the compilation.  This gives the total number of errors and warnings found in the compilation.  Note that if a fatal error is encountered, then this message is never given.

The -$ option can appear anywhere on the command line.  It does not need to go at the start.  ALex does an initial scan of the command line looking for this option alone.


ALex -E Option

Syntax	-E

Remarks

This option specifies that error transitions will not be introduced when generating default states.  This option is closely associated with the -S Option.  Furthermore if the -S option is specified, then the -E option will have no effect on the generated state table.  For more information about default states see -S Option -S.

See Also

-S Option


ALex -R Option

Syntax	-R

Remarks

This option specifies that no default transitions are introduced when compressing the lexical analyser's state table.  Ordinarily, states are assigned a default transition, but only under the following criterion:  Firstly the two values L and G are calculated:

L	The number of lost transitions.  This is a count of the most common transition in the state i.e. the number of transitions going from this state to another different state.  These are the transitions that are removed from the state if it gains a default transition.  They also define what the default transition is.

G	The number of gained transitions.  This is a count of the non-existent, or missing, transitions between characters 0 to 255.  This is the number of error transitions added to the state to deal with the default transition.

If L is greater than G, then the state gains a default transition.

A state can only ever have either a default transition or a default start state.  Default transitions are always calculated first by ALex.

See Also

-S Option


ALex -S Option

Syntax	-S

Remarks

This option specifies that no default states will be introduced when compressing the lexical analyser's state table.  Ordinarily, if one state, say state A, contains transitions that are present in another state, say state B, then it is possible to make state B the default state of state A.  Transitions that are common between both states are then removed from state A.

A state B will become a default state of state A if it matches the following criterion.  Firstly, the default transitions for both states A and B must be the same.  If both states A and B do not have a default transition then this is also counted as being the same.  Secondly, the two values L and G are calculated for states A and B:

L	The number of lost transitions.  This is the number of transitions which are the same between states A and B.  These are the transitions that are removed from state A if state B is made a default state.

G	The number of gained transitions.  This is the number of error transitions which are added to state A to deal with unwanted transitions in state B.

If L is greater than G, then T = L - G represents the total number of lost transitions.  If there is no other state B with a larger value of T, then state B is made the default state of state A.

See Also

-E Option


ALex -T Option

Syntax	-T

Remarks

This option specifies the target language for the generated lexical analyser.  Currently, there are two languages that a lexical analyser can be generated for.  These are C and C++.  The -T options, and their associated target languages are listed below:

Option	Target Language

-Tc	C (default).

-Tcpp	C++.

The default target language, if no -T option is specified, is C.

See Also

-m Option


ALex -c Option

Syntax	-c

Remarks

This option specifies that compatible file names should be used.  This applies to the generated output and include files.  The compatible file name for the output file is lex_yy with an extension.  The extension is either specified by the -e Option or the -T Option.  The -e option taking priority over the -T option.  If neither of these options are specified then the extension is .c.  Note that this corresponds to the default case for the -T option.

An include file is generated if the -i Option has been specified.  Only if it is used without a trailing file name will the -c option have an effect on the include file name.  In such cases, the file name for the include file is lex_yy with an extension.  The extension can be specified by the -x Option.  Otherwise, the default extension .h is used.

The compatible file names contain an underscore, where under UNIX they would have contained a period '.'.  This is to provide compatibility with MSDOS and Windows 3.1.  In such operating systems a period can only be used to separate the file name from its extension.  Under more advanced operating systems however, like Windows 95 and Windows NT, this is no longer the case and periods can be used anywhere within a file name, similar to UNIX.

See Also

-o Option, -i Option -d, -e Option, -x Option


ALex -d Option

Syntax	-d

Remarks

This option disables DFA state minimisation.  This can lead to a faster compilation of the Lex source file, however with today's processor speeds this is not particularly relevant.  The downside to this is a probable increase in the size of the created lexical analyser.

DFA state minimisation is performed by default.  This stage in the lexical analyser creation process attempts to eliminate redundant states in the DFA.  It compares output transitions from a DFA state with ones from another.  If all transitions from both states are the same, then the two DFA states can be merged to form a single DFA state.  This does not, in any way, affect the patterns that are recognised by the lexical analyser.

See Also

-E Option, -R Option, -S Option


ALex -e Option

Syntax	-e[c|cpp]

Remarks

This option specifies the file name extension for the output file.  If the first character of the extension is a period '.' then it is ignored i.e. it is treated as being the period between the file name and the extension.  If this option is omitted then a default extension is used.  This depends upon the target language for which the parser is being generated.  The list of target languages and associated default extensions are listed below:

Target Language	Default Extension

C (default)	.c

C++	.cpp

The target language is specified by the -T Option.

Providing no errors occur, an output file is always generated.  If the -o Option and the -c Option are not specified then the output file name is based upon the input file name, except with a different extension.  The extension is either the default extension, or the one specified by the -e option.  If the -c option is specified then the extension is instead added to the file name lex_yy.  If the -o option is specified then the -e option is ignored.

If the -e option is specified more than once on the command line, then it is the last option specified which takes priority.  No error message is generated.

See Also

-o Option, -c Option, -x Option


ALex -f Option

Syntax	-f

Remarks

This option specifies that #line directive file names are not treated like strings.  Instead they are treated as file names, similar to those used in #include directives.  The Microsoft Visual C++ compiler treats #line directive file names as strings.  The Borland C++ compiler on the other hand treats #line directive file names as proper file names.  The important point about strings is that they have an escape character '\'.  Unfortunately, in Windows this is the directory partitioning character.  Taking an example, the string "c:\\myproj\\mylexer.l" represents the file name c:\myproj\mylexer.l.

See Also

-l Option


ALex -i Option

Syntax	-i [file-name]

Remarks

This option specifies that an include file should be generated for the lexical analyser.  The include file contains the following items:

1	Any start states specified by a Start Declaration.  These are introduced by the %start and %exclusive keywords.

2	Any code placed in an Include Declaration.  These are introduced by the %include keyword, and have a following Declaration Code Block.

3	The name of the lexical analyser specified by a Name Declaration,  These are introduced by the %name keyword.  This only applies to multiple model and multiple instance model C lexical analysers and C++ lexical analysers.  If you are generating a C++ lexical analyser, then a new class definition will be placed in the include file as well.

If an include file is not generated, then the information above is placed in the output file instead.

If the -i option is specified without a file name, then the generated include file name is based upon the output file name, except with a different extension.  By default, the extension is .h, however this can be changed by using the -x Option.

If the -i option is specified with a file name then it cancels the effects of the -c Option, with respect to the include file name, and the -x Option.  It is not possible to specify the -i option, with a trailing file name, more than once on the command line.  If you do so then an error will be generated.

See Also

-o Option, -x Option


ALex -l Option

Syntax	-l

Remarks

This option prevents #line directives from being generated in the output file.  By default, #line directives are generated.  These give the C/C++ compiler information to map back from the output file to the Lex source file.  Sometimes, particularly during debugging, it is advantageous for these mappings not to take place.

See Also

Option -f


ALex -m Option

Syntax	-m[s|m|i]

Remarks

This option specifies the lexical analyser model to be used.  This option only applies to C lexical analysers.  A C++ lexical analyser is always considered as a multiple instance lexical analyser model.  There are three types of model, and they are described below:

1	Single lexical analyser model (-ms)

2	Multiple lexical analyser model (-mm)

3	Multiple instance lexical analyser model (-mi)

The single lexical analyser model means that you can only have one lexical analyser in your application.  This is the default model type for a C lexical analyser.  Trying to compile two single model lexical analysers together in the same application will result in multiple symbol definition errors.

The multiple lexical analyser model is a step up from the single model.  You can now have more than one lexical analyser in the same application.  There are draw backs, however they are slight.  For instance there is an extra pointer de-referencing overhead caused by the introduction of a lexical analyser structure.  Furthermore actions can be slightly more complicated, especially with lexical analyser action functions like yymore etc.  These have a multiple parser equivalent.  In the case given, yymore would become yymmore and take an extra parameter which points to the lexical analyser structure.  Note though that it is possible to include a header file MLCONV.H, which effectively converts yymore to yymmore for you.

The multiple instance lexical analyser model is again a further refinement.  It allows multiple instances of a lexical analyser to be constructed. Lexical analysers of this type will always be constructed on the stack/heap.  They require the use of special creation and destruction routines yymcreatelex and yymdestroylex.  As with the multiple lexical analyser model there is an include file MILCONV.H, which can be used to convert single lexical analyser functions to multiple instance ones.

If this option is specified for a C++ parser then it is ignored.  If the option is specified more than once on the command line, then it is the option that is specified last that is the one used.  No error is generated.

See Also

Option -T, Include Files


ALex -o Option

Syntax	-o file-name

Remarks

This option specifies the file name for the generated output file.  This option can also affect the file name of the generated include file.  If you have specified the -i Option without a trailing file name, then the include file name is based on the output file name except with a different extension.  The extension applied to the include file name is either .h, the default extension, or an extension specified by the -x option.

This option cancels the effects of the -c Option, with respect to output and include file names, and the -e Option.  It is not possible to specify the -o option more than once on the command line.  If you do so then an error will be generated.

See Also

-i Option, -x Option


ALex -r Option

Syntax	-r

Remarks

This option specifies that every regular expression which recognises a string can be matched.  This option is for use with the yyreject and yymreject functions and the yylexer::yyreject member function, all of which hereafter are referred to as yyreject.  Ordinarily, a string can only be matched by one regular expression.  If a string is recognised by two or more regular expressions, then it is always the expression that is listed first in the Lex source file that matches it.  In such circumstances, the next matched string after a call to yyreject will always be shorter than the previous one.  The one notable exception being a string of length one.  In which case, the character is matched by the default rule, output, and then discarded.

With this option selected, the next matched string after a call to yyreject may be the same length as the previous one.  If there is a regular expression further down the Lex source file which recognises the string then it is matched.  Otherwise, as before the next matched string will be shorter.  A consequence of using this option is that no unmatched expression warnings will be generated by Lex.

See Also

yyreject, yymreject, yylexer::yyreject


ALex -s Option

Syntax	-s

Remarks

This option specifies that the generated lexical analyser should use a compact table representation.  By default a fast table representation is used.  A fast table is laid out as a number of large (fast) lookup arrays.  For the transition array, each state has an index into the array.  The value of the current lookahead character is then added onto the index and the array element at that position then determines whether a transition is possible.  As already stated these arrays are large.  There is also the possibility of unused holes appearing in the arrays.

A compact table on the other hand dispenses with large lookup arrays.  Instead it uses lists.  This results in a smaller, although slower lexical analyser.  In this case, each state has a list of transitions which can be performed.  The list is then traversed, trying to match a transition with the current lookahead character.  If a match is found then the corresponding transition is performed, otherwise backup starts in earnest, looking for a pattern match.

There are different functions in the YACC and Lex library for dealing with lexical analysers that have been built with either fast or compact tables.  The list of functions which are table specific are shown below.  Note that they are all C functions:

Fast Table Function	Compact Table Function

yylex	yyclex

yymlex	yymclex


ALex -t Option

Syntax	-t

Remarks

This option specifies that the state transition tables generated by ALex should not be compressed in any way.  This is really a defunct option, but is supported for compatibility reasons.  The option is equivalent to specifying the -E Option, -R Option and the -S Option.

See Also

-E Option, -R Option, -S Option


ALex -u Option

Syntax	-u

Remarks

This options specifies that a case insensitive lexical analyser should be generated.  Any letter used in a regular expression will represent both an upper and lower case version of the letter.  In effect a letter will be substituted by a character class containing both an upper and lower case version of the letter.  For instance, the character 'a' would be replaced with the character class '[Aa]'.  This option can significantly reduce the amount of typing that has to be done for case insensitive lexical analyser's.  It can also increase the readability of the source code.  For instance, the example below shows two implementations of the BASIC keyword THEN.  The second case is where the -u option is used:

[Tt][Hh][Ee][Nn]    { return TK_THEN; }

THEN                { return TK_THEN; }

Note that the string stored in the text buffer is not affected in anyway.  It stores exactly what is input.  By default, a case sensitive lexical analyser is generated.


ALex -x Option

Syntax	-xextension

Remarks

This option specifies the file name extension for the include file.  If the first character of the extension is a period '.' then it is ignored i.e. it is treated as being the period between the file name and the extension.  If this option is omitted then a default extension is used.  For the moment, the default extension is always assumed to be .h.

An include file is only generated if the -i Option is specified.  If the -i option is not specified with a file name then the include file name is based upon the output file name, except with a different extension.  The extension is either the default extension .h, or the one specified by the -x option.  The include file name extension also applies to the output file name specified by the -c Option.  If the -d option is specified with a file name then the -x option is ignored.

If the -x option is specified more than once on the command line, then it is the last option specified which takes priority.  No error message is generated.

See Also

-o Option, -c Option, -e Option


ALex -z Option

Syntax	-z

Remarks

This option specifies that short file names will be used in the generated output and include files.  In particular this option affects any #include and #line directives that are output to the files.  This option is primarily intended to provide support for 16-bit compilers, or those that cannot handle long file names.  Short file names are those that are limited to 8 file name characters and 3 file extension characters.  These file names are also sometimes called 8.3 file names.


AYACC Errors

Fatal Errors

Y1001	Y1002	Y1003	Y1004

Y1005	Y1006	Y1007	Y1008

Y1009	Y1010	Y1011	Y1012

Y1013	Y1014	Y1015	Y1016

Y1017	Y1018

Errors

Y2001	Y2002	Y2003	Y2004

Y2005	Y2006	Y2007	Y2008

Y2009	Y2010	Y2011	Y2012

Y2013	Y2014	Y2015	Y2016

Y2017	Y2018	Y2019	Y2020

Y2021	Y2022	Y2023	Y2024

Y2025	Y2026	Y2027	Y2028

Y2029	Y2030	Y2031

Warnings

Y4001	Y4002	Y4003	Y4004

Y4005	Y4006	Y4007	Y4008

Y4009	Y4010	Y4011


AYACC Error Y1001
AYACC Fatal Error Y1001

out of memory

Your computer has run out of memory. This can be solved by closing any unnecessary tasks, using a simpler parser generation technique (LALR instead of CLR, SLR instead of LALR), pruning your grammar or purchasing more RAM for you machine.


AYACC Error Y1002
AYACC Fatal Error Y1002

unexpected end of file found

An unexpected end of file marker was encountered. This may be caused by forgetting a second quote around a literal.


AYACC Error Y1003
AYACC Fatal Error Y1003

unexpected end of file found in comment

A comment starting with /* did not have a terminating */. Similarly, a comment starting with // did not have a terminating newline character.


AYACC Error Y1004
AYACC Fatal Error Y1004

unable to open file 'file-name'

AYACC could not open the file file-name for reading/writing. This can be caused by the file being non existent, or already open.


AYACC Error Y1005
AYACC Fatal Error Y1005

file output error

AYACC could not write to a file. This generally means that the disk you are trying to write to is full. Note that AYACC deletes output files on fatal errors.


AYACC Error Y1006
AYACC Fatal Error Y1006

invalid option 'option'

An invalid option option was specified on the command line.


AYACC Error Y1007
AYACC Fatal Error Y1007

duplicate source file 'file-name'

AYACC can only process one source file at a time. Remove a secondary source file from the command line.


AYACC Error Y1008
AYACC Fatal Error Y1008

no source file

AYACC must have a source file to process.


AYACC Error Y1009
AYACC Fatal Error Y1009

file expected after option 'option'

An option was used which expected a file name to be specified after it.


AYACC Error Y1010
AYACC Fatal Error Y1010

duplicate output file 'file-name'

An output file name was specified a second time on the command line.


AYACC Error Y1011
AYACC Fatal Error Y1011

unexpected end of file found in code block

A code block introduced with the %{ character sequence did not have a terminating %}.


AYACC Error Y1012
AYACC Fatal Error Y1012

unexpected end of file found in declaration code block

A declaration code block introduced with the { character did not have a terminating }.


AYACC Error Y1013
AYACC Fatal Error Y1013

unexpected end of file found in action

The terminating character for an action was not found. This will in most cases be }, although it can also be ;.


AYACC Error Y1014
AYACC Fatal Error Y1014

duplicate verbose file ‘file-name’

A verbose file name was specified a second time on the command line.


AYACC Error Y1015
AYACC Fatal Error Y1015

unexpected end of file found in action comment

A comment in an action was not terminated correctly.  If a block comment is introduced with the /* character sequence then it should have a terminating */ character sequence.  Correspondingly, if a line comment is introduced with the // character sequence, then it should have a terminating newline character.


AYACC Error Y1016
AYACC Fatal Error Y1016

unexpected end of file found in code block comment

A comment in a code block %{…%} was not terminated correctly.  If a block comment is introduced with the /* character sequence then it should have a terminating */ character sequence.  Correspondingly, if a line comment is introduced with the // character sequence, then it should have a terminating newline character.


AYACC Error Y1017
AYACC Fatal Error Y1017

unexpected end of file found in declaration code block comment

A comment in a declaration code block {…} was not terminated correctly.  If a block comment is introduced with the /* character sequence then it should have a terminating */ character sequence.  Correspondingly, if a line comment is introduced with the // character sequence, then it should have a terminating newline character.


AYACC Error Y1018
AYACC Fatal Error Y1018

duplicate include file 'file-name'

An include file name was specified a second time on the command line.


AYACC Error Y2001

line too long

The line read in from the input file was too long and had to be truncated. The maximum length of a line is 2048 characters.


AYACC Error Y2002

newline in character token

A character token did not have a terminating single quote character '. Note that character tokens cannot cross a line boundary.


AYACC Error Y2003

empty character token

A character token must contain at least one character.


AYACC Error Y2004

unrecognised character '0xxx'

The character xx read from the input file was not recognised by AYACC. The character is simply discarded and the next one read.


AYACC Error Y2005

invalid keyword 'keyword'

The keyword keyword was not recognised by AYACC. A keyword always begins with %, the AYACC escape character.


AYACC Error Y2006

invalid yacc escape sequence

The character following the AYACC escape character % was not recognised as valid. The escape character is deleted, and the next character is analysed independently.


AYACC Error Y2007

syntax error

A syntax error has occurred. It is your responsibility to determine the cause of the problem.


AYACC Error Y2008

syntax error : missing 'token' before 'token'

A syntax error has occurred. The token token was expected but not found.


AYACC Error Y2009

syntax error : 'token'

A syntax error has occurred. The token token was found, but not expected.


AYACC Error Y2010

re-declaration of start symbol

The %start keyword has been used more than once to declare a nonterminal as the initial rule.


AYACC Error Y2011

'token' : re-declaration of token associativity

The associativity of the token token, declared using %left, %right and %nonassoc, has been changed.


AYACC Error Y2012

'token' : re-declaration of token precedence

The precedence of the token token, declared using %left, %right and %nonassoc, has been changed.


AYACC Error Y2013

'symbol' : re-declaration of symbol type

The type associated with a token or nonterminal has been changed.


AYACC Error Y2014

re-initialisation of 'token'

The value associated with the token token has been changed.


AYACC Error Y2015

duplicate union declaration

A union has been declared a second time.


AYACC Error Y2016

undefined nonterminal 'nonterminal'

The nonterminal nonterminal was used in a rule, but never defined.


AYACC Error Y2017

'symbol' : already defined as a token

A token cannot subsequently be declared as a nonterminal.


AYACC Error Y2018

'symbol' : must be a token

A token is expected, not a nonterminal.


AYACC Error Y2019

re-declaration of rule precedence

A syntax error has occurred. A grammar symbol, either a nonterminal or token is expected.


AYACC Error Y2020

empty attribute tag

A tag in an action cannot be empty. It must contain a string of characters identifying a member of a union.


AYACC Error Y2021

attribute offset too big

The offset in an action variable is too big. The number overflowed the internal representation used to store it. At present this is as a 32-bit signed integer. Note that this value will be exceeded in practical terms, long before this occurs.


AYACC Error Y2022

tag missing '>'

A tag did not have a terminating >.  This may not necessarily mean that the > is missing.  The characters between the < and > delimiters must follow a very strict format.  The first character must be a letter or the underscore character '_'.  Following characters can then either be a letter, digit or underscore.


AYACC Error Y2023

'integer' : integer too big

The integer is too big. The number would not fit in the internal representation used to store it. At present this is a 32-bit signed integer.


AYACC Error Y2024

empty tag

A tag cannot be empty. It must contain a string of characters identifying a member of a union.


AYACC Error Y2025

‘symbol’ : already declared as a nonterminal

The symbol has already been declared elsewhere as a nonterminal, when what is required is a token.


AYACC Error Y2026

‘symbol’ : already declared as a token

The symbol has already been declared elsewhere as a token, when what is required is a nonterminal.


AYACC Error Y2027

'symbol' : duplicate symbol destructor definition

A destructor for a grammar symbol, either a nonterminal or token, was defined more than once.  It is only possible to specify one destructor for each grammar symbol.  Note that symbol destructors are optional.  You can have destructors for some symbols, but leave others if they do not require them.  An example of this type of error is shown below:

~A : { /* do somrthing */ } ;

...

~A : { /* do something else */ } ;


AYACC Error Y2028

'symbol' : symbol type is different

This error occurs when trying to tie several symbol destructors to the same action.  All of the grammar symbols on the left hand side must have the same type.  Of course, if types (tags) are not associated with grammar symbols then this error cannot occur.  An example is shown below:

%type <node> A

%type <node> B

%type <symbol> C

%%

...

~A | ~B | ~C : { deletenode($$); }

Here we have the beginnings of a rudimentary node tree structure.  Nonterminal symbols A and B have type node and nonterminal symbol C has type symbol.  In the destructor we have grouped A, B and C together.  A and B are OK since they have the same type, however when we reach C trouble occurs.  What would $$ in the action refer to? The node or the symbol.


AYACC Error Y2029
AYACC Error 2029

hex integers should have at least one hex digit

A hexadecimal integer has been introduced with 0x or OX, however no trailing hexadecimal digits follow.


AYACC Error Y2030
AYACC Error 2030

duplicate name declaration

The name of the parser has been declared a second time.


AYACC Error Y2031
AYACC Error 2031

'token' : token number is not unique

The token number assigned to the specified token is not unique i.e. it clashes with a token number assigned to another token.  You will need to assign a different token number to the token in question, or else find out which token caused the clash and change that instead.  Remember that AYACC declares two tokens automatically for you.  These are $end and error.  They are assigned token numbers of 0 and 256 respectively.  Note that they cannot be changed.


AYACC Error Y4001
AYACC Warning Y4001

‘ch’ : unrecognised character escape sequence

An invalid character escape sequence was used in a character token.


AYACC Error Y4002
AYACC Warning Y4002

hex characters must have at least one hex digit

A hex character specified in a character token must have at least one hex digit. A hex character is started with the escape sequence \x, and has one or more trailing hex digits.


AYACC Error Y4003
AYACC Warning Y4003

shift-reduce conflict on 'token'

A shift-reduce conflict occurred whilst trying to build the parser.  The problem centres around the specified lookahead token.  The parser wants to be able to shift the token onto the stack and reduce a rule at the same time.  Obviously it can't do both.  The conflict is resolved by always shifting the token onto the stack.  The line number given in the error message refers to the rule that needed to be reduced.  There can also be reduce-reduce conflicts, however there can never be a shift-shift conflict.

The presence of shift-reduce (and reduce-reduce) conflicts gives rise to ambiguously specified grammars.  The resultant generated parser may (or may not) work as you had intended.  In order to be sure you should generate a verbose file and examine its contents.  The location of all shift-reduce (and reduce-reduce) errors is shown at the bottom of the file.  Alternatively you can do a search on "shift-reduce".

If the parser is not doing what you wanted, then you will most probably need to re-specify your grammar.  The other alternative is to move to a more powerful parser i.e. from an LALR parser to a CLR parser.  However only do this in extremis.  Even then it is not guaranteed that the parser will be free of all shift-reduce and reduce-reduce conflicts.  Quite the contrary.


AYACC Error Y4004
AYACC Warning Y4004

reduce-reduce conflict on 'token'

A reduce-reduce conflict occurred whilst trying to build the parser.  The problem centres around the specified lookahead token.  The parser wants to be able to reduce two different rules at the same time.  Obviously it can't do both.  The conflict is resolved by always reducing the rule that is specified first in the YACC source file.  The line number given in the error message refers to the rule that is not reduced.  There can also be shift-reduce conflicts, however there can never be a shift-shift conflict.

The presence of reduce-reduce (and shift-reduce) conflicts gives rise to ambiguously specified grammars.  The resultant generated parser may (or may not) work as you had intended.  In order to be sure you should generate a verbose file and examine its contents.  The location of all reduce-reduce (and shift-reduce) errors is shown at the bottom of the file.  Alternatively you can do a search on "reduce-reduce".

If the parser is not doing what you wanted, then you will most probably need to re-specify your grammar.  The other alternative is to move to a more powerful parser i.e. from an LALR parser to a CLR parser.  However only do this in extremis.  Even then it is not guaranteed that the parser will be free of all reduce-reduce and shift-reduce conflicts.  Quite the contrary.


AYACC Error Y4005
AYACC Warning Y4005

'value' : too big for character

The value of a hexadecimal or octal escape in a character token was too big to fit into a character.  The value must be between 0 and 255 inclusive.


AYACC Error Y4006
AYACC Warning Y4006

'token' : token is declared but never used

The token is declared but never used in a rule.  Simply delete the offending token.


AYACC Warning Y4007

rule is never reduced

The rule is never reduced.  This can be for one of two reasons.  The first, and simplest is that the grammar does not make reference to the rule.  Consider the example below:

A: 'a' ;

B: 'b' ;

Here, the starting rule is A because it is defined first.  However because A does not make reference to B, B will never be reduced.  The second, and more obscure reason why rules might not be reduced, is through the introduction of shift-reduce and reduce-reduce conflicts.  In both these cases for each conflict a reduction on the given token is lost.  These losses may be sufficient to mean that a rule will never be reduced.  To solve these types of error generate a verbose file and examine its contents.


AYACC Warning Y4008

'nonterminal' : nonterminal is declared but never used

The nonterminal is declared but never used in a rule.  Simply delete the offending symbol.


AYACC Warning Y4009

'attribute' : attribute is untyped

The attribute used in the action was untyped.  This can be for one of two reasons.  The first is that the attribute refers to a symbol which has not been typed.  If the symbol is a nonterminal then you can assign a type to it using the %type keyword.  Alternatively, if the symbol is a token you can do the same thing using the %token keyword.

%type <node> A

%token <symbol> a

Note that types can only be specified in the declarations section of your YACC source file.  You can also specify token types using the %left, %right keywords etc. in much the same way as %token.

The other reason that the attribute is untyped is that it does not refer to any symbol in the rule.  In such circumstances, the attribute type has to be specified explicitly.  Of particular importance is the $0 attribute, which is often used for passing information to a rule.  For example $0 would then become $<type>0 in the action.


AYACC Warning Y4010

default copy action uses different types

The default copy action for the rule uses attributes with different types.  The default copy action is only used if the right hand side of the rule has one or more symbols and no action.  To remove the warning you must make sure that both $$ and $1 have the same type.  Failing that you can provide your own action which does the necessary conversion.


AYACC Warning Y4011

accept-reduce conflict on 'token'

An accept-reduce conflict occurred whilst trying to build the parser.  The problem centres around the specified lookahead token.  However, this will, in all cases, be the $end token.  The parser wants to be able to accept the input and reduce a rule at the same time.  Obviously it can't do both.  The conflict is resolved by always accepting the input.  The line number given in the error message refers to the rule that needed to be reduced.

The presence of accept-reduce conflicts gives rise to ambiguously specified grammars.  The resultant generated parser may (or may not) work as you had intended.  In order to be sure you should generate a verbose file and examine its contents.  The location of all accept-reduce errors is shown at the bottom of the file.  Alternatively you can do a search on "accept-reduce".

This is a very rare error, and its presence means that you have done something seriously wrong with the root of your grammar.  The simplest grammar which generates this error is shown below i.e. it is recursive and does not have a terminating rule:

A: A;

In all other respects, this error can be thought of as similar to a reduce-reduce conflict.


ALex Errors

Fatal Errors

L1001	L1002	L1003	L1004

L1005	L1006	L1007	L1008

L1009	L1010	L1011	L1012

L1013	L1014	L1015	L1016

L1017	L1018

Errors

L2001	L2002	L2003	L2004

L2005	L2006	L2007	L2008

L2009	L2010	L2011	L2012

L2013	L2014	L2015	L2016

L2017	L2018	L2019	L2020

L2021	L2022	L2023

Warnings

L4001	L4002	L4003	L4004

L4005	L4006	L4007	L4008

L4009	L4010	L4011	L4012

L4013


ALex Error L1001
ALex Fatal Error L1001

out of memory

The computer has run out of memory. This is a serious problem. It can be solved by either increasing the size of the available free RAM, or decreasing the complexity of the regular expressions in the source file.


ALex Error L1002
ALex Fatal Error L1002

unexpected end of file found

An end-of-file marker was unexpectedly found.


ALex Error L1003
ALex Fatal Error L1003

unexpected end of file found in comment

A comment starting with /* did not have a terminating */ or a comment starting with // did not have a terminating newline.


ALex Error L1004
ALex Fatal Error L1004

unable to open file ‘file-name’

The source file file-name could not be found.


ALex Error L1005
ALex Fatal Error L1005

file output error

An error occurred when writing to the output file. This will usually be because the disk is full.


ALex Error L1006
ALex Fatal Error L1006

invalid option 'option'

An invalid option was specified on the command line.


ALex Error L1007
ALex Fatal Error L1007

duplicate source file ‘file-name’

A duplicate source file was specified on the command line.


ALex Error L1008
ALex Fatal Error L1008

no source file

A source file was not specified on the command line.


ALex Error L1009
ALex Fatal Error L1009

file expected after option 'option'

A file name was expected after the option.


ALex Error L1010
ALex Fatal Error L1010

duplicate output file ‘file-name’

A duplicate output file was specified on the command line using the -o option.


ALex Error L1011
ALex Fatal Error L1011

maximum number of errors reached

The maximum number of errors was reached. When ALex encounters 100 errors it stops compiling and issues this message.  Note that warnings are not included in the error count.


ALex Error L1012
ALex Fatal Error L1012

unexpected end of file found in action

An unexpected end of file was found in an action.  This can be caused by a missing right bracket '}' or semi-colon ';'.  Note that ALex keeps track of the number of opening and closing brackets that it sees.  Only when that count reaches zero does it consider the action finished.  ALex also recognises certain C tokens like character strings "a", character constants 'a' and comments /*…*/ and //… Any brackets or semicolons found in these will be ignored.


ALex Error L1013
ALex Fatal Error L1013

unexpected end of file found in code block

An unexpected end of file was found in %{…%}.  Or, more to the point a terminating %} was never seen.  Certain C tokens are recognised in %{…%}.  These are character strings "a", character constants 'a', and comments /*…*/ and //… A %} sequence embedded in one of these tokens will be ignored by AYACC.


ALex Error L1014
ALex Fatal Error L1014

unexpected end of file found in action comment

An unexpected end of file was found in a comment inside an action.  ALex recognises certain C tokens in actions.  These include both block comments /*…*/ and line comments //…


ALex Error L1015
ALex Fatal Error L1015

unexpected end of file found in code block comment

An unexpected end of file was found in a comment inside %{…%}.  ALex recognises certain C tokens in %{…%}.  These include both block comments /*…*/ and line comments //…


ALex Error L1016
ALex Fatal Error L1016

duplicate include file 'file-name'

A duplicate include file was specified on the command line using the -i option.


ALex Error L1017
ALex Fatal Error L1017

unexpected end of file found in declaration code block

A declaration code block introduced with the { character did not have a terminating }.


ALex Error L1018
ALex Fatal Error L1018

unexpected end of file found in declaration code block comment

A comment in a code block %{…%} was not terminated correctly.  If a block comment is introduced with the /* character sequence then it should have a terminating */ character sequence.  Correspondingly, if a line comment is introduced with the // character sequence, then it should have a terminating newline character.


ALex Error L2001

line too long

The line is too long. The maximum number of characters allowed on a line is 2048. This error can be caused by a recursive macro call on the offending line.


ALex Error L2002

newline in character string

A character string cannot cross a line boundary. In order to include a newline in the character string use the escape sequence \n.


ALex Error L2003

empty character string

A character string does not contain any characters i.e. it appears in the source file as "".


ALex Error L2004

unrecognised character '0xhh'

A character from the input stream was not recognised as valid by ALex.


ALex Error L2005

invalid keyword 'keyword'

An invalid keyword was specified. A keyword always starts with the percent % sign, which is the ALex escape character. The keywords currently recognised are %start and %table.  There are also a number of synonyms for these keywords %Start, %S, %s and %T.  Finally there are a number of  keywords that are recognised for portability purposes %a, %e, %k, %n, %o, %p and %option.


ALex Error L2006

invalid lex escape sequence

An invalid ALex escape sequence was found in the source file. They are introduced by the percent % sign.


ALex Error L2007

syntax error

A syntax error occurred in the source file.


ALex Error L2008

syntax error : missing 'token' before 'token'

A syntax error occurred in the source file. The token token was expected but not found.


ALex Error L2009

syntax error : 'token'

A syntax error occurred in the source file. An unexpected token token was found.


ALex Error L2010

empty character class

A character class does not contain any characters i.e. it appears in the in the source file as [].


ALex Error L2011

undefined macro 'macro-name'

An undefined macro macro-name was encountered in the rules section of the ALex source file. This error is either caused by an incorrect macro call, or a non-existent macro definition.


ALex Error L2012

newline in character escape

A character escape cannot cross a line boundary. In order to specify the newline character use the character escape sequence \n.


ALex Error L2013

'value' : integer out of range

The integer value is out of range. It must be within the range 0 to 255 inclusive.


ALex Error L2014

hex integers should have at least one hex digit

A hexadecimal integer has been introduced with 0x or OX, however no trailing hexadecimal digits follow.


ALex Error L2015

out of macro expansion space

The line buffer has overflowed whilst ALex was trying to expand a macro.  This may be caused by a recursive macro definition and reference.  ALex does not explicitly check for recursive macros, it waits for the line buffer to overflow vis-à-vis this error message. Below is an example:

recurse a{recurse}

%%

{recurse}


Alex Error L2016

‘start-state’ : undeclared start state

An undeclared start state was encountered in a rule.  All start states that are used by the rules must first be declared in the declarations section.  This is done through the use of the %start keyword.


ALex Error L2017

macro reference does not have closing '}'

A macro reference in a regular expression does not have a closing right bracket '}'.  An example is shown below:

digit       [0-9]

%%

{digit+     { /* integer */ }

Note the missing '}' between "digit" and the plus sign.


ALex Error L2018

'macro-name' : duplicate macro declaration

The macro macro-name has been declared previously in the source file.  You must remove one of the offending declarations before compilation can preceed.


ALex Error 2019
ALex Error L2019

'start-state' : re-declaration of start state exclusivity

The specified start state start-state has been re-declared with a different exclusivity.  For example, it is not possible to declare a start state with the %start keyword, and then subsequently re-declare it with the %exclusive keyword, or vice-versa.


ALex Error 2020
ALex Error L2020

duplicate name declaration

The name of the lexical analyser has been declared a second time.


ALex Error 2021
ALex Error L2021

newline in character class

A character class cannot cross a line boundary. In order to include a newline in the character class use the escape sequence \n.


ALex Error 2022
ALex Error L2022

empty character class expression

A character class contains an empty character class expression.  This means that the character sequence [::] appears within a character class somewhere.  The simplest example of which is [[::]].  ALex typically expects a name denoting a character set to appear between the colons of a character class expression.  For instance, the character class expression [:alpha:] denotes all alphabetic characters, or letters.


ALex Error L2023

invalid character class expression ‘character-class-expression’

An unrecognised character class expression was specified in a character class.  The currently recognised character class expressions are [:alnum:], [:alpha:], [:blank:], [:cntrl:], [:digit:], [:graph:], [:lower:], [:print:], [:punct:], [:space:], [:upper:] and [:xdigit:].


ALex Error L4001
ALex Warning L4001

hex characters must have at least one hex digit

A hex character specified in a character string or character class must have at least one hex digit. A hex character is started with the escape sequence \x, and has one or more trailing hex digits.


ALex Error L4002
ALex Warning L4002

hex character too big

A hex character must not be greater than 0xff or 255.


ALex Error L4003
ALex Warning L4003

octal character too big

An octal character must not be greater than 0377 or 255.


ALex Error L4004
ALex Warning L4004

'value' : integer too big

An integer value must not be greater than 255.


ALex Error L4005
ALex Warning L4005

'start-state' : duplicate start state

A duplicate start state was specified in the list of start states at the beginning of a regular expression.


ALex Error L4006
ALex Warning L4006

character too big

A character must not be greater than 255.


ALex Error L4007
ALex Warning L4007

'0xhh' : duplicate table entry

A duplicate table entry was specified in the %table command. The previous entry is replaced.


ALex Error L4008
ALex Warning L4008

expression is never matched

The regular expression on the given line is never matched. This may be caused by a duplicate regular expression, or improper ordering of the regular expressions in the source file.  Note that if two regular expressions match the same string, then the first regular expression listed in the file is matched. An example is shown below:

%%

[a-zA-Z_][a-zA-Z0-9_]* { return tok_identifier; }

if                     { return tok_if; }

This (simple) example matches C style identifiers and the if keyword.  The important case is the keyword. Note that both regular expressions match the string “if”.  However because the regular expression for the identifier is listed first it will match it.  The second regular expression will never be matched.  The answer to this problem is to swap the two regular expressions over.

%%

if                     { return tok_if; }

[a-zA-Z_][a-zA-Z0-9_]* { return tok_identifier; }

Now the string “if” will still be matched by the first regular expression, only this time it is the one for the if keyword.


ALex Error L4009
ALex Warning L4009

‘start-state’ : duplicate start state declaration

The indicated start state has been declared earlier in the source file. Simply delete the offending symbol.


ALex Error L4010
ALex Warning L4010

expression can backup completely

The regular expression can backup completely.  Be careful of these expressions.  They are dangerous and can lead to the lexical analyser looping indefinitely.  An example of an expression that can backup completely is shown below.  Note that it makes use of the right context operator /:

a?/bc  { /* do something */}

The expression matches the two strings "abc", and "bc".  However in the second case all of the characters will be backed up and placed in the unput buffer.  This may be intentional or accidental.  In either case you need to deal with it.  If it is intentional then you need to make sure that the string is not matched again.  You can do this by, for instance, changing the start state.  On the other hand, if it is accidental then you will need to revise the regular expression.


ALex Error L4011
ALex Warning L4011

expression will backup completely

The regular expression will backup completely.  Be careful of these expressions.  They are dangerous and can lead to the lexical analyser looping indefinitely.  An example of an expression that will backup completely is shown below.  Note that it makes use of the right context operator /:

/"howdy"    { /* do something (like changing start states) */ }

The expression will match the string "howdy", and then backup all of the characters onto the undo buffer.  Typically, you will do something in the action to make sure that you do not match this string again i.e. change start state.


ALex Error L4012
ALex Warning L4012

reversed character class range

The character class contains a reversed range i.e. the largest character has been specified first.  The compiler automatically deals with this by swapping the characters around.  The example below defines a C identifier:

[a-zA-Z_][z-aA-Z0-9_]   { /* do something */ }

Note that in the second character class, the specification for the lower case letters is z-a, when it should in fact be a-z.

If you have redefined the values associated with characters using a table declaration, then reverse checks are performed before the translations are taken into effect.  For instance, the character 'a' is first converted to 97, its ASCII equivalent.  Similarly, 'z' is converted to 122.  The reverse check is performed, and then all the characters from 97 to 122 inclusive are mapped to their new values, whatever they may be.


ALex Error L4013
ALex Warning L4013

reversed range

The values in a repetition range are reversed i.e. the maximum number of times a regular expression can be matched is specified first.  The compiler automatically deals with this by swapping the values around.  The example below matches the letter A one, two or three times.

A{3,1}  { /* do something */ }

Note that the 1 should precede the 3.


YACC And Lex Library

The YACC and Lex library incorporates much, if not all, of the table driving code for the parsers and lexical analysers generated by AYACC and ALex.  The output files from these tools contain initial (static) stacks and buffers where necessary, the actions to be performed when rules are met and the driving tables themselves.  Everything else is in the library.

Classes

Functions

Variables

Types

Constants

Modifiers

Flags

Include Files


Variables

The parser and lexical analyser variables are used by (C) single model parsers and lexical analysers only.  The common variables are used by all parsers and lexical analysers, including C++ parsers and lexical analysers.

Parser Variables

Lexical Analyser Variables

Common Variables


Parser Variables

These variables only apply to the single parser model.  For multiple parsers, the variables are represented as members in the yymparse_t structure.

Variable	Description

yychar	The current lookahead token.

yyerrorcount	The number of syntax errors which have occurred.

yylookahead	Whether the current lookahead token is valid.

yyparsedebug	Whether debug information should be output.

yyparsedebugflush	Indicates whether debug output should be flushed when it is written.

yyparsedebugout	The stream to which the parser sends debug output.

yyparsedebugstack	Indicates whether parser stack contents should be output along with other debug information.

yyparseerr	The stream to which the parser sends error messages.

yystackgrow	Specifies whether the stack should grow if it ever overflows.

yywipeflg	Specifies whether the symbols used by the parser should be cleaned up if the parse is aborted.


yychar

#include <yacc.h>

int yychar

Remarks

The yychar variable holds the current lookahead token for the parser.  The parser does not always need a lookahead token, and sometimes this variable will be invalid.  Before using yychar check the state of the yylookahead variable first.  If yylookahead is nonzero then yychar holds a valid token.

See Also

yysetin, yyclearin, yyunclearin, yylookahead


yyerrorcount

#include <yacc.h>

int yyerrorcount

Remarks

The yyerrorcount variable holds the number of syntax errors which have occurred since the start of the current parse.  The variable is reset to zero at the start of each parse.  It is incremented for one of two reasons:

1	A syntax error is encountered by the parser (that is the parser can neither shift a token onto the stack, reduce a rule or accept the input) and the parser is not currently in error recovery mode.

2	A syntax error is generated when the yythrowerror or yyforceerror functions are called from an action.  Note that the error is generated only when control returns to the parser and no other higher priority function, i.e. yyexit has been called.  In this case whether the parser is in error recovery mode or not does not matter.

See Also

yysyntaxerror


yylookahead

#include <yacc.h>

unsigned char yylookahead

Remarks

The yylookahead variable specifies whether the current lookahead token, held in the yychar variable, is valid.  If yylookahead is nonzero then yychar holds a valid token.

See Also

yysetin, yyclearin, yyunclearin, yydestructclearin, yywipe, yycwipe, yychar


yyparsedebug

#include <yacc.h>

int yyparsedebug

Remarks

The yyparsedebug variable specifies whether debug information should be output by the parser while it is running.  If yyparsedebug is nonzero then debug information is generated.  By default yyparsedebug is set to zero, i.e. no debugging information is generated.  This variable is only defined for the debug version of the YACC and Lex library.

See Also

yyparsedebugout, yyparsedebugstack, yydebug, yydebugstack


yyparsedebugflush

#inclued <yacc.h>

int yyparsedebugflush

Remarks

The yyparsedebugflush variable specifies whether the debug output stream should be flushed whenever debug information is written to it.  If yyparsedebugflush is nonzero then the steam will be flushed.  By default yyparsedebugflush is set to zero i.e. the stream will not be flushed.  Flushing the debug stream is useful if the application which is being debugged crashes.  In such circumstances, the file written to, assuming it is a file, will contain all of the debug information.  The downside though is that flushing the stream slows down the application significantly.  This variable is only defined for the debug version of the YACC and Lex library.

See Also

yyparsedebugout, yyparsedebug, yydebug, yydebugflush


yyparsedebugout

#include <yacc.h>

FILE *yyparsedebugout

Remarks

The yyparsedebugout variable specifies the output stream to which the parser sends its debugging information.  This variable is typically initialised to the standard output stream stdout.  In some cases this may not happen and it is necessary to call the yyparseinit function.  You can assign a new value to yyparsedebugout in your source code.  This variable is only defined for the debug version of the YACC and Lex library.

In 32-bit Windows versions of the YACC and Lex library, if yyparsedebugout is set to NULL, then debugging information is sent to the debugger.  If the application containing the parser has an associated debugger, then the debug information is sent here.  Failing that, if the system debugger is active then debug information is sent here.  Otherwise any debug information is simply lost.

See Also

yyparseinit, yyparsedebug, yyparsedebugflush, yydebug, yydebugflush


yyparsedebugstack

include <yacc.h>

int yyparsedebugstack

Remarks

The yyparsedebugstack variable specifies whether the contents of the stack should be output along with other debug information.  If yyparsedebugstack is nonzero then the stack contents will be output.  By default yyparsedebugstack is set to zero i.e. stack contents will not be output.  This variable is only defined for the debug version of the YACC and Lex library.

When enabled, the contents of the stack are output every time after a state is pushed onto the stack.  In practice this means every rime after a shift or reduce action takes place.  Note that this extra debug information can significantly increase the size of the generated output.

See Also

yyparsedebug, yydebugstack, yydebug


yyparseerr

#include <yacc.h>

FILE *yyparseerr

Remarks

The yyparseerr variable specifies the output stream to which the yyerror function sends any error messages.  This variable is typically initialised to the standard error stream stderr.  In some cases this may not happen and it is necessary to call the yyparseinit function.  You can assign a new value to yyparseerr in your source code.

See Also

yyerror, yyparseinit


yystackgrow

#include <yacc.h>

unsigned char yystackgrow

Remarks

The yystackgrow variable specifies whether the parser's stack should grow if it ever overflows.  If yystackgrow is nonzero then the stack is allowed to grow.  The initial size of the stack is 100 elements unless you have defined YYSTACK_SIZE in your YACC source file.  The example below defines an initial stack with 1000 elements:

#define YYSTACK_SIZE 1000

By default yystackgrow is set 1 allowing the stack to grow.

If the stack overflows and it is allowed to grow, the parser will double the size of the stack.  If the stack has zero elements (possible if you define YYSTACK_SIZE as 0), then the parser will allocate a 100 element stack instead.  All stack changes are achieved through calls to the yysetstacksize function.

If the stack overflows and it is not allowed to grow, then the parser calls the yystackoverflow function before returning with YYEXIT_FAILURE.

See Also

yysetstacksize, yystackoverflow


yywipeflg

#include <yacc.h>

unsigned char yywipeflg

Remarks

The yywipeflg variable specifies whether the symbols in use by the parser should be cleaned up when, or if, the parser aborts.  If yywipeflg is nonzero then the symbols will be cleaned up.  By default this variable is set to 1 i.e. the symbols currently in use by the parser will be cleaned up.

The parser can abort for one of two reasons.  Either the stack overflows, or an untrapped syntax error is encountered.  In such cases, if symbol clean up is enabled, all symbols (states) will be popped off of the stack, and the current lookahead token cleared.  If any symbol which is discarded has a symbol destructor associated with it, then the destructor will be called for the symbol in question.

In practice, symbol cleanup is achieved through a call to the yywipe or yycwipe function, depending upon whether the parser is a default (fast) parser or a compact parser.

See Also

yywork, yycwork, yywipe, yycwipe


Lexical Analyser Variables

These variables only apply to the single lexical analyser model.  For multiple lexical analysers, the variables are represented as members in the yymlex_t structure.

Variable	Description

yyeol	Whether the previous (buffered) character was a newline character.

yyin	The stream from which the lexical analyser gets its input.

yyleng	The length of the matched string.

yylexdebug	Whether debug information should be output.

yylexdebugflush	Indicates whether debug output should be flushed when it is written.

yylexdebugout	The stream to which the lexical analyser sends debug output.

yylexerr	The stream to which the lexical analyser sends error messages.

yylineno	The current line number.

yyout	The stream to which the lexical analyser sends its output.

yystart	The current start state.

yytext	The matched string.

yytextgrow	Specifies whether the text buffer should grow if it ever overflows.

yyunputgrow	Specifies whether the unput buffer should grow if it ever overflows.


yyeol

#include <lex.h>

unsigned char yyeol

Remarks

The yyeol variable specifies whether the previous (buffered) character read by the lexical analyser was a newline \n character or not.  yyeol is nonzero if the previous character was a newline character, otherwise it is zero.  yyeol is initialised to 1 automatically, and is set to 1 by the yyreset function.

After a string has been matched by the lexical analyser, but before the associated action has been performed, yyeol is updated ready for matching the next string.  yyeol is also updated by the yyless function, should you use one in an action.  Note that if you backup the matched string completely, then yyeol is initialised with yyoldeol.  You can modify yyeol in your actions to suit your needs, providing of course, that you don't make a subsequent call to yyless.

See Also

yystart, yyreset, yyless


yyin

#include <lex.h>

FILE *yyin

Remarks

The yyin variable specifies the input stream from which the yyinput function gets its characters.  This variable is typically initialised to the standard input stream stdin.  In some cases this may not happen and it is necessary to call the yylexinit function.  You can assign a new stream to yyin in your Lex source file.  For example:

int main(void)
{

    yyin = fopen("mytext.txt", "r");

    if (yyin != NULL) {

        return yylex();

    }

    return 1;

}

Here an attempt is made to open the text file "mytext.txt" as the replacement input steam.  If this is successful, indicated by a non-NULL return value from fopen, then the input is processed in the usual way by a call to yylex.  Note that in this example it is not strictly necessary to call fclose in order to close the input stream.  When main exits, all open streams are closed automatically.

See Also

yyinput, yylexinit, yyout


yyleng

#include <lex.h>

int yyleng

Remarks

The yyleng variable specifies the length of the matched string.  Beware of modifying yyleng in your source code, especially if you make use of the yymore, yyless and yyreject functions.  All three functions make use of the variable.

See Also

yytext


yylexdebug

#include <lex.h>

int yylexdebug

Remarks

The yylexdebug variable specifies whether debug information should be output by the lexical analyser while it is running.  If yylexdebug is nonzero then debug information is generated.  By default yylexdebug is set to zero, i.e. no debugging information is generated.  This variable is only defined for the debug version of the YACC and Lex library.

See Also

yylexdebugout, yydebug


yylexdebugflush

#include <lex.h>

int yylexdebugflush

Remarks

The yylexdebugflush variable specifies whether the debug output stream should be flushed whenever debug information is written to it.  If yylexdebugflush is nonzero then the steam will be flushed.  By default yylexdebugflush is set to zero i.e. the stream will not be flushed.  Flushing the debug stream is useful if the application which is being debugged crashes.  In such circumstances, the file written to, assuming it is a file, will contain all of the debug information.  The downside though is that flushing the stream slows down the application significantly.  This variable is only defined for the debug version of the YACC and Lex library.

See Also

yylexdebugout, yylexdebug, yydebug, yydebugflush


yylexdebugout

#include <lex.h>

FILE *yylexdebugout

Remarks

The yylexdebugout variable specifies the output stream to which the lexical analyser sends its debugging information.  This variable is typically initialised to the standard output stream stdout.  In some cases this may not happen and it is necessary to call the yylexinit function.  You can assign a new value to yylexdebugout in your source code.  This variable is only defined for the debug version of the YACC and Lex library.

In 32-bit Windows versions of the YACC and Lex library, if yylexdebugout is set to NULL, then debugging information is sent to the debugger.  If the application containing the parser has an associated debugger, then the debug information is sent here.  Failing that, if the system debugger is active then debug information is sent here.  Otherwise any debug information is simply lost.

See Also

yylexinit, yylexdebug, yylexdebugflush, yydebug, yydebugflush


yylexerr

#include <lex.h>

FILE *yylexerr

Remarks

The yylexerr variable specifies the output stream to which any error messages are sent.  This variable is typically initialised to the standard error stream stderr.  In some cases this may not happen and it is necessary to call the yylexinit function.  You can assign a new value to yylexerr in your source code.

See Also

yytextoverflow, yyunputoverflow, yylexinit


yylineno

#include <lex.h>

int yylineno

Remarks

The yylineno variable holds the current line number.  Line numbers are assumed to start at 1.  The line number is incremented whenever a newline character \n is input, and it is decremented whenever a newline character \n is unput during backup.

This variable is reset to 1 by the yyreset function.

See Also

yyinput, yyunput, yyreset


yyout

#include <lex.h>

FILE *yyout

Remarks

The yyout variable specifies the output stream to which yyoutput sends its characters. This variable is typically initialised to stdout.  In some cases this may not happen and it is necessary to call the yylexinit function.  You can assign a new stream to yyout in your Lex source file.  For example:

int main(void)
{

    yyout = fopen("mytext.txt", "w");

    if (yyout != NULL) {

        return yylex();

    }

    return 1;

}

Here an attempt is made to open the text file "mytext.txt" as the replacement output steam.  If this is successful, indicated by a non-NULL return value from fopen, then the input is processed in the usual way by a call to yylex.  Note that in this example it is not strictly necessary to call fclose in order to close the input stream.  When main exits, all open streams are closed automatically.

See Also

yyoutput, yylexinit, yyin


yystart

#include <lex.h>

int yystart

Remarks

The yystart variable holds the current start state for the lexical analyser.  The start state can be changed by using the yybegin function.  However, sometimes it is useful to be able to use the actual start state value itself.  For instance, consider the example below which is attempting to recognise a C comment:

%start COMMENT

%%

<COMMENT>"*/"   { yybegin(0); }

<COMMENT>.|\n   { /* do nothing */ }

"/*"            { yybegin(COMMENT); }

...

%%

int yywrap(void)
{

    if (yystart == COMMENT) {

        error("unexpected end-of-file found in comment");
    }

    return 1;
}

In the redefined yywrap function above a check is made to see whether the lexical analyser is in the COMMENT start state.  If so then it means that the lexical analyser has found the start of a comment, but has not seen a terminating */.  In these cases, an appropriate error message is generated, before returning with the usual 1 indicating no more input.  It is worth noting that this use of yywrap can be duplicated by trailing end-of-file operators <<EOF>> in your Lex rules.  For more information see Right Context Expression.

See Also

yybegin


yytext

#include <lex.h>

char *yytext

Remarks

The yytext variable holds the matched regular expression string.  The string is zero terminated, although be aware that regular expressions can include zero as a character.  The length of the matched string is specified by the yyleng variable.

See Also

yymore, yyleng, yytextgrow


yytextgrow

#include <lex.h>

unsigned char yytextgrow

Remarks

The yytextgrow variable specifies whether the lexical analyser's text and state buffers should grow if they ever overflow.  If yytextgrow is nonzero then the buffers are allowed to grow.  The initial size of the text and state buffers is 100 elements unless you have defined YYTEXT_SIZE in your Lex source file.  The example below defines a text and state buffer with 1000 elements:

#define YYTEXT_SIZE 1000

By default yytextgrow is set to 1 allowing the text and state buffers to grow.

If the text and state buffers overflow and they are allowed to grow, the lexical analyser will double the size of each buffer.  If the text and state buffers have zero elements (possible if you define YYTEXT_SIZE as 0), then the lexical analyser will allocate 100 elements for each buffer instead.  All buffer changes are achieved through calls to the yysettextsize function.

If the text and state buffers overflow and they are not allowed to grow, then the lexical analyser calls the yytextoverflow function before calling the standard C library function exit with the argument EXIT_FAILURE.

In all of the above cases the text buffer always has one extra character.  This is to cope with the zero terminating character which is added onto the end of all matched strings.

See Also

yysettextsize, yytextoverflow, yyunputgrow


yyunputgrow

#include <lex.h>

unsigned char yyunputgrow

Remarks

The yyunputgrow variable specifies whether the lexical analyser's unput buffer should grow if it ever overflows.  If yyunputgrow is nonzero then the unput buffer is allowed to grow.  The initial size of the buffer is 100 elements unless you have defined YYTEXT_SIZE or YYUNPUT_SIZE in your Lex source file.  The example below defines an initial unput buffer of 1000 elements.

#define YYUNPUT_SIZE 1000

By default yyunputgrow is set to 1 allowing the unput buffer to grow.

If the unput buffer overflows and it is allowed to grow, the lexical analyser will double the size of the buffer.  If the buffer has zero elements (possible if you define YYTEXT_SIZE or YYUNPUT_SIZE as 0), then the lexical analyser will allocate a 100 element buffer instead.  All buffer changes are achieved through calls to the yysetunputsize function.

If the unput buffer overflows and it is not allowed to grow, then the lexical analyser calls the yyunputoverflow function before calling the standard C library function exit with the argument EXIT_FAILURE.

See Also

yysetunputsize, yyunputoverflow, yytextgrow


yydebug

#include <yacc.h> or

#include <lex.h> or

#include <myacc.h> or

#include <mlex.h> or

#include <cyacc.h> or

#include <clex.h>

int yydebug

Remarks

The yydebug variable specifies whether debug information should be output by all parsers and lexical analysers while they are running.  If yydebug is nonzero then debug information is generated.  By default yydebug is set to zero, i.e. no debugging information is generated.  This variable is only defined for the debug version of the YACC and Lex library.

See Also

yydebugflush, yydebugstack, yyparsedebug, yylexdebug, yymparse_t, yymlex_t, yyparser::yydebug, yylexer::yydebug


yydebugflush

#include <yacc.h> or

#include <lex.h> or

#include <myacc.h> or

#include <mlex.h> or

#include <cyacc.h> or

#include <clex.h>

int yydebugflush

Remarks

The yydebugflush variable specifies whether all debug output streams should be flushed whenever debug information is written to them.  If yydebugflush is nonzero then all steams will be flushed.  By default yydebugflush is set to zero i.e. the streams will not be flushed.  Flushing the debug streams is useful if the application which is being debugged crashes.  In such circumstances, the files written to, assuming they are files, will contain all of the debug information.  The downside though is that flushing the streams slows down the application significantly.  This variable is only defined for the debug version of the YACC and Lex library.

See Also

yydebug, yydebugstack, yyparsedebugflush, yylexdebugflush, yymparse_t, yymlex_t, yyparser::yydebugflush, yylexer::yydebugflush


yydebugstack

#include <yacc.h> or

#include <myacc.h> or

#include <cyacc.h> or

int yydebugstack

Remarks

The yydebugstack varible specifies, for all parsers, whether the contents of the stack should be output along with other debug information.  If yydebugstack is nonzero then the stack contents will be output.  By default yydebugstack is set to zero i.e. stack contents will not be output.  This variable is only defined for the debug version of the YACC and Lex library.

When enabled, the contents of the stack are output every time after a state is pushed onto the stack.  In practice this means every rime after a shift or reduce action takes place.  Note that this extra debug information can significantly increase the size of the generated output.

See Also

yydebug, yydebugflush, yyparsedebugstack, yymparse_t, yyparser::yydebugstack


Common Variables

These variables are used, in most cases, by all parsers and lexical analysers, including C++ parsers and lexical analysers.  The exception is yydebugstack which is used by parsers only.  Note that the variables are all associated with the output of debugging information.

Variable	Description

yydebug	Indicates whether debug output should be turned on.

yydebugflush	Indicates whether debug output should be flushed when it is written.

yydebugstack	Indicates whether parser stack contents should be output along with other debug information.


Types

These types are used by (C) multiple and multiple instance parsers and lexical analysers.  They effectively collect all of the global variables used by the single model parsers and lexical analysers and group them together.

Type	Description

yymparse_t	Multiple parser structure.

yymlex_t	Multiple lexical analyser structure.


yymparse_t

#include <myacc.h>

typedef struct yymparse {

    unsigned char yymlookahead;

    unsigned char yymstackgrow;

    unsigned char yymexitflg;

    unsigned char yymretireflg;

    unsigned char yymerrorflg;

    unsigned char yympopflg;

    unsigned char yymwipeflg;

    int yymtop;

    int yymchar;

    int yymskip;

    int yymerrorcount;

    int yymexitcode;

    int yymretirecode;

    int yymerrorpop;

    FILE *yymerr;

    yystack_t *yymstackptr;

    yystack_t *yymsstackptr;

    int yymstack_size;

    int yymsstack_size;

    size_t yymattribute_size;

    void *yymvalptr;

    void *yymlvalptr;

    void *yymattributestackptr;

    void *yymsattributestackptr;

    service functions

    void (*yymstackoverflow)(struct yymparse *yy);

    void (*yymerror)(struct yymparse *yy, const char *text);

    void (*yymsyntaxerror)(struct yymparse *yy);

    void (*yymdiscard)(struct yymparse *yy, int token);

    int (*yymgettoken)(struct yymparse *yy);

    void (*yymparseaction)(struct yymparse *yy, int action);

    driving tables

    const yystateaction_t *yymstateaction; (fast parser)

    const yytokenaction_t *yymtokenaction; (fast parser)

    int yymtokenaction_size; (fast parser)

    const yynontermgoto_t *yymnontermgoto; (fast parser)

    const yystategoto_t *yymstategoto; (fast parser)

    int yymstategoto_size; (fast parser)

    const yytokendest_t *yymtokendestptr; (fast parser)

    int yymtokendest_size; (fast parser)

    int yymtokendestbase; (fast parser)

    const yycstateaction_t *yymcstateaction; (compact parser)

    const yyctokenaction_t *yymctokenaction; (compact parser)

    const yycnontermgoto_t *yymcnontermgoto; (compact parser)

    const yycstategoto_t *yymcstategoto; (compact parser)

    const yyctokendest_t *yymctokendestptr; (compact parser)

    const yyreduction_t *yymreduction;

    const yydestructor_t *yymdestructorptr;

    user defined data

    void *yymdata;

    debug information

    int yymdebug; (debug version only)

    int yymdebugstack; (debug version only)

    int yymdebugflush; (debug version only)

    FILE *yymdebugout; (debug version only)

    const yysymbol_t *yymsymbol; (debug version only)

    const char *const *yymrule; (debug version only)

} yymparse_t;

Remarks

The yymparse_t structure is used by all multiple parsers, whether they are multiple instance or not.  It takes all of the variables, functions and tables used by the single parser and groups them together.  Note that some liberties have been taken with the description above.  For instance YYCONST has been changed to const and the YYCDECL, YYNEARFAR, YYFAR specifiers have been removed etc.  For an exact description of the structure see the multiple YACC header file "myacc.h" in the includes directory.

Member	Description

yymlookahead	Specifies whether the current lookahead token yymchar is valid.  If yymlookahead is nonzero then yymchar is valid, otherwise yymchar is invalid.

See also yymsetin, yymclearin, yymunclearin, yymdestructclearin.

yymstackgrow	Specifies whether the parser's stack should grow if it ever overflows.  If yymstackgrow is nonzero then the stack is allowed to grow.  The initial size of the stack is 100 elements unless you have defined YYSTACK_SIZE in your YACC source file.  The example below defines an initial stack with 1000 elements:

#define YYSTACK_SIZE 1000

By default yymstackgrow is set 1 allowing the stack to grow.

If the stack overflows and it is allowed to grow, the parser will double the size of the stack.  If the stack has zero elements (possible if you define YYSTACK_SIZE as 0), then the parser will allocate a 100 element stack instead.  All stack changes are achieved through calls to the yymsetstacksize function.

If the stack overflows and it is not allowed to grow, then the parser calls the function pointed to by the yymstackoverflow member before returning with YYEXIT_FAILURE.

See also yymsetstacksize.

yymexitflg	Specifies whether one of the yymexit, yymabort or yymaccept functions has been called from within an action.  The flag informs the parser that it should do some special processing.  The flag is closely linked with the yymexitcode member.

yymretireflg	Specifies whether the yymretire function has been called from within an action.  The flag informs the parser that it should do some special processing.  The flag is closely linked with the yymretirecode member.

yymerrorflg	Specifies whether the yymforceerror or the yymthrowerror functions have been called.  The flag informs the parser that it should do some special processing.

yymerrorpop	Specifies the number of states containing error token shift actions to be popped off of the stack before error recovery is attempted.  Used by the yymthrowerror function.

yymwipeflg	Specifies whether the symbols in use by the parser should be cleaned up when, or if, the parser aborts.  If yymwipeflg is nonzero then the symbols will be cleaned up.  By default this member is set to 1 i.e. the symbols currently in use by the parser will be cleaned up.

The parser can abort for one of two reasons.  Either the stack overflows, or an untrapped syntax error is encountered.  In such cases, if symbol clean up is enabled, all symbols (states) will be popped off of the stack, and the current lookahead token cleared.  If any symbol which is discarded has a symbol destructor associated with it, then the destructor will be called for the symbol in question.

In practice, symbol cleanup is achived through a call to the yymwipe or yymcwipe function, depending upon whether the parser is a default (fast) parser or a compact parser.

See also yymwork, yymcwork, yymwipe, yymcwipe.

yymtop	Specifies the current top most state on the stack.  Note that the stack is pre-increment/post-decrement.

yymchar	Specifies the current lookahead token for the parser.  The parser does not always need a lookahead token, and sometimes this variable will be invalid.  Before using yychar check the state of the yymlookahead member first.  If yymlookahead is nonzero then yymchar holds a valid token.

See also yymsetin, yymclearin, yymunclearin.

yymskip	Specifies that the parser is in error recovery if it is greater than zero.  It indicates the number of tokens that must be shifted onto the stack before error recovery is said to be complete.

yymerrorcount	Specifies the number of syntax errors which have occurred since the start of the current parse.  This member is reset to zero at the start of each parse.  It is incremented for one of two reasons:

1	A syntax error is encountered by the parser (that is the parser can neither shift a token onto the stack, reduce a rule or accept the input) and the parser is not currently in error recovery mode.

2	A syntax error is generated when the yymthrowerror or yymforceerror functions are called from an action.  Note that the error is generated only when control returns to the parser and no other higher priority function, i.e. yymexit has been called.  In this case whether the parser is in error recovery mode or not does not matter.

See also yymsyntaxerror.

yymexitcode	Specifies the exit code that was passed to the yymexit function, or the exit code that is implicitly associated with the yymabort and yymaccept functions.  This member is closely linked with the yymexitflg member.

yymretirecode	Specifies the exit code that was passed to the yymretire function.  This member is closely linked with the yymretireflg member.

yymerr	Specifies the output stream to which the yymerror function sends any error messages.  This member is typically initialised to the standard error stream stderr.  In some cases this may not happen and it is necessary to call the yymparseinit function.  You can assign a new value to yymerr in your source code.

See also yymerror, yymparseinit.

yymstackptr	Specifies the (state) stack.

yymsstackptr	Specifies the static (state) stack.

yymstack_size	Specifies the size of the stack.  This applies to both the state stack and the attribute stack.

yymsstack_size	Specifies the size of the static stack.

yymattribute_size	Specifies the size of the attribute.

yymvalptr	Specifies a pointer to yyval.  This holds the attribute $$ for use in actions.

yymlvalptr	Specifies a pointer to yylval.  This holds the attribute associated with any token which is shifted onto the stack.

yymattributestackptr	Specifies the attribute stack.

yymsattributestackptr	Specifies the static attribute stack.

yymstackoverflow	Specifies the function to call if the stack should ever overflow.  By default the function called is yymstackoverflow.  You can redefine yymstackoverflow in your source code.  If you do so however be aware that the function is called by all multiple parsers.  Alternatively you can assign a different function to the member.  In your YACC source file define YYMSTACKOVERFLOW as the function you wish to call.  For example:

void mystackoverflow(yymparse_t *yy)

{

    ...

}

#define YYMSTACKOVERFLOW mystackoverflow

yymerror	Specifies the general error handling function which is called by both yymstackoverflow and yymsyntaxerror. The member is provided for compatibility purposes.  By default the function called is yymerror. You can redefine yymerror in your source code. If you do so however be aware that the function is called by all multiple parsers.  Alternatively you can assign a different function to the member.  In your YACC source file define YYMERROR as the function you wish to call.  For example:

void myerror(yymparse_t *yy, const char *text)

{

    ...

}

#define YYMERROR myerror

yymsyntaxerror	Specifies the function to call if a syntax error should occur.  By default the function called is yymsyntaxerror.  You can redefine yymsyntaxerror in your source code.  If you do so however be aware that the function is called by all multiple parsers.  Alternatively you can assign a different function to the member.  In your YACC source file define YYMSYNTAXERROR as the function you wish to call.  For example:

void mysyntaxerror(yymparse_t *yy)

{

    ...

}

#define YYMSYNTAXERROR mysyntaxerror

yymdiscard	Specifies the function to call when the parser discards a token during error recovery.  By default the function called is yymdiscard.  You can redefine yymdiscard in your source code.  If you do so however be aware that the function is called by all multiple parsers.  Alternatively you can assign a different function to the member.  In your YACC source file define YYMDISCARD as the function you wish to call.  For example:

void mydiscard(yymparse_t *yy, int token)

{

    ...

}

#define YYMDISCARD mydiscard

yymgettoken	Specifies the function to call when the parser needs the next token.  By default the function called is yymgettoken.  You can redefine yymgettoken in your source code.  If you do so however be aware that the function is called by all multiple parsers.  Alternatively you can assign a different function to the member.  In your YACC source file define YYMGETTOKEN as the function you wish to call.  For example:

int mygettoken(yymparse_t *yy)

{

    ...

}

#define YYMGETTOKEN mygetttoken

yymparseaction	Specifies the function to call when an action is to be performed.

yymstateaction	Specifies the state action array.  Used by the fast parser only.

yymtokenaction	Specifies the token action array.  Used by the fast parser only.

yymtokenaction_size	Specifies the size of the token action array.  Used by the fast parser only.

yymnontermgoto	Specifies the nonterminal goto array.  Used by the fast parser only.

yymstategoto	Specifies the state goto array.  Used by the fast parser only.

yymstategoto_size	Specifies the size of the state goto array.  Used by the fast parser only.

yymtokendestptr	Specifies the token destructor array (if any).  Used by the fast parser only.

yymtokendest_size	Specifies the size of the token destructor array.  Used by the fast parser only.

yymtokendestbase	Specifies the base for the token destructor array.  This number is added to the token number to arrive at an index in the token destructor array.  Used by the fast parser only.

yymcstateaction	Specifies the state action array.  Used by the compact parser only.

yymctokenaction	Specifies the token action array.  Used by the compact parser only.

yymcnontermgoto	Specifies the nonterminal goto array.  Used by the compact parser only.

yymcstategoto	Specifies the state goto array.  Used by the compact parser only.

yymctokendestptr	Specifies the token destructor array (if any).  Used by the compact parser only.

yymreduction	Specifies the reduction array.  Used by both the fast and compact parsers.

yymdestructorptr	Specifies the symbol destructor array (if any).  Used by both the fast and compact parsers.

yymdata	Specifies user defined data.  You can assign a value to the member by defining YYMPARSEDATA in your YACC source file.  For example:

double mydata;

#define YYMPARSEDATA &mydata

Note

The default implementation of yymgettoken makes use of yymdata.  It assumes that yymdata points to a multiple lexical analyser.  If you are going to use this, then you must have something like:

#include "mylexer.h"

#define YYMPARSEDATA &mylexer

in your YACC source code.  Note that mylexer is the name given to the lexical analyser, typically through using the %name keyword.

yymdebug	Specifies whether debug information should be output by the parser while it is running.  If yymdebug is nonzero then debug information is generated.  By default yymdebug is set to zero, i.e. no debugging information is generated.  This member is only defined for the debug version of the YACC and Lex library.

See also yydebug, yydebugstack.

yymdebugstack	Specifies whether the contents of the stack should be output along with other debug information.  If yymparsestack is nonzero then the stack contents will be output.  By default yymdebugstack is set to zero i.e. stack contents will not be output.  This member is only defined for the debug version of the YACC and Lex library.

When enabled, the contents of the stack are output every time after a state is pushed onto the stack.  In practice this means every rime after a shift or reduce action takes place.  Note that this extra debug information can significantly increase the size of the generated output.

See also yydebugstack, yydebug.

yymdebugflush	Specifies whether the debug output stream should be flushed whenever debug information is written to it.  If yymdebugflush is nonzero then the steam will be flushed.  By default yymdebugflush is set to zero i.e. the stream will not be flushed.  Flushing the debug stream is useful if the application which is being debugged crashes.  In such circumstances, the file written to, assuming it is a file, will contain all of the debug information.  The downside though is that flushing the stream slows down the application significantly.  This variable is only defined for the debug version of the YACC and Lex library.

See also yydebug, yydebugflush.

yymdebugout	Specifies the output stream to which the parser sends its debugging information.  This member is typically initialised to the standard output stream stdout.  In some cases this may not happen and it is necessary to call the yymparseinit function.  You can assign a new value to yymdebugout in your source code.  This member is only defined for the debug version of the YACC and Lex library.

In 32-bit Windows versions of the YACC and Lex library, if yymdebugout is set to NULL, then debugging information is sent to the debugger.  If the application containing the parser has an associated debugger, then the debug information is sent here.  Failing that, if the system debugger is active then debug information is sent here.  Otherwise any debug information is simply lost.

See also yymparseinit, yydebug, yydebugflush.

yymsymbol	Specifies an array of symbol information.  Used for debugging purposes only.  This member is only present in the debug version.

yymrule	Specifies an array of rule information.  Used for debugging purposes only.  This member is only present in the debug version.

See Also

yymlex_t


yymlex_t

#include <mlex.h>

typedef struct yymlex {

    unsigned char yymmoreflg;

    unsigned char yymrejectflg;

    unsigned char yymreturnflg;

    unsigned char yymtextgrow;

    unsigned char yymunputgrow;

    unsigned char yymeol;

    unsigned char yymoldeol;

    int yymstart;

    FILE *yymin;

    FILE *yymout;

    FILE *yymerr;

    int yymleng;

    int yymlineno;

    int yymunputindex;

    char *yymtext;

    char *yymstext;

    int yymtext_size;

    int yymstext_size;

    int *yymstatebuf;

    int *yymsstatebuf;

    int *yymunputbufptr;

    int *yymsunputbufptr;

    int yymunput_size;

    int yymsunput_size;

    service functions

    int (*yyminput)(struct yymlex *yy);

    void (*yymoutput)(struct yymlex *yy, int ch);

    void (*yymunput)(struct yymlex *yy, int ch);

    int (*yymwrap)(struct yymlex *yy);

    int (*yymgetchar)(struct yymlex *yy);

    void (*yymtextoverflow)(struct yymlex *yy);

    void (*yymunputoverflow)(struct yymlex *yy);

    int (*yymlexaction)(struct yymlex *yy, int action);

    driving tables

    const yytransition_t *yymtransition; (fast lexical analyser)

    int yymtransitionmax; (fast lexical analyser)

    const yyctransition_t *yymctransition; (compact lexical analyser)

    const yymatch_t *yymmatch;

    const yystate_t *yymstate;

    const yybackup_t *yymbackup;

    user defined data

    void *yymdata;

    debug information

    int yymdebug; (debug version only)

    int yymdebugflush; (debug version only)

    FILE *yymdebugout; (debug version only)

} yymlex_t;

Remarks

This structure is used by all multiple lexical analysers, whether they are multiple instance or not.  It takes all of the variables, functions and tables used by the single parser and groups them together.  Note that some liberties have been taken with the description above.  For instance YYCONST has been changed to const and the YYCDECL, YYNEARFAR, YYFAR specifiers have been removed etc.  For an exact description of the structure see the multiple Lex header file "mlex.h" in the includes directory.

Member	Description

yymmoreflg	Specifies whether the next matched string is to be concatenated on the end of the last matched string.  The flag is nonzero if the strings are to be concatenated.  This flag is set by the yymmore function.  It is cleared automatically the next time a string is matched.

yymrejectflg	Specifies whether the input should be re-scanned.  The next matched string will always be the same length as or shorter than the last matched string.  The flag is nonzero if the input is to be re-scanned.  This flag is set by the yymreject function.  It is cleared automatically the next time a string is matched.

yymreturnflg	Specifies whether an action used a return statement.  The flag is nonzero if it did so.  This forces the lexical analyser to itself return immediately with the actions return value.

yymtextgrow	Specifies whether the lexical analyser's text and state buffers should grow if they ever overflow.  If yymtextgrow is nonzero then the buffers are allowed to grow.  The initial size of the text and state buffers is 100 elements unless you have defined YYTEXT_SIZE in your Lex source file.  The example below defines a text and state buffer with 1000 elements:

#define YYTEXT_SIZE 1000

By default yymtextgrow is set to 1 allowing the text and state buffers to grow.

If the text and state buffers overflow and they are allowed to grow, the lexical analyser will double the size of each buffer.  If the text and state buffers have zero elements (possible if you define YYTEXT_SIZE as 0), then the lexical analyser will allocate 100 elements for each buffer instead.  All buffer changes are achieved through calls to the yymsettextsize function.

If the text and state buffers overflow and they are not allowed to grow, then the lexical analyser calls the function pointed to by the yymtextoverflow member before calling the standard C library function exit with the argument EXIT_FAILURE.

In all of the above cases the text buffer always has one extra character.  This is to cope with the zero terminating character which is added onto the end of all matched strings.

See also yymsettextsize.

yymunputgrow	Specifies whether the lexical analyser's unput buffer should grow if it ever overflows.  If yymunputgrow is nonzero then the unput buffer is allowed to grow.  The initial size of the buffer is 100 elements unless you have defined YYTEXT_SIZE or YYUNPUT_SIZE in your Lex source file.  The example below defines an initial unput buffer of 1000 elements.

#define YYUNPUT_SIZE 1000

By default yymunputgrow is set to 1 allowing the unput buffer to grow.

If the unput buffer overflows and it is allowed to grow, the lexical analyser will double the size of the buffer.  If the buffer has zero elements (possible if you define YYTEXT_SIZE or YYUNPUT_SIZE as 0), then the lexical analyser will allocate a 100 element buffer instead.  All buffer changes are achieved through calls to the yymsetunputsize function.

If the unput buffer overflows and it is not allowed to grow, then the lexical analyser calls the function pointed to by the yymunputoverflow member before calling the standard C library function exit with the argument EXIT_FAILURE.

See also yymsetunputsize.

yymeol	Specifies whether the previous (buffered) character read by the lexical analyser was a newline \n character or not.  yymeol is nonzero if the previous character was a newline character, otherwise it is zero.  yymeol is initialised to 1 automatically, and is set to 1 by the yymreset function.

After a string has been matched by the lexical analyser, but before the associated action has been performed, yymeol is updated ready for matching the next string.  yymeol is also updated by the yymless function, should you use one in an action.  Note that if you backup the matched string completely, then yymeol is initialised with yymoldeol.  You can modify yymeol in your actions to suit your needs, providing of course, that you don't make a subsequent call to yymless.

See also yymreset, yymless.

yymoldeol	Specifies a previous yymeol value.  When yymleng is set to zero by the lexical analyser, the value stored in yymeol is copied into yymoldeol.  The value is stored in case any subsequent matched string is backed up completely, in which case yymoldeol will need to be retrieved.

yymstart	Specifies the current start state for the lexical analyser.  The start state can be changed by using the yymbegin function.  However, sometimes it is useful to be able to use the actual start state value itself.  For instance, consider the example below which is attempting to recognise a C comment:

%start COMMENT

%%

<COMMENT>"*/"   { yymbegin(0); }

<COMMENT>.|\n   { /* do nothing */ }

"/*"            { yymbegin(COMMENT); }

...

%%

int yymwrap(yymparse_t *yy)
{

    if (yy->yymstart == COMMENT) {

        error("unexpected end-of-file found in comment");
    }

    return 1;
}

In the redefined yymwrap function above a check is made to see whether the lexical analyser is in the COMMENT start state.  If so then it means that the lexical analyser has found the start of a comment, but has not seen a terminating */.  In these cases, an appropriate error message is generated, before returning with the usual 1 indicating no more input.  It is worth noting that this use of yymwrap can be duplicated by trailing end-of-file operators <<EOF>> in your Lex rules.  For more information see Right Context Expression.

See also yymbegin.

yymin	Specifies the input stream from which the yyminput function gets its characters.  This member is typically initialised to the standard input stream stdin.  In some cases this may not happen and it is necessary to call the yymlexinit function.  You can assign a new stream to yymin in your Lex source file.  For example:

int main(void)
{

    mylexer.yymin = fopen("mytext.txt", "r");

    if (mylexer.yyin != NULL) {

        return yymlex(&mylexer);

    }

    return 1;

}

Here an attempt is made to open the text file "mytext.txt" as the replacement input steam.  If this is successful, indicated by a non-NULL return value from fopen, then the input is processed in the usual way by a call to yymlex.  Note that in this example it is not strictly necessary to call fclose in order to close the input stream.  When main exits, all open streams are closed automatically.

See also yyminput.

yymout	Specifies the output stream to which the yymoutput function sends its characters.  This member is typically initialised to the standard output stream stdout.  In some cases this may not happen and it is necessary to call the yymlexinit function.  You can assign a new stream to yymout in your Lex source file.  For example:

int main(void)
{

    mylexer.yymout = fopen("mytext.txt", "w");

    if (mylexer.yymout != NULL) {

        return yymlex(&mylexer);

    }

    return 1;

}

Here an attempt is made to open the text file "mytext.txt" as the replacement output steam.  If this is successful, indicated by a non-NULL return value from fopen, then the input is processed in the usual way by a call to yymlex.  Note that in this example it is not strictly necessary to call fclose in order to close the input stream.  When main exits, all open streams are closed automatically.

See also yymoutput.

yymerr	Specifies the output stream to which any error messages are sent.  This member is typically initialised to the standard error stream stderr.  In some cases this may not happen and it is necessary to call the yymlexinit function.  You can assign a new stream to yymerr in your source code.

See also yymlexinit.

yymleng	Specifies the length of the matched string.  Beware of modifying yymleng in your source code, especially if you make use of the yymmore, yymless and yymreject functions.  All three functions make use of the member.

yymlineno	Specifies the current line number.  Line numbers are assumed to start at 1.  The line number is incremented whenever a newline character \n is input, and it is decremented whenever a newline character \n is unput during backup.

This variable is reset to 1 by the yymreset function.

See also yyminput, yymunput, yymreset.

yymunputindex	Specifies how many characters are in the unput buffer.

yymtext	Specifies the matched regular expression string.  The string is zero terminated, although be aware that regular expressions can include zero as a character.  The length of the matched string is specified by the yymleng member.

See also yymmore.

yymstext	Specifies the (static) text buffer.

yymtext_size	Specifies the current size of the text and state buffers.

yymstext_size	Specifies the size of the (static) text and state buffers.

yymstatebuf	Specifies the state buffer.

yymsstatebuf	Specifies the (static) state buffer.

yymunputbufptr	Specifies the unput buffer.

yymsunputbufptr	Specifies the (static) unput buffer.

yymunput_size	Specifies the current size of the unput buffer.

yymsunput_size	Specifies the size of the (static) unput buffer.

yyminput	Specifies the function to call when the lexical analyser needs the next (buffered) character.  By default the function called is yyminput.  You can redefine yyminput in your source code.  If you do so however be aware that the function is called by all multiple lexical analysers.  Alternatively you can assign a different function to the member.  In your Lex source file define YYMINPUT as the function you wish to call.  For example:

int myinput(yymlex_t *yy)

{

    ...

}

#define YYMINPUT mygetttoken

yymoutput	Specifies the function to call when the lexical analyser outputs an unmatched character.  By default the function called is yymoutput.  You can redefine yymoutput in your source code.  If you do so however be aware that the function is called by all multiple lexical analysers.  Alternatively you can assign a different function to the member.  In your Lex source file define YYMOUTPUT as the function you wish to call.  For example:

void myoutput(yymlex_t *yy, int ch)

{

    ...

}

#define YYMOUTPUT myoutput

yymunput	Specifies the function to call when the lexical analyser backups a character in the unput buffer.  By default the function called is yymunput.  You can redefine yymunput in your source code.  If you do so however be aware that the function is called by all multiple lexical analysers.  Alternatively you can assign a different function to the member.  In your Lex source file define YYMUNPUT as the function you wish to call.  For example:

void myunput(yymlex_t *yy, int ch)

{

    ...

}

#define YYMUNPUT myunput

yymwrap	Specifies the function to call when the lexical analyser checks to see whether it has any more input to process.  By default the function called is yymwrap.  You can redefine yymwrap in your source code.  If you do so however be aware that the function is called by all multiple lexical analysers.  Alternatively you can assign a different function to the member.  In your Lex source file define YYMWRAP as the function you wish to call.  For example:

int mywrap(yymlex_t *yy)

{

    ...

}

#define YYMWRAP mywrap

yymgetchar	Specifies the function to call when the lexical analyser needs the next (unbuffered) character.  By default the function called is yymgetchar.  You can redefine yymgetchar in your source code.  If you do so however be aware that the function is called by all multiple lexical analysers.  Alternatively you can assign a different function to the member.  In your Lex source file define YYMGETCHAR as the function you wish to call.  For example:

int mygetchar(yymlex_t *yy)

{

    ...

}

#define YYMGETCHAR mygetchar

yymtextoverflow	Specifies the function to call when the text (and state) buffer overflows.  By default the function called is yymtextoverflow.  You can redefine yymtextoverflow in your source code.  If you do so however be aware that the function is called by all multiple lexical analysers.  Alternatively you can assign a different function to the member.  In your Lex source file define YYMTEXTOVERFLOW as the function you wish to call.  For example:

void mytextoverflow(yymlex_t *yy)

{

    ...

}

#define YYMTEXTOVERFLOW mytextoverflow

yymunputoverflow	Specifies the function to call when the unput buffer overflows.  By default the function called is yymunputoverflow.  You can redefine yymunputoverflow in your source code.  If you do so however be aware that the function is called by all multiple lexical analysers.  Alternatively you can assign a different function to the member.  In your Lex source file define YYMUNPUTOVERFLOW as the function you wish to call.  For example:

void myunputoverflow(yymlex_t *yy)

{

    ...

}

#define YYMUNPUTOVERFLOW myunputoverflow

yymlexaction	Specifies the function to call when an action is to be performed.

yymtransition	Specifies the transition array.  Used by the fast lexical analyser only.

yymtransitionmax	Specifies the size of the transition array.  Used by the fast lexical analyser only.

yymctransition	Specifies the compact transition array.  Used by the compact lexical analyser only.

yymmatch	Specifies the match array.  Used by both the fast and compact lexical analysers.

yymstate	Specifies the state array.  Used by both the fast and compact lexical analysers.

yymbackup	Specifies the backup array.  Used by both the fast and compact lexical analysers.

yymdata	Specifies user defined data.  You can assign a value to the member by defining YYMLEXDATA in your Lex source file.  For example:

double mydata;

#define YYMLEXDATA &mydata

yymdebug	Specifies whether debug information should be output by the lexical analyser while it is running.  If yymdebug is nonzero then debug information is generated.  By default yymdebug is set to zero, i.e. no debugging information is generated.  This member is only defined for the debug version of the YACC and Lex library.

See also yydebug.

yymdebugflush	Specifies whether the debug output stream should be flushed whenever debug information is written to it.  If yymdebugflush is nonzero then the steam will be flushed.  By default yymdebugflush is set to zero i.e. the stream will not be flushed.  Flushing the debug stream is useful if the application which is being debugged crashes.  In such circumstances, the file written to, assuming it is a file, will contain all of the debug information.  The downside though is that flushing the stream slows down the application significantly.  This member is only defined for the debug version of the YACC and Lex library.

See also yydebug, yydebugflush.

yymdebugout	Specifies the output stream to which the lexical analyser sends its debugging information.  This member is typically initialised to the standard output stream stdout.  In some cases this may not happen and it is necessary to call the yymlexinit function.  You can assign a new value to yymdebugout in your source code.  This member is only defined for the debug version of the YACC and Lex library.

See also yymlexinit, yydebug.

See Also

yymparse_t


Constants

These are the common constants used by the YACC and Lex library.  They specify common return values for the parsers and lexical analysers generated by YACC and Lex respectively.

Constant	Description

YYEOF	Returned by the yylex, yyclex, yymlex and yymclex functions and the yylexer::yylex member function to indicate that the lexical analyser has got no more input to process.  The constant has a nominal value of 0.

YYEXIT_FAILURE	Returned by the yyparse, yycparse, yymparse, yymcparse, yywork, yycwork, yymwork and yymcwork functions and the yyparser::yyparse and yyparser::yywork member functions to indicate that the parse has failed.  It can also be passed to the yyexit and yymexit functions and the yyparser::yyexit member function to prematurely stop the parse and indicate that it has failed.  The constant has a nominal value of 1.

YYEXIT_SUCCESS	Returned by the yyparse, yycparse, yymparse, yymcparse, yywork, yycwork, yymwork and yymcwork functions and the yyparser::yyparse and yyparser::yywork member functions to indicate that the parse has succeeded.  It can also be passed to the yyexit and yymexit functions and the yyparser::yyexit member function to prematurely stop the parse and indicate that it has succeeded.  The constant has a nominal value of 0.


YYEXIT_SUCCESS = 0


YYEXIT_FAILURE = 1


EOF = -1


Modifiers

These are the common modifiers used by the YACC and Lex library.  They are used by all parsers and lexical analysers, with a few exceptions.  For instance YYCONST is not used by C++ parsers and lexical analysers since the keyword const, which is used to indicate that an object is non-modifiable, is always a valid C++ keyword.

Modifier	Description

YYBASED_CODE	Modifier which is applied to all parser and lexical analyser table arrays.  It is typically used to place the generated table arrays in the code segment.  This is of particular advantage to 16-bit medium memory model applications, and 16-bit Windows application where there must be only one data segment in order for multiple instances of a program to be able to run.  If you are using the 16-bit Visual C++ compiler then YYBASED_CODE is defined as __based(__segname("_CODE")) when you are compiling medium, large or huge memory model programs.  YYBASED_CODE is not defined if you are using the Borland C++ compiler.  This compiler uses a #pragma statement to place data into the code segment, which is not currently supported.  The YYBASED_CODE modifier is closely linked to the YYNEARFAR modifier.  You may want to provide your own definition for YYBASED_CODE in the YYTUDEFS.H file.

YYCDECL	Modifier which is applied to all global functions in the YACC and Lex library.  The modifier typically specifies that the global functions use the C naming and calling conventions.  If you are using the Visual C++ or Borland C++ compilers then YYCDECL is defined as being __cdecl.  For a different compiler, YYCDECL is defined as nothing.  You may want to provide your own definition for YYCDECL in the YYTUDEFS.H file.

YYCONST	Modifier which is applied to any variable or parameter which can be consider constant or non-modifiable.  If you are using the Visual C++ or Borland C++ compilers, or __STDC__ or __cplusplus are defined then YYCDECL is defined as being const.  Otherwise YYCONST is defined as nothing.  You may want to provide your own definition for YYCONST in the YYTUDEFS.H file.

YYDCDECL	Modifier which is applied to all global variables in the YACC and Lex library.  The modifier typically specifies that the global variables use the C naming convention.  If you are using the 16-bit Visual C++ compiler or the Borland C++ compiler then YYDCDECL is defined as being __cdecl.  If you are using the 32-bit Visual C++ compiler then YYDCDECL is defined as nothing.  For a different compiler, YYDCDECL is defined as nothing.  You may want to provide your own definition for YYDCDECL in the YYTUDEFS.H file.

YYFAR	Modifier which is applied to most pointers, excluding parser and lexical analyser table pointers, and class definitions in the YACC and Lex library.  This modifier is primarily intended to support the Borland C++ compiler when building static libraries which are linkable to a DLL.  Note that the Borland YACC and Lex library project supplied with Parser Generator is configured to build static libraries linkable to an application.  The two types of library are not the same.  It is however quite easy to change a library type in the project.  Simply right click on the library you wish to build and select the Target Expert command.

For Win3.1 DLLs, the run-time library functions for the most part use far pointers for argument and return values, rather than a pointer size corresponding to the memory model.  This is fine for large and compact memory model DLLs, where pointers are equivalent to far pointers.  However, for small and medium memory models, there is an inevitable segment lost when a far pointer is assigned to a near pointer.

Since the YACC and Lex library makes frequent use of the runtime library, especially for memory allocation and file IO, this will happen.  The solution is to mark the relevant pointers, and classes, in the YACC and Lex library as far where necessary.  In short, if you are using the Borland C++ compiler, and you are building a Win3.1 DLL, YYFAR is defined as being __far.  This trouble does not occur with Microsoft Visual C++.  In any case building Win3.1 small and medium memory model static libraries linkable to a DLL is slightly obscure.

You may want to provide your own definition for YYFAR in the YYTUDEFS.H file.

YYNEAR	Modifier which is applied to all global variables in the YACC and Lex library.  Note that this does not apply to the parser and lexical analyser table arrays.  The modifier typically specifies, for 16-bit applications, that the global variables reside in the near data segment.  This is of particular importance to 16-bit Windows applications.  If all of the global variables can fit in this segment, then multiple instances of the application can be run.  If you are using the 16-bit Visual C++ compiler, or are building a 16-bit application with the Borland C++ compiler then YYNEAR is defined as being __near.  Otherwise YYNEAR is defined as nothing.  For another compiler you may want to provide your own definition for YYNEAR in the YYTUDEFS.H file.

YYNEARFAR	Modifier which is applied to all parser and lexical analyser table arrays and pointers to those arrays.  The modifier is used, along with the YYBASED_CODE modifier to place the tables in the code segment.  It is intended for supporting 16-bit applications only.  If you are using the 16-bit Visual C++ compiler, YYNEARFAR is defined as being __near if you are building a tiny or small memory model application.  Otherwise if you are building a medium, compact, large or huge memory model application YYNEARFAR is defined as __far.  The important case is the medium memory application.  The default data pointer in such circumstances is a near pointer, which cannot possibly address constant data held in the code segment.  The only place where a near pointer can access the code segment is if you have built a tiny application, where the data and code segments share the same segment.  YYNEARFAR is not defined if you are using the Borland C++ compiler.  This is because table arrays are not currently placed in the code segment when using the Borland C++ compiler.  You may want to provide your own definition for YYNEARFAR in the YYTUDEFS.H file.

See Also

Include Files


Flags

These are the common flags used by the YACC and Lex library.

Flag	Description

YYBUDEFS	If the YYBUDEFS flag is defined then the include file YYBUDEFS.H is included at the bottom of the main YACC and Lex library include files.  That is, the files LEX.H, YACC.H, CLEX.H, CYACC.H, MLEX.H and MYACC.H.  By default, YYBUDEFS is not defined.

YYDEBUG	The YYDEBUG flag has two areas of use, although both of them are  related to generating diagnostic output in the YACC and Lex generated parsers and lexical analysers.  If the YYDEBUG flag is specified when building the YACC and Lex library, then diagnostic functions and various debugging variables are included within it.  Note that the Microsoft Visual C++ libraries supplied with Parser Generator come in two versions, either release or debug.  Similarly for the Borland C++ libraries, although in this case you need to compile the libraries yourself.

Secondly, if you are creating a debug version of an application, in which you are linking a debug version of the YACC and Lex library, then you will need to define YYDEBUG.  Typically, you will do this by specifying it as a project option in your IDE, or as a command line option to your compiler.

By default, YYDEBUG is not defined.

YYNINITSTREAM	If the YYNINITSTREAM flag is defined then the parser streams yyparseerr and yyparsedebugout and the lexical analyser streams yyin, yyout, yylexerr and yylexdebugout are initialised to NULL.  This also includes the corresponding members of the yymparse_t and yymlex_t structures.  If the YYNINITSTREAM flag is not defined, then the streams are initialised to their normal values i.e. the yyparseerr stream is initialised to stderr etc.  It is necessary to define YYNINITSTREAM when compiling the YACC and Lex library in certain configurations.  Principally, when the C run-time library which holds the stream handling code resides in a DLL.  In these circumstances, it is not possible to initialise a stream global variable with one of the standard streams - stdin, stdout or stderr.  The stream global variables instead have to be initialised though a call to a function.  This is where the yylexinit and yyparseinit functions are used.

By default YYNINITSTREAM is defined if you are using the Visual C++ or Borland C++ compilers and the run-time library resides in a DLL.  Note that for Borland C++ the compiler must also be version 5.30 or later.  As an aside, in Visual C++ the flag _DLL specifies that the run-time library resides in a DLL, whereas in Borland C++ it is the flag _RTLDLL.

YYPROTOTYPE	If the YYPROTOTYPE flag is defined then function prototypes are specified for all functions.  By this, it means all arguments that a function takes are specified in the header file.  By default, YYPROTOTYPE is defined if you are using the Visual C++ or Borland C++ compilers, or __STDC__ or __cplusplus are defined.

YYTUDEFS	If the YYTUDEFS flag is defined then the include file YYTUDEFS is included at the top of the main YACC and Lex library include files.  That is, the files LEX.H, YACC.H, CLEX.H, CYACC.H, MLEX.H and MYACC.H.  By default, YYTUDEFS is not defined.

YYUDEFS	The YYUDEFS flag is equivalent to specifying both the YYTUDEFS flag and the YYBUDEFS flag.  By default, YYUDEFS is not defined.

See Also

Include Files


Include Files

These are the include files that are used by the YACC and Lex library.  They are located in the include sub-directory off of the main Parser Generator directory.

Include File	Description

LEX.H	Specifies the variable and function declarations and the type and constant definitions for the (C) single lexical analyser model.

YACC.H	Specifies the variable and function declarations and the type and constant definitions for the (C) single parser model.

CLEX.H	Specifies the definitions for the yylexer, yyflexer and yyclexer C++ classes.  Also includes any relevant variable declarations and type and constant definitions.

CYACC.H	Specifies the definitions for the yyparser, yyfparser and yycparser C++ classes.  Also includes any relevant variable declarations and type and constant definitions.

MLEX.H	Specifies the variable and function declarations and the type and constant definitions for the (C) multiple and multiple instance lexical analyser models.

MYACC.H	Specifies the variable and function declarations and the type and constant definitions for the (C) multiple and multiple instance parser models.

MLCONV.H	Specifies conversion macros to convert a single lexical analyser model to a multiple lexical analyser model.  This applies to all lexical analyser variables and functions.  For instance the single model action function yymore maps to the multiple model function yymmore.  The multiple lexical analyser, which is required as a parameter to yymmore is specified by the YYLEXNAME define.  This is usually defined in your Lex source file using the %name keyword.  Alternatively, YYLEXNAME can be defined directly using a #define statement embedded within a Code Block.

You can use this file to help you to quickly convert from one model to another.  Typically you will include the file at the top of your Lex source file.

MILCONV.H	Specifies conversion macros to convert a single lexical analyser model to a multiple instance lexical analyser model.  This applies to all lexical analyser variables and functions.  For instance the single model action function yymore maps to the multiple model function yymmore.  The multiple instance lexical analyser, which is required as a parameter to yymmore must be held in a variable named yy.  This is of type yymlex_t *. Ordinarily you will not need to worry about this since the yylexaction function, which wraps all of your Lex actions takes a parameter of just this type and name.

You can use this file to help you to quickly convert from one model to another.  Typically you will include the file at the top of your Lex source file.

MYCONV.H	Specifies conversion macros to convert a single parser model to a multiple parser model.  This applies to all parser variables and functions.  For instance the single model action function yyforceerror maps to the multiple model function yymforceerror.  The multiple parser, which is required as a parameter is specified by the YYPARSENAME define.  This is usually defined in your YACC source file using the %name keyword.  Alternatively, YYPARSENAME can be defined directly using a #define statement embedded within a Code Block.

You can use this file to help you to quickly convert from one model to another.  Typically you will include the file at the top of your YACC source file.

MIYCONV.H	Specifies conversion macros to convert a single parser model to a multiple instance parser model.  This applies to all parser variables and functions.  For instance the single model action function yyforceerror maps to the multiple model function yymforceerror.  The multiple instance parser, which is required as a parameter to yymforceerror must be held in a variable named yy.  This is of type yymparse_t *. Ordinarily you will not need to worry about this since the yyparseaction function, which wraps all of your YACC actions takes a parameter of just this type and name.

You can use this file to help you to quickly convert from one model to another.  Typically you will include the file at the top of your YACC source file.

YYTDEFS.H	Specifies common definitions to be included at the top of the LEX.H, YACC.H, MLEX.H and MYACC.H files.  This is where any compiler dependent definitions are placed.  At present there are definitions for the Visual C++ and Borland C++ compilers.  Typically, for each compiler structure alignment will be placed in a known state and the YYCDECL, YYDCDECL, YYNEAR, YYFAR, YYNEARFAR, YYBASED_CODE, YYPROTOTYPE and YYCONST constants will be assigned appropriate values.  Note that it is not necessary to do this, so if you are using a different compiler, default values will be used in the LEX.H, YACC.H, MLEX.H and MYACC.H files.

YYBDEFS.H	Specifies common definitions to be includes at the bottom of the LEX.H, YACC.H, MLEX.H and MYACC.H files.  This is where any compiler dependent definitions are placed.  At present there are definitions for the Visual C++ and Borland C++ compilers.  Typically, for each compiler structure alignment will be returned to its previous value, after it was modified in the YYTDEFS.H file.

YYTUDEFS.H	Specifies other definitions to be included at the top of the LEX.H, YACC.H, MLEX.H and MYACC.H files.  This file is available for you to modify directly.  For instance, if you are using a compiler other than the Visual C++ or Borland C++ compilers then you may want to perform something similar to that mentioned for the YYTDEFS.H file.  Note though that you will first have to create the file.  In order for it to be included you will also have to define YYTUDEFS (or YYUDEFS if you are going to define YYBUDEFS.H as well).  This is typically done in the compiler's IDE, similar to the way you would define YYDEBUG when using the debug version of the YACC and Lex library.

YYBUDEFS.H	Specifies other definitions to be included at the bottom of the LEX.H, YACC.H, MLEX.H and MYACC.H files.  This file is available for you to modify directly.  For instance, if you are using a compiler other than the Visual C++ or Borland C++ compilers then you may want to perform something similar to that mentioned for the YYBDEFS.H file.  Note though that you will first have to create the file.  In order for it to be included you will also have to define YYBUDEFS (or YYUDEFS if you are going to define YYTUDEFS.H as well).  This is typically done in the compiler's IDE, similar to the way you would define YYDEBUG when using the debug version of the YACC and Lex library.

See Also

Flags


File Print command
Print Command (File Menu)

Use this command to print a document.  This command presents a Print dialog box, where you may specify the range of pages to be printed, the number of copies, the destination printer, and other printer setup options.

Shortcuts

Toolbar:

Keys:	CTRL+P


File Print Preview command
Print Preview Command (File Menu)

Use this command to display the active document as it would appear when printed.  When you choose this command, the main window will be replaced with a print preview window in which one or two pages will be displayed in their printed format.  The Print Preview Toolbar offers you options to view either one or two pages at a time; move back and forth through the document; zoom in and out of pages; and initiate a print job.


File Page Setup Command
Page Setup Command (File Menu)

Use this command to set up how the page is to be printed.  This command displays the Page Setup dialog box.  Note that page setup applies to all documents in the application.


Print Preview Toolbar

The Print Preview toolbar offers you the following options:

Print

Bring up the print dialog box, to start a print job.

Next Page

Preview the next printed page.

Prev Page

Preview the previous printed page.

One Page/Two Page

Preview one or two printed pages at a time.

Zoom In

Take a closer look at the printed page.

Zoom Out

Take a larger look at the printed page.

Close

Return from print preview to the editing window.


Print Progress Dialog Box

The Print Progress dialog box is shown during the time that Parser Generator is sending output to the printer.  The page number indicates the progress of the printing.

To abort printing, choose Cancel.


Functions

These functions are used by C parsers and lexical analysers.  The instance functions are only used when creating instances of multiple instance parsers or lexical analysers.

Parser Functions

Instance Functions

General Functions

Service Functions

Action Functions

Lexical Analyser Functions

Instance Functions

General Functions

Service Functions

Action Functions


Parser Instance Functions

These functions allow a parser instance to be created, and correspondingly destroyed.  They are only for use with parsers which have been created as multiple instance parsers.

Function	Description

yymcreateparse	Creates an instance of the parser.

yymdestroyparse	Destroys the parser instance.


yymcreateparse

#include <myacc.h>

int yymcreateparse( yymparse_t *yy, const yymparse_t *src )

Parameter	Description

yy	A pointer to the parser instance to create.  It cannot be NULL.

src	A pointer to the source parser.  This should be a multiple instance parser.  It cannot be NULL.

Return Value

Nonzero if successful, otherwise zero.  Failure occurs if the necessary memory for the initial state and attribute stacks cannot be allocated from the heap.

Remarks

The yymcreateparse function creates the specified parser instance.  The source parser src specifies everything about the parser, including the tables and service functions to use and the size of the initial state and attribute stacks.  This must be a multiple instance parser.

The yymcreateparse function also needs to allocate memory from the heap for the initial state and attribute stacks, and for the two attribute variables yyval and yylval.  In a parser instance, you can only get access to the attribute variables by using the yymvalptr and yymlvalptr members of the yymparse_t structure.  Note that the yymlvalptr member points to the attribute associated with a token whenever it is shifted onto the stack.  In order to access the attribute you must first cast it to a pointer of the required type:

YYSTYPE *yylvalptr = (YYSTYPE *) yy->yylvalptr;

*yylvalptr = 1;

If the function cannot allocate the necessary memory then the function fails and returns zero.  Note that in other parser models the initial stacks and attribute variables are static, and so do not need to be allocated.

The parser instance is destroyed by calling the yymdestroyparse function.

An example of how a parser instance is created, used and destroyed is shown below.  Note that in the example it is assumed that the parser is default (fast) parser:

#include "myparser.h"

int parse(void)
{

    yymparse_t parser;

    if (yymcreateparse(&parser, &myparser)) {

        int n = yymparse(&parser);

        yymdestroyparse(&parser);

        return n;
    }

    return 1;
}

The parser myparser is declared in the include file "myparser.h".  This is generated by AYACC.  Note that it is a multiple instance parser.

See Also

yymdestroyparse, yymparse_t


yymdestroyparse

#include <myacc.h>

void yymdestroyparse( yymparse_t *yy )

Parameter	Description

yy	A pointer to the parser instance to be destroyed.  It cannot be NULL.

Remarks

The yymdestroyparse function destroys the specified parser instance.  This must first have been created by a call to the yymcreateparse function.  The yymdestroyparse function frees up any memory allocated to the state and attribute stacks and the yyval and yylval variables.  The function does not attempt to free up the memory pointed to by the yy parameter itself.  For an example of how parser instances are created and destroyed see yymcreateparse.

The yymdestroyparse function frees the memory allocated to the parser by first calling the yymparsecleanup function.  This frees any memory allocated through calls to the yymsetstacksize function.  The function then frees the initial state and attribute stacks and the yyval and yylval variables allocated by the yymcreateparse function.

See Also

yymcreateparse, yymparse_t


Parser General Functions

These functions allow you to initiate a parse.  There are functions which perform initialisation prior to a parse taking place.  Functions to initiate the parse itself, and cleanup functions for when the parse terminates.

Function	Description

yycparse, yymcparse	Initiates a parse for a compact parser.

yycwipe, yymcwipe	Cleans up and symbols left on the stack and the lookahead token for a compact parser.

yycwork, yymcwork	Performs the main body of work for a compact parser.

yydestructpop, yymdestructpop	Pops the requested number of states off of the stack, calling any symbol destructor actions where necessary.

yyparse, yymparse	Initiates a parse for a default (fast) parser.

yyparsecleanup, yymparsecleanup	Frees any memory that may have been allocated through changing the size of the stack.

yyparseinit, yymparseinit	Initialises the streams associated with the parser.

yysetstacksize, yymsetstacksize	Sets the size of the parser’s stack.

yysetup, yymsetup	Sets up the parser prior to doing the main body of work.

yywipe, yymwipe	Cleans up any symbols left on the stack and the lookahead token for a default (fast) parser.

yywork, yymwork	Performs the main body of work for a default (fast) parser.


yydestructpop, yymdestructpop

#include <yacc.h>

void yydestructpop( int num )

#include <myacc.h>

void yymdestructpop( yymparse_t *yy, int num )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser function.  It cannot be NULL.

num	The number of states to pop off of the stack.  This must be greater than or equal to zero.

Remarks

The yydestructpop function pops the requested number of states off of the stack.  No checks are made on how many states are popped off of the stack so beware of causing a stack underflow.  For each nonterminal popped off of the stack, its destructor is called.  The destructor will typically free up any memory (like a node tree) that may be used by the nonterminal.  If a destructor has not been defined for a nonterminal then no destructor will be called.  Note that a state maps to either a nonterminal symbol or a token.

The yymdestructpop function is the multiple parser equivalent of the yydestructpop function.

See Also

yyretire, yymretire, yymparse_t


yyparse, yycparse, yymparse, yymcparse

#include <yacc.h>

int yyparse( void )

int yycparse( void )

#include <myacc.h>

int yymparse( yymparse_t *yy )

int yymcparse( yymparse_t *yy )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser functions.  It cannot be NULL.

Return Value

Whether the parse succeeded or not, or a user defined value.  YYEXIT_SUCCESS is returned on success and YYEXIT_FAILURE on failure.

Remarks

The yyparse and yycparse functions initiate a parse for the specified parser.  Call the yyparse function if you have built a default (fast) parser.  Call the yycparse function if you have built a compact parser.  The default implementation of main in the YACC and Lex library calls yyparse.  It is defined very simply as:

int main(void)

{

    return yyparse();

}

The yyparse and yycparse functions do not do much themselves.  Instead they call other functions to do the work for them.  Both yyparse and yycparse first call yysetup to set-up the parser and push the initial state onto the stack.  The yyparse function then calls the yywork function, while the yycparse function calls the yycwork function.  Both these functions perform the main body of work for the parser.

The yymparse function is the multiple parser equivalent of the yyparse function.  The yymcparse function is the multiple parser equivalent of the yycparse function.  A number of the functions mentioned above also have multiple parser equivalents.  For completeness they are listed below:

Function	Multiple Parser Equivalent Function

yysetup	yymsetup

yywork	yymwork

yycwork	yymcwork

See Also

yysetup, yywork, yycwork, yymsetup, yymwork, yymcwork, yymparse_t


yyparsecleanup, yymparsecleanup

#include <yacc.h>

void yyparsecleanup( void )

#include <myacc.h>

void yymparsecleanup( yymparse_t *yy )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser function.  It cannot be NULL.

Remarks

The yyparsecleanup function frees any memory that may have been allocated through calls to the yysetstacksize function.  This function is either called directly by yourself, or indirectly by the parser if it ever needs to grow the stack.  Do not call this function while the parser is running.

The yymparsecleanup function is the multiple parser equivalent of the yyparsecleanup function.  The yymsetstacksize function is the multiple parser equivalent of the yysetstacksize function.

The yymparsecleanup function will not free any memory allocated by the yymcreateparse function.  In order to do this you will need to call the yymdestroyparse function.  Note that this function in turn calls yymparsecleanup as part of its operation.

See Also

yysetstacksize, yymsetstacksize, yymparse_t


yyparseinit, yymparseinit

#include <yacc.h>

void yyparseinit( void )

#include <myacc.h>

void yymparseinit( yymparse_t *yy )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser function.  It cannot be NULL.

Remarks

The yyparseinit function initialises the streams used by the parser.  Usually, this will be done automatically.  There are however instances where this does not happen.  In Visual C++ and Borland C++Builder it is possible to have the run-time libraries in a DLL.  In such circumstances it is not possible to initialise a global variable with one of the standard streams.  The assignment needs to take place inside a function.

When a stream used by the parser cannot be initialised with one of the standard streams then it is initialised to NULL.  The debug version of the YACC and Lex library checks for NULL stream pointers whenever it performs any operations on them.  The default values for the parser streams are shown below:

Stream	Default Value

yyparseerr	stderr

yyparsedebugout	stdout

As an alternative to calling yyparseinit, you can simply initialise the streams yourself.

The yymparseinit function is the multiple parser equivalent of the yyparseinit function.  Instead of initialising the streams mentioned above, the yymparseinit function initialises their multiple parser equivalents:

Stream	Multiple Parser Equivalent

yyparseerr	yymerr member of the yymparse_t structure.

yyparsedebugout	yymdebugout member of the yymparse_t structure.

See Also

yyparseerr, yyparsedebugout, yymparse_t


yysetstacksize, yymsetstacksize

#include <yacc.h>

int yysetstacksize( int size )

#include <myacc.h>

int yymsetstacksize( yymparse_t *yy, int size )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser function.  It cannot be NULL.

size	The requested size of the parsers stack.  This must be greater than or equal to zero.

Return Value

Nonzero if the parsers stack was set to the specified size, otherwise zero in which case the size of the stack remains unchanged.

Remarks

The yysetstacksize function sets the parsers stack to the specified size.  Note that both the (state) stack and the attribute stack are changed.  You can change the size of the stack while the parser is running.  Note that if you specify a size that is less than the amount of stack currently in use by the parser, then the function will fail (and return zero).

This function may allocate memory from the heap if it needs to.  If the specified size is less that or equal to the size of the initial (static) stack, then it will use this.  By default this is 100 elements in size, unless of course you have redefined YYSTACK_SIZE in your YACC source file.  If it is greater than the initial stack size then it will allocate memory from the heap.

Note that this function is also used by the parser when it needs to automatically grow the stack.

The yymsetstacksize function is the multiple parser equivalent of the yysetstacksize function.  Note that a multiple instance parser does not have an initial (static) stack.  Instead the initial stack is allocated from memory by the yymcreateparse function.

See Also

yyparsecleanup, yymparsecleanup, yystackgrow, yymparse_t


yysetup, yymsetup

#include <yacc.h>

int yysetup( void )

#include <myacc.h>

int yymsetup( yymparse_t *yy )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser function.  It cannot be NULL.

Return Value

Zero if the parser has been set-up correctly ready for parsing, otherwise nonzero.

Remarks

The yysetup function is called by the yyparse and yycparse functions to initialise the parser's variables and push the initial state onto the parser's stack.  The function fails if it needs to allocate memory for the stack and it is unable to do so.

You can use this function along with the yywork and yycwork functions as an alternative to using yyparse and yycparse.  You first need to call yysetup and then the relevant work function depending upon whether you have created a default (fast) parser or a compact parser.

The yymsetup function is the multiple parser equivalent of the yysetup function.  A number of the functions mentioned above also have multiple parser equivalents.  For completeness they are listed below

Function	Multiple Parser Equivalent Function

yyparse	yymparse

yycparse	yymcparse

yywork	yymwork

yycwork	yymcwork

See Also

yyparse, yycparse, yywork, yycwork, yyretire, yymparse, yymcparse, yymwork, yymcwork, yymretire, yymparse_t


yywipe, yycwipe, yymwipe, yymcwipe

#include <yacc.h>

void yywipe( void )

void yycwipe ( void )

#include <myacc.h>

void yymwipe( yymparse_t *yy )

void yymcwipe( yymparse_t *yy )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser function.  It cannot be NULL.

Remarks

The yywipe and yycwipe functions cleanup any symbols that are currently in use by the parser.  The yywipe function performs cleanup for a default (fast) parser, and the yycwipe function performs cleanup for a compact parser.  The cleanup is destructive and involves popping all symbols (states) off of the stack and clearing the current lookahead token.  For each symbol discarded, if the symbol has a destructor associated with it, then the destructor will be called for that symbol in question.  It is the destructor itself which typically performs the cleanup.  For instance, if a node tree was being built by the parser, then the destructor would delete the node to which the symbol referred.  Note that in this case only nonterminal symbols would have associated destructors.  Tokens, typically, do not refer to nodes.  There are however situations in which it is desirable to be able to perform cleanup for a token.

The yywipe and yycwipe functions are useful for when calls are made to the yyexit, yyaccept or yyabort functions or indeed even the yyretire function.  These functions cause the parser to prematurely terminate the parse.  This will usually leave symbols on the stack, and possibly the current lookahead token holding a valid token.  Calling yywipe, or yycwipe will then ensure that the parser is cleaned up correctly.  Note that if the parse is not prematurely terminated, it will terminate on one of two conditions.  Either the parser will accept the input supplied to it and return with YYEXIT_SUCCESS, or it will fail in some way and return with YYEXIT_FAILURE.  In the first case, the stack will hold two symbols.  The first symbol being $accept (the auxiliary grammar rule which is used to construct the parser), and the second symbol being the start rule for the grammar.  The lookahead token will be set to $end.  The second case however, when the parse fails is the interesting one.  This can occur if the stack overflows, or an untrapped syntax error is encountered by the parser.  When this happens there will be symbols on the stack and/or the current lookahead token will hold a valid token.  In such cases, the parser is usually cleaned up automatically with a call to yywipe, or yycwipe.  This is controlled by the yywipeflg variable which is set to one by default i.e. the parser is cleaned up.

The yymwipe function is the multiple parser equivalent of the yywipe function, and the yymcwipe function is the multiple parser equivalent of the yymcwipe function.  A number of the functions mentioned above also have multiple parser equivalents.  For completeness they are listed below:

Function	Multiple Parser Equivalent Function

yyexit	yymexit

yyaccept	yymaccept

yyabort	yymabort

yyretire	yymretire

See Also

yyexit, yyaccept, yyabort, yyretire, yymexit, yymaccept, yymabort, yymretire, yywipeflg, yymparse_t


yywork, yycwork, yymwork, yymcwork

#include <yacc.h>

int yywork( void )

int yycwork( void )

#include <myacc.h>

int yymwork( yymparse_t *yy )

int yymcwork( yymparse_t *yy )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser functions.  It cannot be NULL.

Return Value

Whether the parse succeeded or not, or a user defined value.  YYEXIT_SUCCESS is returned on success and YYEXIT_FAILURE on failure.

Remarks

The yywork and yycwork functions carry out the main body of work for the parser.  The yywork function carries out the main body of work for a default (fast) parser and the yycwork function carries out the main body of work for a compact parser.  The yyparse and yycparse functions make use of these functions when a parse is initiated.

You can use these functions along with the yysetup function as an alternative to using yyparse and yycparse.  You first need to call yysetup and then the relevant work function depending upon whether you have created a default (fast) parser or a compact parser.

The work functions, and the associated yysetup function are primarily for use with the yyretire function.  If you call this function from within an action the parser will finish reducing the rule that the action is associated with and then exit with the supplied exit code.  Note that changes made to the $$ attribute are preserved when the new nonterminal symbol is shifted onto the stack.  You can then resume parsing at a later date with a call to the relevant work function.

Control will remain in either yywork or yycwork until one of the following conditions is met:

1	The parse succeeds.  In which case YYEXIT_SUCCESS is returned.

2	The parse fails.  In which case YYEXIT_FAILURE is returned.  Failure can occur if a syntax error is encountered and there are no error productions to handle it, or there is a stack overflow.

3	An action calls one of the yyexit, yyaccept, yyabort or yyretire functions.  In which case the exit code passed to the relevant function is returned.

The work functions also call a number of Service Functions.  These can be customised to change the way the parser behaves.

The yymwork function is the multiple parser equivalent of the yywork function.  The yymcwork function is the multiple parser equivalent of the yycwork function.  A number of the functions mentioned above also have multiple parser equivalents.  For completeness they are listed below:

Function	Multiple Parser Equivalent Function

yyparse	yymparse

yycparse	yymcparse

yysetup	yymsetup

yyexit	yymexit

yyaccept	yymaccept

yyabort	yymabort

yyretire	yymretire

See Also

yysetup, yyparse, yycparse, yyretire, yymsetup, yymparse, yymcparse, yymretire, yymparse_t


Parser Service Functions

These functions are called by the parser to perform some form of service.  It is possible, and indeed desirable to redefine these functions to customise the way the parser behaves.  For multiple parsers it is also possible to assign a different function to the relevant service function member in the yymparse_t structure.

Function	Description

yydiscard, yymdiscard	Called when the current lookahead token is discarded during error recovery.

yyerror, yymerror	Called when an error, either syntax or stack overflow occurs.

yygettoken, yymgettoken	Called by the parser whenever it needs a new token.

yystackoverflow, yymstackoverflow	Called when the parser’s stack overflows.

yysyntaxerror, yymsyntaxerror	Called when a syntax error occurs.


yydiscard, yymdiscard

#include <yacc.h>

void yydiscard( int token )

#include <myacc.h>

void yymdiscard( yymparse_t *yy, int token )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser function.  It cannot be NULL.

token	The token which is being discarded.

Remarks

The yydiscard function is called when the parser is in error recovery and it discards the current lookahead token.  It does this whenever it cannot successfully shift the token onto the stack.  Typically the parser will remain in error recovery until it has shifted three consecutive tokens onto the stack.  Note that error recovery can be completed by use of the yyerrok function.

The default implementation of yydiscard simply does nothing.  Redefine the function in your source code if you wish to report when tokens are discarded.

The yymdiscard function is the multiple parser equivalent of the yydiscard function.  The yymerrok function is the multiple parser equivalent of the yyerrok function.  If you do redefine yymdiscard in your source code then, by default, all multiple parsers will make use of the new function.  As an alternative, you can define a new function with the same prototype and assign it to the yymdiscard member of the yymparse_t structure.  You can do this either statically or at run time.  To do it statically, you need to define YYMDISCARD as the function in your YACC source file.  This needs to be done in the declarations section:

%{

void mydiscard(yymparse_t *yy, int token)

{

    /* do something */
}

#define YYMDISCARD mydiscard

%}

See Also

yysyntaxerror, yymsyntaxerror, yymparse_t


yyerror, yymerror

#include <yacc.h>

void yyerror( const char *text )

#include <myacc.h>

void yymerror( yymparse_t *yy, const char *text )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser function.  It cannot be NULL.

text	The zero terminated error string.

Remarks

The yyerror function writes the string text to the stream pointed to by the yyparseerr variable, followed by a newline \n character.  The yyerror function is called by the yysyntaxerror and yystackoverflow functions when they report an error.  Redefine yyerror in your source code if you wish to change how errors are reported.

The yymerror function is the multiple parser equivalent of the yyerror function.  The yymsyntaxerror function is the multiple parser equivalent of the yysyntaxerror function, and the yymstackoverflow function is the multiple parser equivalent of the yystackoverflow function.

If you do redefine yymerror in your source code then, by default, all multiple parsers will make use of the new function.  As an alternative, you can define a new function with the same prototype and assign it to the yymerror member of the yymparse_t structure.  You can do this either statically or at run time.  To do it statically, you need to define YYMERROR as the function name in your YACC source file.  This needs to be done in the declarations section:

%{

void myerror(yymparse_t *yy, const char *text)

{

    /* do something */
}

#define YYMERROR myerror

%}

See Also

yysyntaxerror, yystackoverflow, yymsyntaxerror, yymstackoverflow, yymparse_t


yygettoken, yymgettoken

#include <yacc.h>

int yygettoken( void )

#include <myacc.h>

int yymgettoken( yymparse_t *yy )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser function.  It cannot be NULL.

Return Value

Zero or negative if there are no more tokens for the parser, otherwise a positive value indicating the next token.

Remarks

The yygettoken function is called by the parser whenever it needs a new token.  The function returns a zero or negative value if there are no more tokens, otherwise a positive value indicating the next token.  The default implementation of yygettoken simply calls yylex:

int yygettoken(void)
{

    return yylex();
}

Redefine yygettoken in your source code if you wish to change how the parser gets its tokens.  For instance you may want to run some code before calling yylex.  You will need to redefine the yygettoken function if you are going to use a compact lexical analyser.  Instead of calling yylex, you will need to call yyclex:

int yygettoken(void)

{

    return yyclex();
}

The parser is also expecting any attribute associated with the token to be assigned to yylval.  This variable has the same type as that specified by the %union keyword, or the YYSTYPE define in your YACC source file.

The yymgettoken function is the multiple parser equivalent of the yygettoken function.  The yymlex function is the multiple lexical analyser equivalent of the yylex function, and the yymclex function is the multiple parser equivalent of the yyclex function.

The default implementation of yymgettoken is slightly different to that for yygettoken.  It assumes that the yymdata member of the yymparse_t structure points to a multiple lexical analyser.  Instead of the function calling yylex, it calls yymlex with this pointer:

int yymgettoken(yymparse_t *yy)
{

    yymlex_t *p = (yymlex_t *) yy->yymdata;

    return yymlex(p);
}

You can assign a value to the yymdata member either statically or at run time.  To do it statically, you need to define YYMPARSEDATA as the address of the multiple lexical analyser in your YACC source file.  This needs to be done in the declarations section.  Note that the header file "mylexer.h" holds the name of the multiple lexical analyser:

%{

#include "mylexer.h"

#define YYMPARSEDATA &mylexer
%}

If you do redefine yymgettoken in your source code then, by default, all multiple parsers will make use of the new function.  As an alternative, you can define a new function with the same prototype and assign it to the yymgettoken member of the yymparse_t structure.  You can do this either statically or at run time.  To do it statically, you need to define YYMGETTOKEN as the function name in your YACC source file.  This needs to be done in the declarations section:

%{

int mygettoken(yymparse_t *yy)

{

    /* do something */
}

#define YYMGETTOKEN mygettoken

%}

Multiple parsers also expect attributes to be associated with tokens.  For a multiple parser (without instances) it is possible to assign the attribute to the yylval variable.  It is also possible to retrieve a pointer to the yylval variable from the yymparse_t structure.  This is done by casting the yymlvalptr member to a YYSTYPE * type.  For multiple instance parsers it is only possible to access the yylval variable through this pointer.

See Also

yylex, yyclex, yymlex, yymclex, yymparse_t


yystackoverflow, yymstackoverflow

#include <yacc.h>

void yystackoverflow( void )

#include <myacc.h>

void yymstackoverflow( yymparse_t *yy )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser function.  It cannot be NULL.

Remarks

The yystackoverflow function is called if the parser’s stack ever overflows. This can be for one of three reasons:

1	The stack is not growable, and has simply overflowed.

2	The stack is growable, however the memory block requested by the parser is simply too big to fit in an internal representation. For instance, in 16-bit DOS applications, the largest block of memory that can be allocated is 65,535 bytes in size.  Note also that when the stack grows both the (state) stack and the attribute stack grow.  For a 16-bit application, a state is 2 bytes in size. Similarly, an attribute is also 2 bytes in size if the default int type is used.  However, if you redefine the attribute type you may find its storage requirements increase.  Correspondingly this will lead to an increase in the size of the attribute stack.

3	The stack is growable, however the computer has run out of memory. Note this is not very likely on modern machines.

The default implementation of yystackoverflow simply calls yyerror with the error message "yacc stack overflow".  Redefine this function in your source code if you wish to handle stack overflow reporting differently.  Note that when yystackoverflow returns, the calling parser function (either yyparse, yycparse, yywork or yycwork) will itself return immediately with the exit code YYEXIT_FAILURE.

The yymstackoverflow function is the multiple parser equivalent of the yystackoverflow function.  A number of the other functions listed above also have multiple parser equivalent functions.  For completeness they are listed below:

Function	Multiple Parser Equivalent Function

yyerror	yymerror

yyparse	yymparse

yycparse	yymcparse

yywork	yymwork

yycwork	yymcwork

If you do redefine yymstackoverflow in your source code then, by default, all multiple parsers will make use of the new function.  As an alternative, you can define a new function with the same prototype and assign it to the yymstackoverflow member of the yymparse_t structure.  You can do this either statically or at run time.  To do it statically, you need to define YYMSTACKOVERFLOW as the function name in your YACC source file.  This needs to be done in the declarations section:

%{

void mystackoverflow(yymparse_t *yy)

{

    /* do something */
}

#define YYMSTACKOVERFLOW mystackoverflow

%}

See Also

yyerror, yysetstacksize, yymerror, yymsetstacksize, yystackgrow, yymparse_t


yysyntaxerror, yymsyntaxerror

#include <yacc.h>

void yysyntaxerror( void )

#include <myacc.h>

void yymsyntaxerror( yymparse_t *yy )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser function.  It cannot be NULL.

Remarks

The yysyntaxerror function is called by the parser whenever a syntax error occurs.  The function simply calls yyerror with the error message "syntax error".  Redefine this function in your source code if you wish to handle syntax error reporting differently.

The yymsyntaxerror function is the multiple parser equivalent of the yysyntaxerror function.  The yymerror function is the multiple parser equivalent of the yyerror function.

If you do redefine yymsyntaxerror in your source code then, by default, all multiple parsers will make use of the new function.  As an alternative, you can define a new function with the same prototype and assign it to the yymsyntaxerror member of the yymparse_t structure.  You can do this either statically or at run time.  To do it statically, you need to define YYMSYNTAXERROR as the function name in your YACC source file.  This needs to be done in the declarations section:

%{

void mysyntaxerror(yymparse_t *yy)

{

    /* do something */
}

#define YYMSYNTAXERROR mysyntaxerror

%}

See Also

yyerror, yymerror, yymparse_t


Parser Action Functions

These functions are typically called from an action to modify the way the parser behaves.  Indeed in order for some functions to have an effect they must be called from an action.

Function	Description

yyabort, yymabort	Forces the parser to return with the YYEXIT_FAILURE exit code.

yyaccept, yymaccept	Forces the parser to return with the YYEXIT_SUCCESS exit code.

yycdestructclearin, yymcdestructclearin	Clears the current lookahead token for a compact parser, calling a symbol destructor for the token if necessary.

yyclearin, yymclearin	Clears the current lookahead token.

yydestructclearin, yymdestructclearin	Clears the current lookahead token for a default (fast) parser, calling a symbol destructor for the token if necessary.

yyerrok, yymerrok	Re-synchronises error handling.

yyexit, yymexit	Forces the parser to return with an exit code.

yyforceerror, yymforceerror	Forces the parser to generate a syntax error.

yypopping, yympopping	Indicates whether the parser has popped any states off of the stack whilst looking for an error token transition.

yyrecovering, yymrecovering	Indicates whether the parser is in error recovery.

yyretire, yymretire	Forces the parser to return with an exit code after first reducing the current rule.

yysetin, yymsetin	Sets the current lookahead token to the specified token.

yythrowerror, yymthrowerror	Forces the parser to generate a syntax error.  The parser will then skip the specified number of error token transitions before attempting recovery.

yyunclearin, yymunclearin	Returns the lookahead token to its previous value if it is currently clear.


yyclearin, yymclearin

#include <yacc.h>

void yyclearin( void )

#include <myacc.h>

void yymclearin( yymparse_t *yy )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser function.  It cannot be NULL.

Remarks

The yyclearin function discards the current lookahead token for the parser.  This function has no effect if there is no current lookahead token.  The effects of the yyclearin function can be reversed by calling the yyunclearin function.

The yymclearin function is the multiple parser equivalent of the yyclearin function.  The yymunclearin function is the multiple parser equivalent of the yyunclearin function.

Compatibility

In traditional YACC, the yyclearin function is implemented as a macro without any trailing parentheses.  If you want to use this implementation of yyclearin then you need to define YY_COMPATIBLE.  This needs to be done before the header file yacc.h has been included in your source files.  The easiest way to do this is by using a command line option.  If you are using an IDE like Visual C++ or Borland C++ then you can do this more easily through the relevant project option dialog boxes.

If you do define YY_COMPATIBLE then it is not just the implementation of yyclearin which changes.  The yyerrok function is changed too.  Similar to yyclearin, it is implemented as a macro without any trailing parentheses.

See Also

yyunclearin, yymunclearin, yychar, yylookahead, yymparse_t


yydestructclearin, yycdestructclearin, yymdestructclearin, yymcdestructclearin

#include <yacc.h>

void yydestructclearin( void )

void yycdestructclearin( void )

#include <myacc.h>

void yymdestructclearin( yymparse_t *yy )

void yymcdestructclearin( yymparse_t *yy )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser function.  It cannot be NULL.

Remarks

The yydestructclearin and yycdestructclearin functions discard the current lookahead token for the parser.  The yydestructclearin function discards the current lookahead token for a default (fast) parser, and the yycdestructclearin function discards the lookahead token for a compact parser.  Unlike yyclearin, yydestructclearin and yycdestructclearin also perform clean up for the token being discarded.  That is, if the token has a symbol destructor associated with it, then the destructor will be called for that token.  The effects of a call to yydestructclearin or yycdestructclearin may be fully, or partially restored by a call to yyunclearin.  Note that this will depend upon what the symbol destructor, if any, associated with the lookahead token does.

The yymdestructclearin function is the multiple parser equivalent of the yydestructclearin function, and the yymcdestructclearin function is the multiple parser equivalent of the yymcdestructclearin function.  A number of the other functions mentioned above also mhave multiple parser equivalents.  For completeness they are listed below:

Function	Multiple Parser Equivalent Function

yyclearin	yymclearin

yyunclearin	yymunclearin

See Also

yyclearin, yydestructpop, yymclearin, yymdestructpop, yychar, yylookahead, yymparse_t


yyerrok, yymerrok

#include <yacc.h>

void yyerrok( void )

#include <myacc.h>

void yymerrok( yymparse_t *yy );

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser function.  It cannot be NULL.

Remarks

The yyerrok function informs the parser that error recovery is complete.  This function has no effect if the parser is not currently in error recovery.  Typically, error recovery is entered after a syntax error is encountered by the parser.  It is possible for an action to generated a syntax error through the use of the yythrowerror function.

When the parser encounters a syntax error it inserts an error token in front of the current lookahead token, if any.  The parser then examines the state on top of the stack to see whether it has an error token transition.  If not the parser pops the state off of the stack and examines the next one.  It repeats this until either an error token transition is found or all of the states are popped off of the stack.  If this happens the parse fails, and the parser returns with the exit code YYEXIT_FAILURE.

When, or if an error token transition is found, the parser can do one of two things.  Note that the action depends upon whether the yythrowerror function generated the syntax error or not, and if so the value of the pop parameter passed to it:

1	If the yythrowerror function generated the syntax error and the pop parameter is greater than zero then it is decremented.  The state is popped off of the stack, and the parser continues to look for error token transitions.  Note that if the state is a nonterminal symbol state, and that symbol has a destructor associated with it, then the destructor will be called for the state in question.

2	If the yythrowerror function generated the syntax error and the pop parameter is zero, or the function didn't generate the error, then the error token is shifted onto the stack and the parser enters error recovery.

Note that when it says that the error token is shifted onto the stack, it doesn’t mean the token itself, but rather the state that the error token transition goes to.

Error token transitions themselves are formed by including error productions in your grammar.  The identifier error is predefined by YACC, and is used to represent the error token.

A: B C      { /* do something */ }

 | error a  { /* do something */ }

 ;

The parser stays in error recovery until three consecutive tokens have been shifted onto the stack.  Note that this does not include the error token itself.  During this time no more syntax errors are reported, including any that are generated by the yythrowerror function.  Any tokens which cannot be shifted onto the stack are discarded.

The yymerrok function is the multiple parser equivalent of the yyerrok function.  The yymthrowerror function is the multiple parser equivalent of the yythrowerror function.

Compatibility

In traditional YACC, the yyerrok function is implemented as a macro without any trailing parentheses.  If you want to use this implementation of yyerrok then you need to define YY_COMPATIBLE.  This needs to be done before the header file yacc.h has been included in your source files.  The easiest way to do this is by using a command line option.  If you are using an IDE like Visual C++ or Borland C++ then you can do this more easily through the relevant project option dialog boxes.

If you do define YY_COMPATIBLE then it is not just the implementation of yyerrok which changes.  The function yyclearin is changed too.  Similar to yyerrok, it is implemented as a macro without any trailing parentheses.

See Also

yythrowerror, yymthrowerror, yymparse_t


yyexit, yyaccept, yyabort, yymexit, yymaccept, yymabort

#include <yacc.h>

void yyexit( int exitcode )

void yyaccept( void )

void yyabort( void )

#include <myacc.h>

void yymexit( yymparse_t *yy, int exitcode )

void yymaccept( yymparse_t *yy )

void yymabort( yymparse_t *yy )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser functions.  It cannot be NULL.

exitcode	The exit code that the parser is to return with.  The parser itself reserves two exit codes.  These are YYEXIT_SUCCESS and YYEXIT_FAILURE.  YYEXIT_SUCCESS is returned if the parse succeeds, and YYEXIT_FAILURE is returned if the parse fails.  Failure can occur if the stack overflows or an untrapped syntax error is encountered.

Remarks

The yyexit function forces the parser to return with the given exit code.  The function has some limitations on where it can be used, however these are not generally restrictive.  If the function is called from elsewhere then it will have no effect.  The yyexit function can be:

1	Called from an action.  This does not include actions associated with symbol destructors.

2	Called from some function, but program flow of control must return to an action at some point.  Again this does not include actions associated with symbol destructors.

The yyexit function does not take effect immediately, but rather when control returns from the action to the parser.  The rule that the action is associated with is not reduced, and any assignment to the $$ attribute is lost.  Assignments to the $n attributes will stand.

There are a number of other functions which take effect when control returns from an action to the parser.  These are the yythrowerror, yyforceerror and yyretire functions.  The yyexit function takes priority over all of these functions.  The effect of any calls to them will be lost if yyexit is called.

If the yyexit function is called several times from within an action, then the exit code for the last call is the value that is returned by the parser.

The yyaccept and yyabort functions are shorthand's for the most commonly used yyexit calls.  The yyaccept function is equivalent to yyexit(YYEXIT_SUCCESS), and the yyabort function is equivalent to yyexit(YYEXIT_FAILURE).

The yymexit function is the multiple parser equivalent of the yyexit function.  Correspondingly, the yymaccept function is the multiple parser equivalent of the yyaccept function, and the yymabort function is the multiple parser equivalent of the yyabort function.  A number of the other functions mentioned above also have multiple parser equivalent functions.  For completeness they are listed below:

Function	Multiple Parser Equivalent Function

yythrowerror	yymthrowerror

yyforceerror	yymforceerror

yyretire	yymretire

Compatibility

The function yyaccept is also implemented as the macro YYACCEPT, and the function yyabort is implemented as the macro YYABORT.  The macros do not have any trailing parentheses.  In traditional YACC, these macros will cause an immediate jump to the relevant accept or abort code.  This means that no code following the YYACCEPT or YYABORT macros will be executed.  In traditional YACC, due to implementation details, these macros can only be placed in the actions themselves.  They cannot be placed in a function called from an action.  Note that this does not apply to AYACC, where YYACCEPT and YYABORT can be placed anywhere that yyaccept and yyabort can.

See Also

yyretire, yywipe, yymretire, yymwipe, yymparse_t


yypopping, yympopping

#include <yacc.h>

int yypopping( void )

#include <myacc.h>

int yympopping( yymparse_t *yy )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser function.  It cannot be NULL.

Return Value

Nonzero if the parser has been popping states off of the stack whilst looking for an error token transition, otherwise zero.

Remarks

The yypopping function returns whether the parser has been popping states off of the stack whilst looking for an error token transition.  This occurs when the parser encounters a syntax error.  The parser first of all examines the top most state to see whether it has an error token transition.  If not then it pops the state off of the stack, and looks at the next state.  It repeats this until either all of the states on the stack have been popped off of it or an error transition is found.  Note that if the yythrowerror function caused the syntax error, then there is the possibility that one or more states with error token transitions will also be popped off of the stack as well.

The value returned by the yypopping function will remain the same until another syntax error is encountered by the parser.  For instance, if the parser encounters a syntax error and recovery first takes place after popping one or more states, then the yypopping function will return a nonzero value from that point onwards.  It is only when another syntax error is encountered that there is a possibility of the return value changing i.e. if no states are popped off of the stack whilst attempting error recovery.

The yypopping function is useful in conjunction with the yythrowerror function for skipping over error productions.  For example, the rule below recognises a left parenthesis.  It also has an error production for reporting when said left parenthesis is missing.  In it, it has a test to see whether the parser has been popping states whilst trying to look for an error token transition.  If so then the yythrowerror function is called.  This will cause the parser to skip over this error production (the 1), and look for another one.  The reasoning behind this is quite simple.  If the parser has been popping states off of the stack then it means that there was something after the left parenthesis.  Reporting a missing parenthesis for an error which has occurred later on does not make sense:

LParen

    : '('

    | error {

                if (yypopping()) {

                    yythrowerror(1);

                }

                else {

                    yyerror("missing \'(\'"); }

                }

            }

    ;

The yympopping function is the multiple parser equivalent of the yypopping function.  Similarly the yymthrowerror function is the multiple parser equivalent of the yythrowerror function.

See Also

yythrowerror, yymthrowerror, yymparse_t


yyrecovering, yymrecovering

#include <yacc.h>

int yyrecovering( void )

#include <myacc.h>

int yymrecovering( yymparse_t *yy )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser function.  It cannot be NULL.

Return Value

Nonzero if the parser is in error recovery, otherwise zero.

Remarks

The yyrecovering function returns whether the parser is in error recovery or not.  Typically, error recovery is entered after a syntax error is encountered by the parser.  It is possible for an action to generated a syntax error through the use of the yythrowerror function.

When the parser encounters a syntax error it inserts an error token in front of the current lookahead token, if any.  The parser then examines the state on top of the stack to see whether it has an error token transition.  If not the parser pops the state off of the stack and examines the next one.  It repeats this until either an error token transition is found or all of the states are popped off of the stack.  If this happens the parse fails, and the parser returns with the exit code YYEXIT_FAILURE.

When, or if an error token transition is found, the parser can do one of two things.  Note that the action depends upon whether the yythrowerror function generated the syntax error or not, and if so the value of the pop parameter passed to it:

1	If the yythrowerror function generated the syntax error and the pop parameter is greater than zero then it is decremented.  The state is popped off of the stack, and the parser continues to look for error token transitions.  Note that if the state is a nonterminal symbol state, and that symbol has a destructor associated with it, then the destructor will be called for the state in question.

2	If the yythrowerror function generated the syntax error and the pop parameter is zero, or the function didn't generate the error, then the error token is shifted onto the stack and the parser enters error recovery.

Note that when it says that the error token is shifted onto the stack, it doesn’t mean the token itself, but rather the state that the error token transition goes to.

Error token transitions themselves are formed by including error productions in your grammar.  The identifier error is predefined by YACC, and is used to represent the error token.

A: B C      { /* do something */ }

 | error a  { /* do something */ }

 ;

The parser stays in error recovery until three consecutive tokens have been shifted onto the stack.  Note that this does not include the error token itself.  During this time no more syntax errors are reported, including any that are generated by the yythrowerror function.  Any tokens which cannot be shifted onto the stack are discarded.  Note that it is possible to re-synchronise error recovery through the yyerrok function.

The yymrecovering function is the multiple parser equivalent of the yyrecovering function.  The yymthrowerror function is the multiple parser equivalent of the yythrowerror function, and the yymerrok function is the multiple parser equivalent of the yyerrok function.

Compatibility

The function yyrecovering is also implemented as the macro YYRECOVERING.  The macro does not have any trailing parentheses.

See Also

yyerrok, yymerrok, yymparse_t


yyretire, yymretire

#include <yacc.h>

void yyretire( int exitcode )

#include <myacc.h>

void yymretire( yymparse_t *yy, int exitcode )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser function.  It cannot be NULL.

exitcode	The exit code that the parser is to return with.  The parser itself reserves two exit codes.  These are YYEXIT_SUCCESS and YYEXIT_FAILURE.  YYEXIT_SUCCESS is returned if the parse succeeds, and YYEXIT_FAILURE is returned if something goes wrong.  For instance the stack overflows, or an untrapped syntax error is encountered.

Remarks

The yyretire function forces the parser to return with the given exit code.  The function has some limitations on where it can be used, however these are not generally restrictive.  If the function is called from elsewhere then it will have no effect.  The yyretire function can be:

1	Called from an action.  This does not include actions associated with symbol destructors.

2	Called from some function, but program flow of control must return to an action at some point.  Again this does not include actions associated with symbol destructors.

The yyretire function does not take effect immediately, but rather when control returns from the action to the parser.  The rule that the action is associated with is reduced, and any assignment to the $$ attribute is saved on the attribute stack.  Compare this to the effects of the yyexit function.

You can continue the parse at a subsequent date by calling one of the work functions.  The state and attribute stacks are preserved to allow you to do this.  If you have created a default (fast) parser then you would call yywork, otherwise for a compact parser you would call yycwork.

There are a number of other functions which take effect when control returns from an action to the parser.  These are the yyexit, yyaccept, yyabort, yythrowerror and yyforceerror and functions.  All of these functions take priority over the yyretire function.  The effect of a call to yyretire will be lost if any of them are called.

If the yyretire function is called several times from within an action, then the exit code for the last call is the value that is returned by the parser.

The yymretire function is the multiple parser equivalent of the yyretire function.  A number of the other functions mentioned above also have multiple parser equivalent functions.  For completeness they are listed below:

Function	Multiple Parser Equivalent Function

yywork	yymwork

yycwork	yymcwork

yyexit	yymexit

yyaccept	yymaccept

yyabort	yymabort

yythrowerror	yymthrowerror

yyforceerror	yymforceerror

See Also

yywork, yycwork, yyexit, yymwork, yymcwork, yymexit, yymparse_t


yysetin, yymsetin

#include <yacc.h>

void yysetin( int token )

#include <myacc.h>

void yymsetin( yymparse_t *yy, int token )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser function.  It cannot be NULL.

token	The value to set the current lookahead token to.  A value of zero or a negative number indicates the $end (EOF, DONE etc.) token.

Remarks

The yysetin function sets the current lookahead token to the given value.  No checks are made whether the token is valid or not.  If the current lookahead token already holds a value then it is overwritten.

The yymsetin function is the multiple parser equivalent of the yysetin function.

See Also

yyclearin, yymclearin, yymparse_t


yythrowerror, yyforceerror, yymthrowerror, yymforceerror

#include <yacc.h>

void yythrowerror( int pop )

void yyforceerror( void )

#include <myacc.h>

void yymthrowerror( yymparse_t *yy, int pop )

void yymforceerror( yymparse_t *yy )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser functions.  It cannot be NULL.

pop	The number of states with error token transitions to pop off of the stack before entering error recovery with the next one found.  This parameter must be zero or a positive number.

Remarks

The yythrowerror function forces the parser to generate a syntax error.  The function has some limitations on where it can be used, however these are not generally restrictive.  If the function is called from elsewhere then it will have no effect.  The yythrowerror function can be:

1	Called from an action.  This does not include actions associated with symbol destructors.

2	Called from some function, but program flow of control must return to an action at some point.  Again this does not include actions associated with symbol destructors.

The yythrowerror function does not take effect immediately, but rather when control returns from the action to the parser.  The rule that the action is associated with is not reduced, and any assignment to the $$ attribute is lost.  Assignments to the $n attributes will stand.

When control returns to the parser it inserts an error token in front of the current lookahead token, if any.  The parser then examines the state on top of the stack to see whether it has an error token transition.  If not the parser pops the state off of the stack and examines the next one.  It repeats this until either an error token transition is found or all of the states are popped off of the stack.  If this happens the parse fails, and the parser returns with the exit code YYEXIT_FAILURE.

When, or if an error token transition is found, the parser can do one of two things.  Note that the action is determined by the value of the pop parameter passed to the yythrowerror function:

1	If the pop parameter is greater than zero then it is decremented.  The state is popped off of the stack, and the parser continues to look for error token transitions.  Note that if the symbol represented by the state has a symbol destructor associated with it then the destructor will be called for the state in question.

2	If the pop parameter is zero then the error token is shifted onto the stack and the parser enters error recovery.

Note that when it says that the error token is shifted onto the stack, it doesn’t mean the token itself, but rather the state that the error token transition goes to.

Error token transitions themselves are formed by including error productions in your grammar.  The identifier error is predefined by YACC, and is used to represent the error token.

A: B C      { /* do something */ }

 | error a  { /* do something */ }

 ;

The parser stays in error recovery until three consecutive tokens have been shifted onto the stack.  Note that this does not include the error token itself.  During this time no more syntax errors are reported, including any that are generated by the yythrowerror function.  Any tokens which cannot be shifted onto the stack are discarded.  Note that it is possible to re-synchronise error recovery through the yyerrok function.

There are a number of other functions which take effect when control returns from an action to the parser.  These are the yyexit, yyaccept, yyabort, and yyretire functions.  The yythrowerror function takes priority over the yyretire function, however the yyexit, yyaccept and yyabort functions take priority over yythrowerror.  The effect of any calls to yyretire will be lost if yythrowerror is called.

If the yythrowerror function is called several times from within an action, then the pop argument for the last call is the value that is used by the parser.

The yyforceerror function is a shorthand for the most commonly used yythrowerror(0) call.

The yymthrowerror function is the multiple parser equivalent of the yythrowerror function.  Similarly, the yymforceerror function is the multiple parser equivalent of the yyforceerror function, and the yymerrok function is the multiple parser equivalent of the yyerrok function.

Compatibility

The function yyforceerror is also implemented as the macro YYERROR.  The macro does not have any trailing parentheses.  In traditional YACC, this macro will cause an immediate jump to the relevant error handling code.  This means that no code following the YYERROR macro will be executed.  In traditional YACC, due to implementation details, this macro can only be placed in an action itself.  It cannot be placed in a function called from an action.  Note that this does not apply to AYACC, where YYERROR can be placed anywhere that yyforceerror can.

See Also

yyerrok, yymerrok, yymparse_t


yyunclearin, yymunclearin

#include <yacc.h>

int yyunclearin( void )

#include <myacc.h>

int yymunclearin( yymparse_t *yy )

Parameter	Description

yy	A pointer to the parser.  This parameter is only used by the multiple parser function.  It cannot be NULL.

Return Value

Nonzero if the current lookahead token could be restored to its previous value, otherwise zero.

Remarks

The yyunclearin function restores the current lookahead token to its previous value.  The function succeeds if the current lookahead token is empty and it has previously held a value.

The yymunclearin function is the multiple parser equivalent of the yyunclearin function.

See Also

yyclearin, yymclearin, yymparse_t


Lexical Analyser Instance Functions

These functions allow a lexical analyser instance to be created, and correspondingly destroyed.  They are only for use with lexical analysers which have been created as multiple instance lexical analysers.

Function	Description

yymcreatelex	Creates an instance of the lexical analyser.

yymdestroylex	Destroys the lexical analyser instance.


yymcreatelex

#include <mlex.h>

int yymcreatelex( yymlex_t *yy, const yymlex_t *src )

Parameter	Description

yy	A pointer to the lexical analyser instance to create.  It cannot be NULL.

src	A pointer to the source lexical analyser.  This should be a multiple instance lexical analyser.  It cannot be NULL.

Return Value

Nonzero if successful, otherwise zero.  Failure occurs if the necessary memory for the initial text, state and unput buffers cannot be allocated from the heap.

Remarks

The yymcreatelex function creates the specified lexical analyser instance.  The source lexical analyser src specifies everything about the lexical analyser, including the tables and service functions to use and the sizes of the initial text, state and unput buffers.  This must be a multiple instance lexical analyser.

The yymcreatelex function also needs to allocate memory from the heap for the initial text, state and unput buffers.  If the function cannot allocate the necessary memory then the function fails and returns zero.  Note that in other lexical analyser models the initial buffers are static, and so do not need to be allocated.

The lexical analyser instance is destroyed by calling the yymdestroylex function.

An example of how a lexical analyser instance is created, used and destroyed is shown below.  Note that in the example it is assumed that the lexical analyser is a default (fast) lexical analyser:

#include "mylexer.h"

int analyse(void)
{

    yymlex_t lexer;

    if (yymcreatelex(&lexer, &mylexer)) {

        int n = yymlex(&lexer);

        yymdestroylex(&lexer);

        return n;

   }

   return 1;
}

The lexical analyser mylexer is declared in the include file "mylexer.h".  This is generated by ALex.  Note that it is a multiple instance lexical analyser.

See Also

yymdestroylex, yymlex_t


yymdestroylex

#include <mlex.h>

void yymdestroylex( yymlex_t *yy )

Parameter	Description

yy	A pointer to the lexical analyser instance to be destroyed.  It cannot be NULL.

Remarks

The yymdestroylex function destroys the specified lexical analyser instance.  This must first have been created by a call to the yymcreatelex function.  The yymdestroylex function frees up any memory allocated to the text, state and unput buffers.  The function does not attempt to free up the memory pointed to by the yy parameter itself.  For an example of how lexical analyser instances are created and destroyed see yymcreatelex.

The yymdestroylex function frees the memory allocated to the lexical analyser by first calling the yymlexcleanup function.  This frees any memory allocated through calls to the yymsettextsize and yymsetunputsize functions.  The function then frees the initial text, state and unput buffers allocated by the yymcreatelex function.

See Also

yymcreatelex, yymlex_t


Lexical Analyser General Functions

These functions allow you to perform lexical analysis on your input.  There are functions which initialise the lexical analyser prior to lexical analysis taking place.  Functions to perform lexical analysis itself, and cleanup functions for when lexical analysis finishes.

Function	Description

yyclex, yymclex	Performs lexical analysis for a compact lexical analyser.

yylex, yymlex	Performs lexical analysis for a default (fast) lexical analyser.

yylexcleanup, yymlexcleanup	Frees any memory that may have been allocated through changing the size of the text and unput buffers.

yylexinit, yymlexinit	Initialises the streams associated with the lexical analyser.

yyreset, yymreset	Resets the lexical analyser.

yysettextsize, yymsettextsize	Sets the size of the lexical analyser’s text buffer and state buffer.

yysetunputsize, yymsetunputsize	Sets the size of the lexical analyser’s unput buffer.


yylex, yyclex, yymlex, yymclex

#include <lex.h>

int yylex( void )

int yyclex( void )

#include <mlex.h>

int yymlex( yymlex_t *yy )

int yymclex( yymlex_t *yy )

Parameter	Description

yy	A pointer to the lexical analyser.  This parameter is only used by the multiple lexical analyser functions.  It cannot be NULL.

Return Value

Zero if the lexical analyser has no more processing to do, otherwise user defined.

Remarks

The yylex and yyclex functions cause the lexical analyser to process input.  The yylex function processes input for a default (fast) lexical analyser and the yyclex function processes input for a compact lexical analyser.

Control will remain in either yylex or yyclex until one of the following conditions is met:

1	The lexical analyser has run out of input.  In which case zero is returned.

2	An action executes a return statement.  In which case the expression specified in the statement is returned by yylex or yyclex.

The yylex and yyclex functions also call a number of Service Functions.  These can be customised to change the way the lexical analyser behaves.

The yymlex function is the multiple lexical analyser equivalent of the yylex functions.  The yymclex function is the multiple lexical analyser equivalent of the yyclex function.

See Also

yyreset, yymreset, yymlex_t


yylexcleanup, yymlexcleanup

#include <lex.h>

void yylexcleanup( void )

#include <mlex.h>

void yymlexcleanup( yymlex_t *yy )

Parameter	Description

yy	A pointer to the lexical analyser.  This parameter is only used by the multiple lexical analyser function.  It cannot be NULL.

Remarks

The yylexcleanup function frees any memory that may have been allocated through calls to the yysettextsize and yysetunputsize functions.  These functions are either called directly by yourself, or indirectly by the lexical analyser if it ever needs to grow one of the buffers.  Do not call this function while the lexical analyser is running.

The yymlexcleanup function is the multiple lexical analyser equivalent of the yylexcleanup function.  A number of the functions mentioned above also have multiple lexical analyser equivalents:

Function	Multiple Lexical Analyser Equivalent Function

yysettextsize	yymsettextsize

yysetunputsize	yymsetunputsize

The yymlexcleanup function will not free any memory allocated by the yymcreatelex function.  In order to do this you will need to call the yymdestroylex function.  Note that this function in turn calls yymlexcleanup as part of its operation.

See Also

yysettextsize, yysetunputsize, yymsettextsize, yymsetunputsize, yymlex_t


yylexinit, yymlexinit

#include <lex.h>

void yylexinit( void )

#include <mlex.h>

void yymlexinit( yymlex_t *yy )

Parameter	Description

yy	A pointer to the lexical analyser.  This parameter is only used by the multiple lexical analyser functions.  It cannot be NULL.

Remarks

The yylexinit function initialises the streams used by the lexical analyser.  Usually, this will be done automatically.  There are however instances where this does not happen.  In Visual C++ and Borland C++Builder it is possible to have the run-time libraries in a DLL.  In such circumstances it is not possible to initialise a global variable with one of the standard streams.  The assignment needs to take place inside a function.

When a stream used by the lexical analyser cannot be initialised with one of the standard streams then it is initialised to NULL.  The debug version of the YACC and Lex library checks for NULL stream pointers whenever it performs any operations on them.  The default values for the lexical analyser streams are shown below:

Stream	Default Value

yyin	stdin

yyout	stdout

yylexerr	stderr

yylexdebugout	stdout

As an alternative to calling yylexinit, you can simply initialise the streams yourself.

The yymlexinit function is the multiple lexical analyser equivalent of the yylexinit function.  Instead of initialising the streams mentioned above, the yymlexinit function initialises their multiple lexical analyser equivalents:

Stream	Multiple Parser Equivalent

yyin	yymin member of the yymlex_t structure.

yyout	yymout member of the yymlex_t structure.

yylexerr	yymerr member of the yymlex_t structure.

yylexdebugout	yymdebugout member of the yymlex_t structure.

See Also

yyin, yyout, yylexerr, yylexdebugout, yymlex_t


yyreset, yymreset

#include <lex.h>

void yyreset( void )

#include <mlex.h>

void yymreset( yymlex_t *yy )

Parameter	Description

yy	A pointer to the lexical analyser.  This parameter is only used by the multiple lexical analyser function.  It cannot be NULL.

Remarks

The yyreset function resets the lexical analyser’s variables back to their default values.  It effectively flushes all the buffers, sets the line count back to 1, and ensures that the states are set up correctly ready for a new stream of input.  Note that this function does not have any impact on the sizes of the text and unput buffers.  To reset these back to their initial sizes you will need to call the yylexcleanup function.

It is only necessary to call this function if you are going to re-use the lexical analyser.  Note that the variables start off initialised.  It is however your responsibility to make sure that the input stream has been set-up correctly ready for the lexical analyser to receive new characters.

The yymreset function is the multiple lexical analyser equivalent of the yyreset function.  In the multiple lexical analysers case, the variables mentioned above correspond to members of the yymlex_t structure.  The yymlexcleanup function is the multiple lexical analyser of the yylexcleanup function.

See Also

yylexcleanup, yywrap, yymlexcleanup, yymwrap, yymlex_t


yysettextsize, yymsettextsize

#include <lex.h>

int yysettextsize( int size )

#include <mlex.h>

int yymsettextsize( yymlex_t *yy, int size )

Parameter	Description

yy	A pointer to the lexical analyser.  This parameter is only used by the multiple lexical analyser function.  It cannot be NULL.

size	The requested size of the lexical analysers text buffer and state buffer.  This must be greater than or equal to zero.  The text buffer always gains one extra element.  This is to cope with the zero terminating character that is appended to all matched strings.

Return Value

Nonzero if the lexical analysers text and state buffers were set to the specified size, otherwise zero in which case the size of the buffers remains unchanged.

Remarks

The yysettextsize function sets the lexical analysers text and state buffers to the specified size.  You can change the size of the buffers while the lexical analyser is running.  Note that if you specify a value that is less than yyleng, then the function will fail (and return zero).  In some circumstances you may want to explicitly set yyleng to zero to ensure that this function succeeds.

This function may allocate memory from the heap if it needs to.  If the specified size is less that or equal to the size of the initial (static) text and state buffers, then it will use these.  By default they are 100 elements in size, unless of course you have redefined YYTEXT_SIZE in your Lex source file.  If it is greater than the initial text and state buffer size then it will allocate memory from the heap.

Note that this function is also used by the lexical analyser when it needs to automatically grow the text and state buffers.

The yymsettextsize function is the multiple lexical analyser equivalent of the yysettextsize function.  The yymleng member of the yymlex_t structure is the multiple parser equivalent of the yyleng variable.  Note that a multiple instance lexical analyser does not have initial (static) text and state buffers.  Instead the initial buffers are allocated from memory by the yymcreatelex function.

See Also

yylexcleanup, yytextoverflow, yysetunputsize, yymlexcleanup, yymtextoverflow, yymsetunputsize, yytextgrow, yyleng, yymlex_t


yysetunputsize, yymsetunputsize

#include <lex.h>

int yysetunputsize( int size )

#include <mlex.h>

int yymsetunputsize( yymlex_t *yy, int size )

Parameter	Description

yy	A pointer to the lexical analyser.  This parameter is only used by the multiple lexical analyser function.  It cannot be NULL.

size	The requested size of the lexical analysers unput buffer.  This must be greater than or equal to zero.

Return Value

Nonzero if the lexical analysers unput buffer was set to the specified size, otherwise zero in which case the size of the buffer remains unchanged.

Remarks

The yysetunputsize function sets the unput buffer to the specified size.  You can change the size of the unput buffer while the lexical analyser is running.  Note that if you specify a value that is less than amount of unput buffer currently in use, then the function will fail (and return zero).

This function may allocate memory from the heap if it needs to.  If the specified size is less that or equal to the size of the initial (static) unput buffer, then it will use this.  By default this is 100 elements in size, unless of course you have redefined YYTEXT_SIZE or YYUNPUT_SIZE in your Lex source file.  If it is greater than the initial unput buffer size then it will allocate memory from the heap.

Note that this function is also used by the lexical analyser when it needs to automatically grow the unput buffer.

The yymsetunputsize function is the multiple lexical analyser equivalent of the yysetunputsize function.  Note that a multiple instance lexical analyser does not have an initial (static) unput buffer.  Instead the initial buffer is allocated from memory by the yymcreatelex function.

See Also

yylexcleanup, yyunputoverflow, yysettextsize, yymlexcleanup, yymunputoverflow, yymsettextsize, yyunputgrow, yymlex_t


Lexical Analyser Service Functions

These functions are called by the lexical analyser to perform some form of service.  It is possible, and indeed desirable to redefine these functions to customise the way the lexical analyser behaves.  For multiple lexical analysers it is also possible to assign a different function to the relevant service function member in the yymlex_t structure.

Function	Description

yygetchar, yymgetchar	Called to get the next character for the lexical analyser.

yyinput, yyminput	Called to get the next, possibly buffered, character for the lexical analyser

yyoutput, yymoutput	Called to output an unmatched character.

yytextoverflow, yymtextoverflow	Called when the text buffer overflows.

yyunput, yymunput	Called to back up a character.

yyunputoverflow, yymunputoverflow	Called when the unput buffer overflows.

yywrap, yymwrap	Called when EOF is encountered by the lexical analyser.


yygetchar, yymgetchar

#include <lex.h>

int yygetchar( void )

#include <mlex.h>

int yymgetchar( yymlex_t *yy )

Parameter	Description

yy	A pointer to the lexical analyser.  This parameter is only used by the multiple lexical analyser function.  It cannot be NULL.

Return Value

The next (unbuffered) input character needed by yyinput or yyminput.  The value must be in the range 0 - 255 or -1 to indicate the end-of-file condition.

Remarks

The yygetchar function is called by the yyinput function to get the next (unbuffered) input character.  The default implementation of yygetchar simply gets a character from the FILE pointed to by the yyin variable:

int yygetchar(void)
{

    return getc(yyin);
}

Redefine yygetchar in your source code if you wish to change how the lexical analyser gets its input.  Note though that it is possible to re-assign a different stream to yyin.  For instance you may want to get input from a character array instead of a file.  The redefined yygetchar function might look something like this.

int yygetchar(void)
{

    if (*p == ‘\0’) {

        return -1;
    }

    return (unsigned char) *p++;
}

Note the cast of the contents of p to an unsigned char.  This eliminates any potential problems if char is signed by default.  Incidentally p is a char pointer.  This must be initialised to point to the start of the character array prior to the start of the lexical analysis.

The yygetchar function may be called by the yyinput function after it has returned a -1 (end-of-file) value.  As such if you do redefine yygetchar, be sure to take this into account.  In the example above, the contents of p are first checked for the end-of-file condition before are pointer incrementing takes place.

The yymgetchar function is the multiple lexical analyser equivalent of the yygetchar function.  The yyminput function is the multiple lexical analyser equivalent of the yyinput function.

The default implementation of yymgetchar is slightly different to that for yygetchar.  Instead of the function getting input from the yyin variable, it gets input from the yymin member of the yymlex_t structure.

If you do redefine yymgetchar in your source code then, by default, all multiple lexical analysers will make use of the new function.  As an alternative, you can define a new function with the same prototype and assign it to the yymgetchar member of the yymlex_t structure.  You can do this either statically or at run time.  To do it statically, you need to define YYMGETCHAR as the function name in your Lex source file.  This needs to be done in the declarations section:

%{

int mygetchar(yymlex_t *yy)
{

    /* do something */
}
#define YYMGETCHAR mygetchar

%}

See Also

yyinput, yyminput, yymlex_t


yyinput
yyinput, yyminput

#include <lex.h>

int yyinput( void )

#include <mlex.h>

int yyminput( yymlex_t *yy )

Parameter	Description

yy	A pointer to the lexical analyser.  This parameter is only used by the multiple lexical analyser function.  It cannot be NULL.

Return Value

The next (buffered) input character needed by the lexical analyser.  The value is in the range 0 - 255 or -1 to indicate the end-of-file condition.

Remarks

The yyinput function is called by the lexical analyser to get the next (buffered) input character.  It gets this character either from the unput buffer or by calling the yygetchar function.

This function is also responsible for incrementing the yylineno variable.  Whenever yyinput returns a newline character \n (either from the unput buffer or by calling the yygetchar function) it increments the variable by 1.

The yyminput function is the multiple lexical analyser equivalent of the yyinput function.  Instead of calling yygetchar, yyminput calls the function pointed to by the yymgetchar member of the yymlex_t structure.  In a similar fashion whenever yyminput returns a newline character the yymlineno member is incremented.

Compatibility

The use of the yygetchar function has reduced the need to redefine the yyinput function.  However for compatibility reasons yyinput can also be defined as the macro input.  Note that the macro does not take any parameters.  In the generated lexical analyser there is a small section of code, similar to that below which then converts this macro into the function yyinput:

#ifdef input

int yyinput(void)

{

    return input();
}

#endif

The same also applies to the yyminput function, although not for multiple instance lexical analysers.

If you do redefine yyminput in your source code then, by default, all multiple lexical analysers will make use of the new function.  As an alternative, you can define a new function with the same prototype and assign it to the yyminput member of the yymlex_t structure.  You can do this either statically or at run time.  To do it statically, you need to define YYMINPUT as the function name in your Lex source file.  This needs to be done in the declarations section:

%{

int myinput(yymlex_t *yy)
{

    /* do something */
}
#define YYMINPUT myinput

%}

See Also

yygetchar, yyunput, yymgetchar, yymunput, yymlex_t


yyoutput, yymoutput

#include <lex.h>

void yyoutput( int ch )

#include <mlex.h>

void yymoutput( yymlex_t *yy, int ch )

Parameter	Description

yy	A pointer to the lexical analyser.  This parameter is only used by the multiple lexical analyser function.  It cannot be NULL.

ch	The unmatched character to output.

Remarks

The yyoutput function is called by the lexical analyser to output any unmatched characters.  The function is also called by the yyecho function to output the matched string.  The default implementation of yyoutput simply writes the character to the FILE pointed to by the yyout variable:

void yyoutput(int ch)
{

    putc(ch, yyout)
}

Redefine yyoutput in your source code if you wish to change how the lexical analyser outputs unmatched characters.  Note though that it is possible to re-assign a different stream to yyout.

The yymoutput function is the multiple lexical analyser equivalent of the yyoutput function.  The yymecho function is the multiple lexical analyser equivalent of the yyecho function.

If you do redefine yymoutput in your source code then, by default, all multiple lexical analysers will make use of the new function.  As an alternative, you can define a new function with the same prototype and assign it to the yymoutput member of the yymlex_t structure.  You can do this either statically or at run time.  To do it statically, you need to define YYMOUTPUT as the function name in your Lex source file.  This needs to be done in the declarations section:

%{

void myoutput(yymlex_t *yy, int ch)
{

    /* do something */
}
#define YYMOUTPUT myoutput

%}

Compatibility

For compatibility reasons yyoutput can also be defined as the macro output.  In the generated lexical analyser there is a small section of code, similar to that below which then converts this macro into the function yyoutput:

#ifdef output

void yyoutput(int ch)

{

    output(ch);
}

#endif

The same also applies to the yymoutput function, although not for multiple instance lexical analysers.

See Also

yyinput, yyunput


yytextoverflow, yymtextoverflow

#include <lex.h>

void yytextoverflow( void )

#include <mlex.h>

void yymtextoverflow( yymlex_t *yy )

Parameter	Description

yy	A pointer to the lexical analyser.  This parameter is only used by the multiple lexical analyser function.  It cannot be NULL.

Remarks

The yytextoverflow function is called if the lexical analysers text buffer yytext ever overflows. Note that this also includes the accompanying state buffer.  This can be for one of three reasons:

1	The text buffer is not growable, and has simply overflowed.

2	The text buffer is growable, however the memory block requested by the lexical analyser is simply too big to fit in an internal representation.  For instance, in 16-bit DOS applications, the largest block of memory that can be allocated is 65,535 bytes in size.  Note also that when the text buffer grows, the accompanying state buffer also grows.  For a 16-bit application, a state is 2 bytes in size.

3	The text buffer is growable, however the computer has run out of memory. Note this is not very likely on modern machines.

The default implementation of yytextoverflow simply outputs the error message "lex text buffer overflow (size)\n", where size is the current size of the buffer.  The error message is sent to the FILE pointed to by the yylexerr variable.  Redefine this function in your source code if you wish to handle text buffer overflow reporting differently.  Note that when yytextoverflow returns, the calling lexical analyser function (either yylex or yyclex) will call the standard C library function exit immediately with the exit code EXIT_FAILURE.

The yymtextoverflow function is the multiple lexical analyser equivalent of the yytextoverflow function.  Instead of the yytext buffer overflowing the buffer pointed to by the yymtext member of the yymlex_t structure overflows.  Similarly, in the default implementation of yymtextoverflow, the error message is send to the FILE pointed to by the yymerr member.

The yymlex function is the multiple lexical analyser equivalent of the yylex function, and the yymclex function is the multiple lexical analyser equivalent of the yyclex function.

If you do redefine yymtextoverflow in your source code then, by default, all multiple lexical analysers will make use of the new function.  As an alternative, you can define a new function with the same prototype and assign it to the yymtextoverflow member of the yymlex_t structure.  You can do this either statically or at run time.  To do it statically, you need to define YYMTEXTOVERFLOW as the function name in your Lex source file.  This needs to be done in the declarations section:

%{

void mytextoverflow(yymlex_t *yy)
{

    /* do something */
}
#define YYMTEXTOVERFLOW mytextoverflow

%}

See Also

yyunputoverflow, yymunputoverflow, yytextgrow, yymlex_t


yyunput, yymunput

#include <lex.h>

void yyunput( int ch )

#include <mlex.h>

void yymunput( yymlex_t *yy, int ch )

Parameter	Description

yy	A pointer to the lexical analyser.  This parameter is only used by the multiple lexical analyser function.  It cannot be NULL.

ch	The character to place in the unput buffer.  The character must be in the range 0-255.

Remarks

The yyunput function is called by the lexical analyser whenever it needs to backup a character.  This can occur if the lexical analyser has read too many characters when trying to match a regular expression.  If the function cannot backup the character then it will call the yyunputoverflow function, followed by the standard C library function exit with EXIT_FAILURE as its argument.  Note that this is unlikely to happen since, by default yyunputgrow is set to one, enabling the unput buffer to grow if it needs to.

The initial size of the unput buffer can be specified in the Lex source file by defining YYUNPUT_SIZE in the declarations section.  If you do not define YYUNPUT_SIZE, then it assumes the same value as that defined for YYTEXT_SIZE.  This in turn has a default value of 100.  Note that YYTEXT_SIZE specifies the initial size of the input buffer yytext.  The example below specifies an initial unput buffer 1000 characters in size:

%{

#define YYUNPUT_SIZE 1000

%}

This function is also responsible for decrementing the yylineno variable.  Whenever yyunput receives a newline character \n it decrements the variable by 1.

The yymunput function is the multiple lexical analyser equivalent of the yyunput function.  The yymunputoverflow function is the multiple lexical analyser equivalent of the yyunputoverflow function.  A number of the variables mentioned above also have multiple lexical analyser equivalents:

Variable	Multiple Lexical Analyser Equivalent

yyunputgrow	yymunputgrow member of the yymlex_t structure.

yytext	yymtext member of the yymlex_t structure.

yylineno	yymlineno member of the yymlex_t structure.

Compatibility

It is not common to redefine the yyunput function.  For compatibility reasons though, yyunput can also be defined as the macro unput.  In the generated lexical analyser there is a small section of code, similar to that below which then converts this macro into the function yyunput:

#ifdef unput

void yyunput(int ch)
{

    unput(ch);
}

#endif

The same also applies to the yymunput function, although not for multiple instance lexical analysers.

If you do redefine yymunput in your source code then, by default, all multiple lexical analysers will make use of the new function.  As an alternative, you can define a new function with the same prototype and assign it to the yymunput member of the yymlex_t structure.  You can do this either statically or at run time.  To do it statically, you need to define YYMUNPUT as the function name in your Lex source file.  This needs to be done in the declarations section:

%{

void myunput(yymlex_t *yy, int ch)
{

    /* do something */
}
#define YYMUNPUT myunput

%}

See Also

yyinput, yyoutput, yyminput, yymoutput, yyunputgrow, yymlex_t


yyunputoverflow, yymunputoverflow

#include <lex.h>

void yyunputoverflow( void )

#include <mlex.h>

void yymunputoverflow( yymlex_t *yy )

Parameter	Description

yy	A pointer to the lexical analyser.  This parameter is only used by the multiple lexical analyser function.  It cannot be NULL.

Remarks

This yyunputoverflow function is called if the lexical analysers unput buffer ever overflows.  This can be for one of three reasons:

1	The unput buffer is not growable, and has simply overflowed.

2	The unput buffer is growable, however the memory block requested by the lexical analyser is simply too big to fit in an internal representation.  For instance, in 16-bit DOS applications, the largest block of memory that can be allocated is 65,535 bytes in size.

3	The unput buffer is growable, however the computer has run out of memory. Note this is not very likely on modern machines.

The default implementation of yyunputoverflow simply outputs the error message "lex unput buffer overflow (size)\n", where size is the current size of the buffer.  The error message is sent to the FILE pointed to by the yylexerr variable.  Redefine this function in your source code if you wish to handle unput buffer overflow reporting differently.  Note that when yyunputoverflow returns, the calling lexical analyser function (either yylex or yyclex) will call the standard C library function exit immediately with the exit code EXIT_FAILURE.

The yymunputoverflow function is the multiple lexical analyser equivalent of the yyunputoverflow function.  In the default implementation of yymunputoverflow the error message is sent to the FILE pointed to by the yymerr member of the yymlex_t structure.

The yymlex function is the multiple lexical analyser equivalent of the yylex function, and the yymclex function is the multiple lexical analyser equivalent of the yyclex function.

If you do redefine yymunputoverflow in your source code then, by default, all multiple lexical analysers will make use of the new function.  As an alternative, you can define a new function with the same prototype and assign it to the yymunputoverflow member of the yymlex_t structure.  You can do this either statically or at run time.  To do it statically, you need to define YYMUNOUTOVERFLOW as the function name in your Lex source file.  This needs to be done in the declarations section:

%{

void myunputoverflow(yymlex_t *yy)
{

    /* do something */
}
#define YYMUNOUTOVERFLOW myunputoverflow

%}

See Also

yytextoverflow, yymtextoverflow, yyunputgrow, yymlex_t


yywrap, yymwrap

#include <lex.h>

int yywrap( void )

#include <mlex.h>

int yymwrap( yymlex_t *yy )

Parameter	Description

yy	A pointer to the lexical analyser.  This parameter is only used by the multiple lexical analyser function.  It cannot be NULL.

Return Value

Nonzero if there is no more input for the lexical analyser to process, otherwise zero.

Remarks

The yywrap function is called by the lexical analyser whenever it inputs an EOF as the first character when trying to match a regular expression.  The function returns a nonzero value to indicate that there is no more input for the lexical analyser to process, otherwise zero.  The default implementation of yywrap simply returns 1, i.e. there is no more input to process:

int yywrap(void)
{

    return 1;
}

Redefine yywrap if you wish to return a value of zero and process more input.  Note that it is your responsibility to ensure that this new input is available for the lexical analyser.  For instance, this may involve assigning a new stream to the yyin variable.

The processing of more input is not the only reason for overriding yywrap.  For instance you can use the function to generate lexical analysis results.  Taking a simple example, a word count program, you can use the yywrap function to print out the number of words that have been counted:

int yywrap(void)
{

    printf(“word count: %d\n”, wc);

    return 1;
}

The yymwrap function is the multiple lexical analyser equivalent of the yywrap function.  For multiple lexical analysers the yyin variable is equivalent to the yymin member of the yymlex_t structure.

If you do redefine yymwrap in your source code then, by default, all multiple lexical analysers will make use of the new function.  As an alternative, you can define a new function with the same prototype and assign it to the yymwrap member of the yymlex_t structure.  You can do this either statically or at run time.  To do it statically, you need to define YYMWRAP as the function name in your Lex source file.  This needs to be done in the declarations section:

%{

int mywrap(yymlex_t *yy)
{

    /* do something */
}
#define YYMWRAP mywrap

%}

See Also

yyinput, yyminput, yymlex_t


Lexical Analyser Action Functions

These functions are typically called from an action to modify the way the lexical analyser behaves.  Indeed in order for the yyreject and yymreject functions to have an effect they must be called from an action.

Function	Description

yybegin, yymbegin	Places the lexical analyser in the requested start state.

yyecho, yymecho	Echoes the matched string.

yyless, yymless	Backs up the matched string until it has the requested length.

yymore, yymmore	Concatenates the next matched string on the end of the next matched string.

yynewline, yymnewline	Specifies whether the previous read character was a newline character or not.

yyreject, yymreject	Forces the lexical analyser to match another different regular expression.

yyunputcount, yymunputcount	Retrieves the number of characters stored in the unput buffer.


yybegin, yymbegin

#include <lex.h>

void yybegin( int state )

#include <mlex.h>

void yymbegin( yymlex_t *yy, int state )

Parameter	Description

yy	A pointer to the lexical analyser.  This parameter is only used by the multiple lexical analyser function.  It cannot be NULL.

state	The new start state to place the lexical analyser in, or zero to return to the default start state.

Remarks

The yybegin function changes the lexical analysers start state to the requested state.  Start states are introduced in a Lex source file through the use of the %start keyword.  As an alternative to passing INITIAL as a parameter to return to the default start state, it is possible to simply pass the value zero.

The yymbegin function is the multiple parser equivalent of the yybegin function.

Compatibility

The function yybegin is also implemented as the macro BEGIN.  The macro does not have any trailing parentheses.  An example of its use is shown below:

BEGIN aa;

See Also

yynewline, yymnewline, yystart, yymlex_t


yyecho, yymecho

#include <lex.h>

void yyecho( void )

#include <mlex.h>

void yymecho( yymlex_t *yy )

Parameter	Description

yy	A pointer to the lexical analyser.  This parameter is only used by the multiple lexical analyser function.  It cannot be NULL.

Remarks

The yyecho function echoes the matched string.  It does this by passing each character in the matched string to the yyoutput function.  This is the function that is called by the lexical analyser whenever it cannot match a character.  The function simply copies the character to the stream specified by the yyout variable.  By default this is assigned to the stdout stream.  To redirect output elsewhere you can either assign a new stream to yyout, or redefine the yyoutput function in your Lex source file.

The yymecho function is the multiple lexical analyser equivalent of the yyecho function.  The yymoutput function is the multiple lexical analyser equivalent of the yyoutput function.  However the yymecho function makes use of the yymoutput member of the yymlex_t structure to echo the string.  This points to the function to be called for that particular lexical analyser.

Compatibility

The function yyecho is also implemented as the macro ECHO.  The macro does not have any trailing parentheses.  An example of its use is shown below:

ECHO;

See Also

yyoutput, yymoutput, yymlex_t


yyless, yymless

#include <lex.h>

void yyless( int length )

#include <mlex.h>

void yymless( yymlex_t *yy, int length )

Parameter	Description

yy	A pointer to the lexical analyser.  This parameter is only used by the multiple lexical analyser function.  It cannot be NULL.

length	The new length of the matched string.  This must be greater than or equal to zero, and less than or equal to the current length of the string.

Remarks

The yyless function changes the length of the matched regular expression string.  The new length must be greater than or equal to zero and less than or equal to the current length of the string.  Any characters that are not wanted are backed up.  They are removed one by one from the end of the string and passed to the yyunput function.

Be careful of setting the length of the string to zero.  Unless you are planning on doing something else as well (such as changing start states) this will cause the lexical analyser to match the same expression forever:

abc     { yyless(0); /* forever */ }

An alternative to using yyless is to use the right context operator /.  The two regular expressions below, with their associated actions both match the same string:

abc/de  { ... }

abcde   { yyless(3); ... }

The choice whether to use the lookahead operator or yyless is one of taste.  It is worth noting that using yyless is slightly faster than using the lookahead operator.  Having said that the operator provides a clearer specification of the string that is actually matched.

The yymless function is the multiple lexical analyser equivalent of the yyless function.  The yymunput function is the multiple parser equivalent of the yyunput function.  However the yymless function makes use of the yymunput member of the yymlex_t structure to backup up the string.  This points to the function to be called for that particular lexical analyser.

See Also

yymore, yymmore, yymlex_t


yymore, yymmore

#include <lex.h>

void yymore( void )

#include <mlex.h>

void yymmore( yymlex_t *yy )

Parameter	Description

yy	A pointer to the lexical analyser.  This parameter is only used by the multiple lexical analyser function.  It cannot be NULL.

Remarks

The yymore function causes the next matched string to be concatenated on the end of the currently matched string.  The resultant string is stored in the yytext variable.  Note also that the yyleng variable holds the combined length of the two strings.

The yymmore function is the multiple lexical analyser equivalent of the yymore function.  For the multiple lexical analyser, the resultant string is stored in the yymtext member of the yymlex_t structure.  Similarly, the yymleng member holds the combined length of the two strings.

See Also

yyless, yymless, yymlex_t


yynewline, yymnewline

#include <lex.h>

void yynewline( int newline )

#include <mlex.h>

void yymnewline( yymlex_t *yy, int newline )

Parameter	Description

yy	A pointer to the lexical analyser.  This parameter is only used by the multiple lexical analyser function.  It cannot be NULL.

newline	Nonzero if the previous character read by the lexical analyser was a newline character, otherwise zero.

Remarks

The yynewline function specifies whether the last character read by the lexical analyser was a newline character or not.  The lexical analyser uses this value when it tries to match the next string.  Regular expressions starting with the leading newline operator ^ will be preferred over those which don't.

Ordinarily, the lexical analyser decides whether the last character read was a newline character or not.  It does this after a string has been matched, but before any action associated with the regular expression has been called.  This is to enable the use of the yynewline function in an action.

The yyless function destroys the result of any call to the yynewline function, so if you call both functions in an action, make sure that the yynewline call goes after the yyless call.

The yymnewline function is the multiple lexical analyser equivalent of the yynewline function.  The yymless function is the multiple parser equivalent of the yyless function.

See Also

yybegin, yymbegin, yymlex_t


yyreject, yymreject

#include <lex.h>

void yyreject( void )

#include <mlex.h>

void yymreject( yymlex_t *yy )

Parameter	Description

yy	A pointer to the lexical analyser.  This parameter is only used by the multiple lexical analyser function.  It cannot be NULL.

Remarks

The yyreject function forces the lexical analyser to re-scan the input looking for another regular expression to match.  The function has some limitations on where it can be used, however these are not generally restrictive.  If the function is called from elsewhere then it will have no effect.  The yyreject function can be:

1	Called from an action.

2	Called from some function, but program flow of control must return to an action at some point.

The yyreject function does not take effect immediately, but rather when control returns from the action to the lexical analyser.  If a return statement is executed in an action, then the results of any call to the yyreject function will be lost.

The yyreject function does not cause the input to be re-scanned as such, but rather the result of the previous match to be re-examined.  The result of any re-scanning also depends upon whether the -r option has been specified for ALex or not.  If so then it is possible for the same string to be matched by more than one regular expression.  The regular expressions following the order in which they were defined in the Lex source file.  Otherwise the length of the next matched string will always be less than the current one.

If a string of length one has been matched, and any other regular expressions matched if the -r option has been specified, then the character will be matched by the lexical analysers default rule.  This will simply call yyoutput with the character in question.

An example best illustrates how the yyreject function behaves.  The example below matches the string "fu", and one or more letters.  In each action there is a call to the yyreject function:

%%

"fu"        { yyreject(); }

[a-zA-Z]+   { yyreject(); }

If the character stream "fubar" is now input to the resultant lexical analyser, and the -r option has been specified, the following strings are matched:

"fubar", "fuba", "fub", "fu", "fu", "f", "f", "ubar", "uba", "ub", "u", "u", "bar", "ba", "b", "b", "ar", "a", "a", "r", "r"

Notice that the string "fu" is matched twice in succession.  In the first case it is matched by the rule for "fu", and in the second case the rule for one or more letters.  The characters "f", "u", "b" etc. are also matched twice in succession.  However here the reasons are different.  In the first case they are matched by the rule for one or more letters, and in the second case the default rule.

If the -r option is not specified, then the matched strings are similar, however there is one important difference.  The string "fu" is only matched once:

"fubar", "fuba", "fub", "fu", "f", "f", "ubar", "uba", "ub", "u", "u", "bar", "ba", "b", "b", "ar", "a", "a", "r", "r"

If, on the other hand, there are no calls to the yyreject function, then the following strings are matched:

"fu", "bar"

The yymreject function is the multiple lexical analyser equivalent of the yyreject function.  The yymoutput function is the multiple parser equivalent of the yyoutput function.  However the default rule makes use of the yymoutput member of the yymlex_t structure to output the unmatched character.  This points to the function to be called for that particular lexical analyser.

Compatibility

The function yyreject is also implemented as the macro REJECT.  The macro does not have any trailing parentheses.  An example of its use is shown below:

"fu"        { REJECT; }

In traditional Lex, this macro will cause an immediate jump to the relevant reject handling code.  This means that no code following the REJECT macro will be executed.  In traditional YACC, due to implementation details, this macro can only be placed in an action itself.  It cannot be placed in a function called from an action.  Note that this does not apply to ALex, where REJECT can be placed anywhere that yyreject can.

See Also

yymlex_t


yyunputcount, yymunputcount

#include <lex.h>

int yyunputcount( void )

#include <mlex.h>

int yymunputcount( yymlex_t *yy )

Parameter	Description

yy	A pointer to the lexical analyser.  This parameter is only used by the multiple lexical analyser function.  It cannot be NULL.

Return Value

The number of characters stored in the unput buffer.

Remarks

The yyunputcount function returns the number of characters stored in the unput buffer.  The unput buffer is used to hold any excess characters that the lexical analyser may have read whilst trying to match a regular expression.

The yymunputcount function is the multiple lexical analyser equivalent of the yyunputcount function.

See Also

yyinput, yyunput, yyminput, yymunput, yymlex_t


Classes

These are the base classes that are used by the parsers and lexical analysers generated by AYACC and ALex.  The two main base classes are yyparser and yylexer, and hold most of the parser and lexical analyser code.  Derived from yyparser are yyfparser and yycparser.  These two classes have different implementations of the yywork pure virtual function, and the necessary table member variables to drive them.  Similarly, yyflexer and yyclexer are derived from yylexer.  In these cases the pure virtual function is yylex.

Class	Description

yyparser	Abstract base class for all parsers.

yyfparser	Abstract base class for a default (fast) parser.

yycparser	Abstract base class for a compact parser.

yylexer	Abstract base class for all lexical analysers.

yyflexer	Abstract base class for a default (fast) lexical analyser.

yyclexer	Abstract base class for a compact lexical analyser.


yyparser

#include <cyacc.h>

class yyparser;

Remarks

The yyparser class provides the basic functionality for all C++ parsers generated by AYACC.  It has no base class.  The yyparser class is an abstract class so you cannot create an object from this class directly.  It must be from a derived class.  The pure virtual functions which make yyparser an abstract class are yywork and yyparseaction.

The yyparser class has two derived classes, yyfparser and yycparser.  Both of these classes are abstract classes too.  The yyfparser class implements the functionality for a default (fast) parser, and the yycparser class implements the functionality for a compact parser.  The lexical analyser class generated by AYACC will be derived from one of these classes, depending upon the options you have selected.  Note though that you need to provide additional information to YACC in order for it to be able to do this.  For instance you must provide, at the very least, a name for the new class.  This achieved by using a Name Declaration.  Basically, all you need to place in the Declarations Section of your YACC source file, is the keyword %name, followed by the name of the derived class.  For example, the name declaration below will declare the new derived class myparser:

%name myparser

This will then place the relevant class definition in the generated include (or output) file, and generate a default constructor for the class.  You can also use name declarations to place additional member functions and member variables in the new class definition, and initialisation code in the constructor.  It is possible, if you so wish, to duplicate the effects of a name declaration by placing the necessary, equivalent code, in the include and output files using Include Declarations and Code Blocks.  For more information see also Defines.

See Also

yyparser Members, yyfparser, yycparser


yyparser Members

These are the commonly used member variables and functions in the yyparser class.  For a complete list of available member variables and functions for the class look in the CYACC.H header file.

Member Variables

yychar	The current lookahead token.

yydebug	Whether debug information should be output.

yydebugflush	Indicates whether debug output should be flushed when it is written.

yydebugout	The stream to which the parser sends debug output.

yydebugstack	Indicates whether parser stack contents should be output along with other debug information.

yyerr	The stream to which the parser sends error messages.

yyerrorcount	The number of syntax errors which have occurred.

yylookahead	Whether the current lookahead token is valid.

yystackgrow	Specifies whether the stack should grow if it ever overflows.

yywipeflg	Specifies whether the symbols used by the parser should be cleaned up if the parse is aborted.

Construction Member Functions

yyparser	Constructs a yyparser object.

yycreate	Creates the parser, allocating any memory where necessary.

yydestroy	Destroys the parser, freeing any allocated memory.

General Member Functions

yycleanup	Frees any memory that may have been allocated through changing the stack.

yydestructpop	Pops the requested number of states off of the stack, calling any symbol destructor actions where necessary.

yyparse	Initiates a parse for a parser (either fast or compact).

yysetstacksize	Sets the size of the parser's stack.

yysetup	Sets up the parser prior to doing the main body of work.

yywipe	Cleans up any symbols left on the stack and the lookahead token (either fast or compact).

yywork	Performs the main body of work for a parser (either fast or compact).

Service Member Functions

yydiscard	Called when the current lookahead token is discarded during error recovery.

yyerror	Called when an error, either syntax or a stack overflow occurs.

yygettoken	Called by the parser when it needs a new token.

yystackoverflow	Called when the parsers stack overflows.

yysyntaxerror	Called when a syntax error occurs.

Action Member Functions

yyabort	Forces the parser to return with the YYEXIT_FAILURE exit code.

yyaccept	Forces the parser to return with the YYEXIT_SUCCESS exit code.

yyclearin	Clears the current lookahead token.

yydestructclearin	Clears the current lookahead token, calling a symbol destructor for the token if necessary (either fast or compact).

yyerrok	Re-synchronises error handling.

yyexit	Forces the parser to return with an exit code.

yyforceerror	Forces the parser to generate a syntax error.

yypopping	Indicates whether the parser has popped any states off of the stack whilst looking for an error token transition.

yyrecovering	Indicates whether the parser is in error recovery.

yyretire	Forces the parser to return with an exit code after first reducing the current rule.

yysetin	Sets the current lookahead token to the specified token.

yythrowerror	Forces the parser to generate a syntax error.  The parser will then skip the specified number of error token transitions before attempting recovery

yyunclearin	Returns the lookahead token to its previous value if it is currently clear.

See Also

yyparser


yyparser::yychar

#include <cyacc.h>

unsigned char yychar;

Remarks

The yychar member variable holds the current lookahead token for the parser.  The parser does not always need a lookahead token, and sometimes this member will be invalid.  Before using yychar check the state of the yylookahead member variable first.  If yylookahead is nonzero then yychar holds a valid token.

See Also

yyparser::yysetin, yyparser::yyclearin, yyparser::yyunclearin, yyparser::yylookahead


yyparser::yydebug

#include <cyacc.h>

int yydebug;

Remarks

The yydebug member variable specifies whether debug information should be output by the parser while it is running.  If yydebug is nonzero then debug information is generated.  By default yydebug is set to zero, i.e. no debugging information is generated.  This member is only defined for the debug version of the YACC and Lex library.

See Also

yyparser::yydebugout, yyparser::yydebugstack, yydebug, yydebugstack


yyparser::yydebugflush

#include <cyacc.h>

int yydebugflush;

Remarks

The yydebugflush member variable specifies whether the debug output stream should be flushed whenever debug information is written to it.  If yydebugflush is nonzero then the steam will be flushed.  By default yydebugflush is set to zero i.e. the stream will not be flushed.  Flushing the debug stream is useful if the application which is being debugged crashes.  In such circumstances, the file written to, assuming it is a file, will contain all of the debug information.  The downside though is that flushing the stream slows down the application significantly.  This member is only defined for the debug version of the YACC and Lex library.

See Also

yyparser::yydebugout, yyparser::yydebug, yydebug, yydebugflush


yyparser::yydebugout

#include <cyacc.h>

FILE* yydebugout;

Remarks

The yydebugout member variable specifies the output stream to which the parser sends its debugging information.  This member is initialised to the standard output stream stdout by default.  You can assign a new value to yydebugout in your source code.  This member is only defined for the debug version of the YACC and Lex library.

In 32-bit Windows versions of the YACC and Lex library, if yydebugout is set to NULL, then debugging information is sent to the debugger.  If the application containing the parser has an associated debugger, then the debug information is sent here.  Failing that, if the system debugger is active then debug information is sent here.  Otherwise any debug information is simply lost.

See Also

yyparser::yydebug, yyparser::yydebugflush, yydebug, yydebugflush


yyparser::yydebugstack

#include <cyacc.h>

int yydebugstack;

Remarks

The yydebugstack member variable specifies whether the contents of the stack should be output along with other debug information.  If yydebugstack is nonzero then the stack contents will be output.  By default yydebugstack is set to zero i.e. stack contents will not be output.  This member is only defined for the debug version of the YACC and Lex library.

When enabled, the contents of the stack are output every time after a state is pushed onto the stack.  In practice this means every rime after a shift or reduce action takes place.  Note that this extra debug information can significantly increase the size of the generated output.

See Also

yyparser::yydebug, yydebugstack, yydebug


yyparser::yyerr

#include <cyacc.h>

FILE* yyerr;

Remarks

The yyerr member variable specifies the output stream to which the yyerror member function sends any error messages.  This variable is initialised to the standard error stream stderr by default.  You can assign a new value to yyerr in your source code.

See Also

yyparser::yyerror


yyparser::yyerrorcount

#include <cyacc.h>

int yyerrorcount;

Remarks

The yyerrorcount member variable holds the number of syntax errors which have occurred since the start of the current parse.  The member is reset to zero at the start of each parse.  It is incremented for one of two reasons:

1	A syntax error is encountered by the parser (that is the parser can neither shift a token onto the stack, reduce a rule or accept the input) and the parser is not currently in error recovery mode.

2	A syntax error is generated when the yythrowerror or yyforceerror member functions are called from an action.  Note that the error is generated only when control returns to the parser and no other higher priority function, i.e. yyexit has been called.  In this case whether the parser is in error recovery mode or not does not matter.

See Also

yyparser::yysyntaxerror


yyparser::yylookahead

#include <cyacc.h>

unsigned char yylookahead;

Remarks

The yylookahead member variable specifies whether the current lookahead token, held in the yychar variable, is valid.  If yylookahead is nonzero then yychar holds a valid token.

See Also

yyparser::yysetin, yyparser::yyclearin, yyparser::yyunclearin, yyparser::yydestructclearin, yyparser::yywipe, yyparser::yychar


yyparser::yystackgrow

#include <cyacc.h>

unsigned char yystackgrow;

Remarks

The yystackgrow member variable specifies whether the parser's stack should grow if it ever overflows.  If yystackgrow is nonzero then the stack is allowed to grow.  The initial size of the stack is 100 elements unless you have defined YYSTACK_SIZE in your YACC source file.  The example below defines an initial stack with 1000 elements:

#define YYSTACK_SIZE 1000

By default yystackgrow is set 1 allowing the stack to grow.

If the stack overflows and it is allowed to grow, the parser will double the size of the stack.  If the stack has zero elements (possible if you define YYSTACK_SIZE as 0), then the parser will allocate a 100 element stack instead.  All stack changes are achieved through calls to the yysetstacksize member function.

If the stack overflows and it is not allowed to grow, then the parser calls the yystackoverflow member function before returning with YYEXIT_FAILURE.

See Also

yyparser::yysetstacksize, yyparser::yystackoverflow


yyparser::yywipeflg

#include <cyacc.h>

unsigned char yywipeflg;

Remarks

The yywipeflg member variable specifies whether the symbols in use by the parser should be cleaned up when, or if, the parser aborts.  If yywipeflg is nonzero then the symbols will be cleaned up.  By default this variable is set to 1 i.e. the symbols currently in use by the parser will be cleaned up.

The parser can abort for one of two reasons.  Either the stack overflows, or an untrapped syntax error is encountered.  In such cases, if symbol clean up is enabled, all symbols (states) will be popped off of the stack, and the current lookahead token cleared.  If any symbol which is discarded has a symbol destructor associated with it, then the destructor will be called for the symbol in question.

In practice, symbol cleanup is achieved through a call to the yywipe member function.

See Also

yyparser::yywork, yyparser_yywipe


yyparser::yyparser

#include <cyacc.h>

yyparser( );

Remarks

Constructs a yyparser object.  The parser will not be ready for use until the yycreate member function has been called.

See Also

yyparser::yycreate


yyparser::yycreate

#include <cyacc.h>

int yycreate( yylexer* lexerptr = NULL );

Parameter	Description

lexerptr	A pointer to the lexical analyser object associated with the parser, or NULL if there is none.

Return Value

Nonzero if successful, otherwise zero.  Failure occurs if the necessary memory for the initial state and attribute stacks cannot be allocated.

Remarks

The yycreate member function creates the parser.  This function must be called before the parser can be used.  The function needs to allocate memory from the heap for the initial state and attribute stacks, and for the two attribute variables yyval and yylval. If the yycreate member function cannot allocate the necessary memory then the function fails and returns zero.

The lexerptr parameter passed to the yycreate member function is assigned to the yylexerptr member variable.  Unless overridden, this member variable is assumed to point to a valid yylexer object by the yygettoken member function.

In a C++ parser, you can only get access to the attribute variables yyval and yylval by using the yyvalptr and yylvalptr member variables.  Note that the yylvalptr member variable points to the attribute associated with a token whenever it is shifted onto the stack.  In order to access the attribute you must first cast it to a pointer of the required type, and then assign it to a reference variable:

YYSTYPE& yylval = *(YYSTYPE*)yylvalptr;

yylval = 1;

The parser is destroyed by calling the yydestroy member function.

An example of how a parser is created, used and destroyed is shown below.  The example also includes the creation and destruction of a yylexer object.  Note that the created parser and lexical analyser are both passed pointers to each other:

#include "myparser.h"

#include "mylexer.h"

int parse()
{

    int n = 1;

    myparser parser;

    mylexer lexer;

    if (parser.yycreate(&lexer)) {

        if (lexer.yycreate(&parser)) {

            n = parser.yyparse();

            lexer.yydestroy();

        }
        parser.yydestroy();

    }

    return n;
}

The parser class myparser is defined in the include file "myparser.h".  The lexical analyser class mylexer is defined in the include file "mylexer.h".  The file "myparser.h" would have been generated by AYACC, and "mylexer.h" by ALex.  Note that the calls to the yydestroy and yylexer::destroy member functions are not really necessary, since they will be called by the destructors anyway.

See Also

yyparser::yyparser, yyparser::yydestroy


yyparser::yydestroy

#include <cyacc.h>

void yydestroy( );

Remarks

The yydestroy member function destroys the parser.  A parser is usually created first by a call to the yycreate member function.  The yydestroy member function frees up any memory allocated to the state and attribute stacks and the yyval and yylval variables.  The function does not attempt to free up the memory associated with the yyparser object itself.  For an example of how parsers are created and destroyed see yycreate.

The yydestroy member function frees the memory allocated to the parser by first calling the yycleanup member function.  This frees any memory allocated through calls to the yysetstacksize function member.  The function then frees the initial state and attribute stacks and the yyval and yylval variables allocated by the yycreate member function.

This member function is also called by the yyparser destructor as part of clean up.  As such, this member function can be called multiple times, and it is not necessary to call yycreate beforehand.

See Also

yyparser::yycreate


yyparser::yycleanup

#include <cyacc.h>

void yycleanup( );

Remarks

The yycleanup member function frees any memory that may have been allocated through calls to the yysetstacksize function.  This function is either called directly by yourself, or indirectly by the parser if it ever needs to grow the stack.  Do not call this function while the parser is running.

The yycleanup member function will not free any memory allocated by the yycreate function.  In order to do this you will need to call the yydestroy function.  Note that this function in turn calls yycleanup as part of its operation.

See Also

yyparser::yysetstacksize


yyparser::yydestructpop

#include <cyacc.h>

void yydestructpop( int num );

Parameter	Description

num	The number of states to pop off of the stack.  This must be greater than or equal to zero.

Remarks

The yydestructpop member function pops the requested number of states off of the stack.  No checks are made on how many states are popped off of the stack so beware of causing a stack underflow.  For each nonterminal popped off of the stack, its destructor is called.  The destructor will typically free up any memory (like a node tree) that may be used by the nonterminal.  If a destructor has not been defined for a nonterminal then no destructor will be called.  Note that a state maps to either a nonterminal symbol or a token.

See Also

yyparser::yyretire


yyparse::yyparse
yyparser::yyparse

#include <cyacc.h>

int yyparse( );

Return Value

Whether the parse succeeded or not, or a user defined value.  YYEXIT_SUCCESS is returned on success and YYEXIT_FAILURE on failure.

Remarks

The yyparse member function initiates a parse for the specified parser.  The function does not do much in itself.  Instead it calls other functions to do the work for it.  The function first calls yysetup to set-up the parser and push the initial state onto the stack.  It then calls the yywork member function which performs the main body of work for the parser.

See Also

yyparser::yysetup, yyparser::yywork


yyparser::yysetstacksize

#include <cyacc.h>

int yysetstacksize( int size );

Parameter	Description

size	The requested size of the parsers stack.  This must be greater than or equal to zero.

Return Value

Nonzero if the parsers stack was set to the specified size, otherwise zero in which case the size of the stack remains unchanged.

Remarks

The yysetstacksize member function sets the parsers stack to the specified size.  Note that both the (state) stack and the attribute stack are changed.  You can change the size of the stack while the parser is running.  Note that if you specify a size that is less than the amount of stack currently in use by the parser, then the function will fail (and return zero).

This function may allocate memory from the heap if it needs to.  If the specified size is less that or equal to the size of the initial stack, then it will use this.  By default this is 100 elements in size, unless of course you have redefined YYSTACK_SIZE in your YACC source file.  If it is greater than the initial stack size then it will allocate memory from the heap.

Note that this function is also used by the parser when it needs to automatically grow the stack.

See Also

yyparser::yycleanup, yyparser::yystackgrow


yyparser::yysetup

#include <cyacc.h>

int yysetup( );

Return Value

Zero if the parser has been set-up correctly ready for parsing, otherwise nonzero.

Remarks

The yysetup member function is called by the yyparse member function to initialise the parser's member variables and push the initial state onto the parser's stack.  The function fails if it needs to allocate memory for the stack and it is unable to do so.

You can use this function along with the yywork member function as an alternative to using yyparse.  You first need to call yysetup followed by yywork.

See Also

yyparser::yyparse, yyparser::yywork, yyparser::yyretire


yyparser::yywipe

#include <cyacc.h>

void yywipe( );

Remarks

The yywipe member function cleans up any nonterminals that are left on the stack.  This can happen if you use the yyexit, yyaccept or yyabort member functions or indeed even the yyretire member function.  For each nonterminal on the stack, its destructor is called.  The destructor will typically free up any memory (like a node tree) that may be used by the nonterminal.  If a destructor has not been defined for a nonterminal then no destructor will be called.  Note that although the stack itself contains states, each state maps to either a nonterminal symbol or a token.

See Also

yyparser::yyexit, yyparser::yyaccept, yyparser::yyabort, yyparser::yyretire


yyparser::yywork

#include <cyacc.h>

virtual int yywork( ) = 0;

Return Value

Whether the parse succeeded or not, or a user defined value.  YYEXIT_SUCCESS is returned on success and YYEXIT_FAILURE on failure.

Remarks

The yywork function carries out the main body of work for the parser.  It is specified as a pure virtual function in the yyparser base class.  Proper virtual functions are provided in the yyfparser and yycparser derived classes.  Each one providing the necessary functionality to deal with default (fast) and compact tables respectively.  The yyparse member function makes use of this function when a parse is initiated.

You can use this function along with the yysetup member function as an alternative to using yyparse.  You first need to call yysetup and then yywork.

The yywork member function, and the associated yysetup member function are primarily for use with the yyretire member function.  If you call this function from within an action the parser will finish reducing the rule that the action is associated with and then exit with the supplied exit code.  Note that changes made to the $$ attribute are preserved when the new nonterminal symbol is shifted onto the stack.  You can then resume parsing at a later date with a call to yywork.

Control will remain in yywork until one of the following conditions is met:

1	The parse succeeds.  In which case YYEXIT_SUCCESS is returned.

2	The parse fails.  In which case YYEXIT_FAILURE is returned.  Failure can occur if a syntax error is encountered and there are no error productions to handle it, or there is a stack overflow.

3	An action calls one of the yyexit, yyaccept, yyabort or yyretire member functions.  In which case the exit code passed to the relevant function is returned.

The yywork member functions also calls a number of service functions.  These can be customised to change the way the parser behaves. By overriding the relevant virtual functions.  For a list of the available service functions see yyparser Members.

See Also

yyparser::yysetup, yyparser::yyparse, yyparser::yyretire


yyparser::yydiscard

#include <cyacc.h>

virtual void yydiscard( int token );

Parameter	Description

token	The token which is being discarded.

Remarks

The yydiscard member function is called when the parser is in error recovery and it discards the current lookahead token.  It does this whenever it cannot successfully shift the token onto the stack.  Typically the parser will remain in error recovery until it has shifted three consecutive tokens onto the stack.  Note that error recovery can be completed by use of the yyerrok member function.

The default implementation of yydiscard simply does nothing.  Override this function in your AYACC generated class if you wish to report when tokens are discarded.  For example:

%name myparser { virtual void yydiscard(); }

...

void myparser::yydiscard()
{

    /* do something */

}

Beware of redefining yyparser::yydiscard directly.  Some compilers, Visual C++ for instance, do not permit member functions held in libraries to be redefined.  You always have to derive a new class from the one you want to modify, and then override the relevant member functions where necessary.  Note that this does not apply to C functions, where you can generally redefine any function held in a library that you want to.

See Also

yyparser::yysyntaxerror


yyparser::yyerror

#include <cyacc.h>

virtual void yyerror( const char* text );

Parameter	Description

text	The zero terminated error string.

Remarks

The yyerror member function writes the string text to the stream specified by the yyerr member variable, followed by a newline \n character.  The yyerror function is called by the yysyntaxerror and yystackoverflow member functions when they report an error.  Override this function in your AYACC generated class if you wish to change how errors are reported.  For example:

%name myparser { virtual void yyerror(); }

...

void myparser::yyerror()
{

    /* do something */

}

Beware of redefining yyparser::yyerror directly.  Some compilers, Visual C++ for instance, do not permit member functions held in libraries to be redefined.  You always have to derive a new class from the one you want to modify, and then override the relevant member functions where necessary.  Note that this does not apply to C functions, where you can generally redefine any function held in a library that you want to.

See Also

yyparser::yysyntaxerror, yyparser::yystackoverflow


yyparser::yygettoken

#include <cyacc.h>

virtual int yygettoken( );

Return Value
Zero or negative if there are no more tokens for the parser, otherwise a positive value indicating the next token.

Remarks

The yygettoken member function is called by the parser whenever it needs a new token.  The function returns a zero or negative value if there are no more tokens, otherwise a positive value indicating the next token.  The default implementation simply calls the yylexer::yylex member function of the yylexer object specified by the yylexerptr member variable.  This member is initialised in a call to the yycreate member function.

int yyparser::yygettoken()
{

    return yylexerptr->yylex();
}

Override this function in your AYACC generated class if you wish to change how the parser gets its tokens.  For instance you may want to run some code before calling yylex.  For example:

%name myparser { virtual int yygettoken(); }

...

int myparser::yygettoken()
{

    /* do something */

    return yylexerptr->yylex();
}

Beware of redefining yyparser::yygettoken directly.  Some compilers, Visual C++ for instance, do not permit member functions held in libraries to be redefined.  You always have to derive a new class from the one you want to modify, and then override the relevant member functions where necessary.  Note that this does not apply to C functions, where you can generally redefine any function held in a library that you want to.

The parser is also expecting any attribute associated with the token to be assigned to the contents of the location specified by the yylvalptr member variable.  This member is of type void *, so it is first necessary to cast it to a YYSTYPE * type.  The example below shows a simple way of getting access to the token attribute so that it can be easily modified.

YYSTYPE& yylval = *(YYSTYPE*)yylvalptr;

Something similar is usually placed at the top of the Lex action function, although in this case it is necessary to go through the yyparserptr member of the yylexer class in order to reach the yylvalptr member variable.

See Also

yylexer::yylex


yyparser::yystackoverflow

#include <cyacc.h>

virtual void yystackoverflow( );

Remarks

The yystackoverflow member function is called if the parser’s stack ever overflows. This can be for one of three reasons:

1	The stack is not growable, and has simply overflowed.

2	The stack is growable, however the memory block requested by the parser is simply too big to fit in an internal representation. For instance, in 16-bit DOS applications, the largest block of memory that can be allocated is 65,535 bytes in size.  Note also that when the stack grows both the (state) stack and the attribute stack grow.  For a 16-bit application, a state is 2 bytes in size. Similarly, an attribute is also 2 bytes in size if the default int type is used.  However, if you redefine the attribute type you may find its storage requirements increase.  Correspondingly this will lead to an increase in the size of the attribute stack.

3	The stack is growable, however the computer has run out of memory. Note this is not very likely on modern machines.

The default implementation of yystackoverflow simply calls yyerror with the error message "yacc stack overflow".  Override this function in your AYACC generated class if you wish to handle stack overflow reporting differently.  Note that when yystackoverflow returns, the calling parser function (either yyparse, or yywork) will itself return immediately with the exit code YYEXIT_FAILURE.  For example:

%name myparser { virtual void yystackoverflow(); }

...

void myparser::yystackoverflow()
{

    /* do something */
}

Beware of redefining yyparser::yystackoverflow directly.  Some compilers, Visual C++ for instance, do not permit member functions held in libraries to be redefined.  You always have to derive a new class from the one you want to modify, and then override the relevant member functions where necessary.  Note that this does not apply to C functions, where you can generally redefine any function held in a library that you want to.

See Also

yyparser::yyerror, yyparser::yysetstacksize, yyparser::yystackgrow


yyparser::yysyntaxerror

#include <cyacc.h>

virtual void yysyntaxerror( );

Remarks

The yysyntaxerror member function is called by the parser whenever a syntax error occurs.  The function simply calls yyerror with the error message "syntax error".  Override this function in your AYACC generated class if you wish to handle syntax error reporting differently.  For example:

%name myparser { virtual void yysyntaxerror(); }

...

void myparser::yysyntaxerror()
{

    /* do something */
}

Beware of redefining yyparser::yysyntaxerror directly.  Some compilers, Visual C++ for instance, do not permit member functions held in libraries to be redefined.  You always have to derive a new class from the one you want to modify, and then override the relevant member functions where necessary.  Note that this does not apply to C functions, where you can generally redefine any function held in a library that you want to.

See Also

yyparser::yyerror


yyparser::yyclearin

#include <cyacc.h>

void yyparser::yyclearin( );

Remarks

The yyclearin member function discards the current lookahead token for the parser.  This function has no effect if there is no current lookahead token.  The effects of the yyclearin function can be reversed by calling the yyunclearin member function.

Compatibility

In traditional YACC, the yyclearin function is implemented as a macro without any trailing parentheses.  If you want to use this implementation of yyclearin then you need to define YY_COMPATIBLE.  This needs to be done before the header file CYACC.H has been included in your source files.  The easiest way to do this is by using a command line option.  If you are using an IDE like Visual C++ or Borland C++ then you can do this more easily through the relevant project option dialog boxes.

If you do define YY_COMPATIBLE then it is not just the implementation of yyclearin which changes.  The yyerrok member function is changed too.  Similar to yyclearin, it is implemented as a macro without any trailing parentheses.

See Also

yyparser::yyunclearin, yyparser::yychar, yyparser::yylookahead


yyparser::yydestructclearin

#include <cyacc.h>

virtual void yydestructclearin( ) = 0;

Remarks

The yydestructclearin member function discards the current lookahead token for the parser.  It is specified as a pure virtual function in the yyparser base class.  Proper virtual functions are provided in the yyfparser and yycparser derived classes.  Each one providing the necessary functionality to deal with default (fast) and compact parsers respectively.  Unlike yyclearin, yydestructclearin also performs clean up for the token being discarded.  That is, if the token has a symbol destructor associated with it, then the destructor will be called for that token.  The effects of a call to yydestructclearin may be fully, or partially restored by a call to yyunclearin.  Note that this will depend upon what the symbol destructor, if any, associated with the lookahead token does.

See Also

yyparser::yyclearin, yyparser::yydestructpop, yyparser::yychar, yyparser::yylookahead


yyparser::yyerrok

#include <cyacc.h>

void yyerrok( );

Remarks

The yyerrok member function informs the parser that error recovery is complete.  This function has no effect if the parser is not currently in error recovery.  Typically, error recovery is entered after a syntax error is encountered by the parser.  It is possible for an action to generated a syntax error through the use of the yythrowerror function.

When the parser encounters a syntax error it inserts an error token in front of the current lookahead token, if any.  The parser then examines the state on top of the stack to see whether it has an error token transition.  If not the parser pops the state off of the stack and examines the next one.  It repeats this until either an error token transition is found or all of the states are popped off of the stack.  If this happens the parse fails, and the parser returns with the exit code YYEXIT_FAILURE.

When, or if an error token transition is found, the parser can do one of two things.  Note that the action depends upon whether the yythrowerror function generated the syntax error or not, and if so the value of the pop parameter passed to it:

1	If the yythrowerror function generated the syntax error and the pop parameter is greater than zero then it is decremented.  The state is popped off of the stack, and the parser continues to look for error token transitions.  Note that if the state is a nonterminal symbol state, and that symbol has a destructor associated with it, then the destructor will be called for the state in question.

2	If the yythrowerror function generated the syntax error and the pop parameter is zero, or the function didn't generate the error, then the error token is shifted onto the stack and the parser enters error recovery.

Note that when it says that the error token is shifted onto the stack, it doesn’t mean the token itself, but rather the state that the error token transition goes to.

Error token transitions themselves are formed by including error productions in your grammar.  The identifier error is predefined by YACC, and is used to represent the error token.

A: B C      { /* do something */ }

 | error a  { /* do something */ }

 ;

The parser stays in error recovery until three consecutive tokens have been shifted onto the stack.  Note that this does not include the error token itself.  During this time no more syntax errors are reported, including any that are generated by the yythrowerror function.  Any tokens which cannot be shifted onto the stack are discarded.

Compatibility

In traditional YACC, the yyerrok function is implemented as a macro without any trailing parentheses.  If you want to use this implementation of yyerrok then you need to define YY_COMPATIBLE.  This needs to be done before the header file CYACC.H has been included in your source files.  The easiest way to do this is by using a command line option.  If you are using an IDE like Visual C++ or Borland C++ then you can do this more easily through the relevant project option dialog boxes.

If you do define YY_COMPATIBLE then it is not just the implementation of yyerrok which changes.  The yyclearin function is changed too.  Similar to yyerrok, it is implemented as a macro without any trailing parentheses.

See Also

yyparser::throwerror


yyparser::yyexit
yyparser::yyexit, yyparser::accept, yyparser::yyabort

#include <cyacc.h>

void yyexit( int exitcode );

void yyabort( );

void yyaccept( );

Parameter	Description

exitcode	The exit code that the parser is to return with.  The parser itself reserves two exit codes.  These are YYEXIT_SUCCESS and YYEXIT_FAILURE.  YYEXIT_SUCCESS is returned if the parse succeeds, and YYEXIT_FAILURE is returned if the parse fails.  Failure can occur if the stack overflows or an untrapped syntax error is encountered.

Remarks

The yyexit member function forces the parser to return with the given exit code.  The function has some limitations on where it can be used, however these are not generally restrictive.  If the function is called from elsewhere then it will have no effect.  The yyexit function can be:

1	Called from an action.  This does not include actions associated with symbol destructors.

2	Called from some function, but program flow of control must return to an action at some point.  Again this does not include actions associated with symbol destructors.

The yyexit function does not take effect immediately, but rather when control returns from the action to the parser.  The rule that the action is associated with is not reduced, and any assignment to the $$ attribute is lost.  Assignments to the $n attributes will stand.

There are a number of other functions which take effect when control returns from an action to the parser.  These are the yythrowerror, yyforceerror and yyretire member functions.  The yyexit function takes priority over all of these functions.  The effect of any calls to them will be lost if yyexit is called.

If the yyexit function is called several times from within an action, then the exit code for the last call is the value that is returned by the parser.

The yyaccept and yyabort functions are shorthand's for the most commonly used yyexit calls.  The yyaccept function is equivalent to yyexit( YYEXIT_SUCCESS ), and the yyabort function is equivalent to yyexit( YYEXIT_FAILURE ).

Compatibility

The yyaccept function is also implemented as the YYACCEPT macro, and the yyabort function is implemented as the YYABORT macro.  The macros do not have any trailing parentheses.  In traditional YACC, these macros will cause an immediate jump to the relevant accept or abort code.  This means that no code following the YYACCEPT or YYABORT macros will be executed.  In traditional YACC, due to implementation details, these macros can only be placed in the actions themselves.  They cannot be placed in a function called from an action.  Note that this does not apply to AYACC, where YYACCEPT and YYABORT can be placed anywhere that yyaccept and yyabort can.

See Also

yyparser::yyretire


yyparser::yypopping

#include <cyacc.h>

int yypopping( );

Return Value

Nonzero if the parser has been popping states off of the stack whilst looking for an error token transition, otherwise zero.

Remarks

The yypopping member function returns whether the parser has been popping states off of the stack whilst looking for an error token transition.  This occurs when the parser encounters a syntax error.  The parser first of all examines the top most state to see whether it has an error token transition.  If not then it pops the state off of the stack, and looks at the next state.  It repeats this until either all of the states on the stack have been popped off of it or an error transition is found.  Note that if the yythrowerror function caused the syntax error, then there is the possibility that one or more states with error token transitions will also be popped off of the stack as well.

The value returned by the yypopping function will remain the same until another syntax error is encountered by the parser.  For instance, if the parser encounters a syntax error and recovery first takes place after popping one or more states, then the yypopping function will return a nonzero value from that point onwards.  It is only when another syntax error is encountered that there is a possibility of the return value changing i.e. if no states are popped off of the stack whilst attempting error recovery.

The yypopping function is useful in conjunction with the yythrowerror function for skipping over error productions.  For example, the rule below recognises a left parenthesis.  It also has an error production for reporting when said left parenthesis is missing.  In it, it has a test to see whether the parser has been popping states whilst trying to look for an error token transition.  If so then the yythrowerror function is called.  This will cause the parser to skip over this error production (the 1), and look for another one.  The reasoning behind this is quite simple.  If the parser has been popping states off of the stack then it means that there was something after the left parenthesis.  Reporting a missing parenthesis for an error which has occurred later on does not make sense:

LParen

    : '('

    | error {

                if (yypopping()) {

                    yythrowerror(1);

                }

                else {

                    yyerror("missing \'(\'"); }

                }

            }

    ;

See Also

yyparser::yythrowerror


yyparser::yyrecovering

#include <cyacc.h>

int yyrecovering( ) const;

Return Value

Nonzero if the parser is in error recovery, otherwise zero.

Remarks

The yyrecovering member function returns whether the parser is in error recovery or not.  Typically, error recovery is entered after a syntax error is encountered by the parser.  It is possible for an action to generated a syntax error through the use of the yythrowerror function.

When the parser encounters a syntax error it inserts an error token in front of the current lookahead token, if any.  The parser then examines the state on top of the stack to see whether it has an error token transition.  If not the parser pops the state off of the stack and examines the next one.  It repeats this until either an error token transition is found or all of the states are popped off of the stack.  If this happens the parse fails, and the parser returns with the exit code YYEXIT_FAILURE.

When, or if an error token transition is found, the parser can do one of two things.  Note that the action depends upon whether the yythrowerror function generated the syntax error or not, and if so the value of the pop parameter passed to it:

1	If the yythrowerror function generated the syntax error and the pop parameter is greater than zero then it is decremented.  The state is popped off of the stack, and the parser continues to look for error token transitions.  Note that if the state is a nonterminal symbol state, and that symbol has a destructor associated with it, then the destructor will be called for the state in question.

2	If the yythrowerror function generated the syntax error and the pop parameter is zero, or the function didn't generate the error, then the error token is shifted onto the stack and the parser enters error recovery.

Note that when it says that the error token is shifted onto the stack, it doesn’t mean the token itself, but rather the state that the error token transition goes to.

Error token transitions themselves are formed by including error productions in your grammar.  The identifier error is predefined by YACC, and is used to represent the error token.

A: B C      { /* do something */ }

 | error a  { /* do something */ }

 ;

The parser stays in error recovery until three consecutive tokens have been shifted onto the stack.  Note that this does not include the error token itself.  During this time no more syntax errors are reported, including any that are generated by the yythrowerror function.  Any tokens which cannot be shifted onto the stack are discarded.  Note that it is possible to re-synchronise error recovery through the yyerrok function.

Compatibility

The yyrecovering function is also implemented as the YYRECOVERING macro.  The macro does not have any trailing parentheses.

See Also

yyparser::yyerrok


yyparser::yyretire

#include <cyacc.h>

void yyretire( int exitcode );

Parameter	Description

exitcode	The exit code that the parser is to return with.  The parser itself reserves two exit codes.  These are YYEXIT_SUCCESS and YYEXIT_FAILURE.  YYEXIT_SUCCESS is returned if the parse succeeds, and YYEXIT_FAILURE is returned if something goes wrong.  For instance the stack overflows, or an untrapped syntax error is encountered.

Remarks

The yyretire member function forces the parser to return with the given exit code.  The function has some limitations on where it can be used, however these are not generally restrictive.  If the function is called from elsewhere then it will have no effect.  The yyretire function can be:

1	Called from an action.  This does not include actions associated with symbol destructors.

2	Called from some function, but program flow of control must return to an action at some point.  Again this does not include actions associated with symbol destructors.

The yyretire function does not take effect immediately, but rather when control returns from the action to the parser.  The rule that the action is associated with is reduced, and any assignment to the $$ attribute is saved on the attribute stack.  Compare this to the effects of the yyexit function.

You can continue the parse at a subsequent date by calling the yywork function.  The state and attribute stacks are preserved to allow you to do this.

There are a number of other functions which take effect when control returns from an action to the parser.  These are the yyexit, yyaccept, yyabort, yythrowerror and yyforceerror member functions.  All of these functions take priority over the yyretire function.  The effect of a call to yyretire will be lost if any of them are called.

If the yyretire function is called several times from within an action, then the exit code for the last call is the value that is returned by the parser.

See Also

yyparser::work, yyparser::yyexit


yyparser::yysetin

#include <cyacc.h>

void yysetin( int token );

Parameter	Description

token	The value to set the current lookahead token to.  A value of zero or a negative number indicates the $end (EOF, DONE etc.) token.

Remarks

The yysetin member function sets the current lookahead token to the given value.  No checks are made whether the token is valid or not.  If the current lookahead token already holds a value then it is overwritten.

See Also

yyparser::yyclearin


yyparser::yythrowerror
yyparser::yythrowerror, yyparser::yyforceerror

#include <cyacc.h>

void yyparser::yyforceerror( );

void yyparser::yythrowerror( int pop );

Parameter	Description

pop	The number of states with error token transitions to pop off of the stack before entering error recovery with the next one found.  This parameter must be zero or a positive number.

Remarks

The yythrowerror member function forces the parser to generate a syntax error.  The function has some limitations on where it can be used, however these are not generally restrictive.  If the function is called from elsewhere then it will have no effect.  The yythrowerror function can be:

1	Called from an action.  This does not include actions associated with symbol destructors.

2	Called from some function, but program flow of control must return to an action at some point.  Again this does not include actions associated with symbol destructors.

The yythrowerror function does not take effect immediately, but rather when control returns from the action to the parser.  The rule that the action is associated with is not reduced, and any assignment to the $$ attribute is lost.  Assignments to the $n attributes will stand.

When control returns to the parser it inserts an error token in front of the current lookahead token, if any.  The parser then examines the state on top of the stack to see whether it has an error token transition.  If not the parser pops the state off of the stack and examines the next one.  It repeats this until either an error token transition is found or all of the states are popped off of the stack.  If this happens the parse fails, and the parser returns with the exit code YYEXIT_FAILURE.

When, or if an error token transition is found, the parser can do one of two things.  Note that the action is determined by the value of the pop parameter passed to the yythrowerror function:

1	If the pop parameter is greater than zero then it is decremented.  The state is popped off of the stack, and the parser continues to look for error token transitions.  Note that if the symbol represented by the state has a symbol destructor associated with it then the destructor will be called for the state in question.

2	If the pop parameter is zero then the error token is shifted onto the stack and the parser enters error recovery.

Note that when it says that the error token is shifted onto the stack, it doesn’t mean the token itself, but rather the state that the error token transition goes to.

Error token transitions themselves are formed by including error productions in your grammar.  The identifier error is predefined by YACC, and is used to represent the error token.

A: B C      { /* do something */ }

 | error a  { /* do something */ }

 ;

The parser stays in error recovery until three consecutive tokens have been shifted onto the stack.  Note that this does not include the error token itself.  During this time no more syntax errors are reported, including any that are generated by the yythrowerror function.  Any tokens which cannot be shifted onto the stack are discarded.  Note that it is possible to re-synchronise error recovery through the yyerrok function.

There are a number of other functions which take effect when control returns from an action to the parser.  These are the yyexit, yyaccept, yyabort, and yyretire member functions.  The yythrowerror function takes priority over the yyretire function, however the yyexit, yyaccept and yyabort functions take priority over yythrowerror.  The effect of any calls to yyretire will be lost if yythrowerror is called.

If the yythrowerror function is called several times from within an action, then the pop argument for the last call is the value that is used by the parser.

The yyforceerror function is a shorthand for the most commonly used yythrowerror(0) call.

Compatibility

The yyforceerror function is also implemented as the YYERROR macro.  The macro does not have any trailing parentheses.  In traditional YACC, this macro will cause an immediate jump to the relevant error handling code.  This means that no code following the YYERROR macro will be executed.  In traditional YACC, due to implementation details, this macro can only be placed in an action itself.  It cannot be placed in a function called from an action.  Note that this does not apply to AYACC, where YYERROR can be placed anywhere that yyforceerror can.

See Also

yyparser::yyerrok


yyparser::yyunclearin

#include <cyacc.h>

int yyunclearin( );

Return Value

Nonzero if the current lookahead token could be restored to its previous value, otherwise zero.

Remarks

The yyunclearin member function restores the current lookahead token to its previous value.  The function succeeds if the current lookahead token is empty and it has previously held a value.

See Also

yyparser::yyclearin


yyfparser

#include <cyacc.h>

class yyfparser : public yyparser;

Remarks

The yyfparser class provides the basic functionality for a default (fast) C++ parser generated by AYACC.  It is derived from yyparser, which provides most of the functionality.  The yyfparser class has an implementation of the yywork pure virtual function.  It also has a number of extra member variables which hold table data used by yywork.

The yyfparser class is an abstract class so you cannot create an object from this class directly.  The pure virtual function which makes yyfparser an abstract class is yyaction.

If a default (fast) parser is generated by AYACC, then the generated class is derived from yyfparser.  The generated class provides an implementation of the yyaction pure virtual function.

See Also

yyparser, yyparser Members


yycparser

#include <cyacc.h>

class yycparser : public yyparser;

Remarks

The yycparser class provides the basic functionality for a compact C++ parser generated by AYACC.  It is derived from yyparser, which provides most of the functionality.  The yycparser class has an implementation of the yywork pure virtual function.  It also has a number of extra member variables which hold table data used by yywork.

The yycparser class is an abstract class so you cannot create an object from this class directly.  The pure virtual function which makes yycparser an abstract class is yyaction.

If a compact parser is generated by AYACC, then the generated class is derived from yycparser.  The generated class provides an implementation of the yyaction pure virtual function.

See Also

yyparser, yyparser Members


yylexer

#include <clex.h>

class yylexer;

Remarks

The yylexer class provides the basic functionality for all C++ lexical analysers generated by ALex.  It has no base class.  The yylexer class is an abstract class so you cannot create an object from this class directly.  It must be from a derived class.  The pure virtual functions which make yylexer an abstract class are yylex and yyaction.

The yylexer class has two derived classes, yyflexer and yyclexer.  Both of these classes are abstract classes too.  The yyflexer class implements the functionality for a default (fast) lexical analyser, and the yyclexer class implements the functionality for a compact lexical analyser.  The lexical analyser class generated by ALex will be derived from one of these classes, depending upon the options you have selected.  Note though that you need to provide additional information to Lex in order for it to be able to do this.  For instance you must provide, at the very least, a name for the new class.  This achieved by using a Name Declaration.  Basically, all you need to place in the Declarations Section of your Lex source file, is the keyword %name, followed by the name of the derived class.  For example, the name declaration below will declare the new derived class mylexer:

%name mylexer

This will then place the relevant class definition in the generated include (or output) file, and generate a default constructor for the class.  You can also use name declarations to place additional member functions and member variables in the new class definition, and initialisation code in the constructor.  It is possible, if you so wish, to duplicate the effects of a name declaration by placing the necessary, equivalent code, in the include and output files using Include Declarations and Code Blocks.  For more information see also Defines.

See Also

yylexer Members, yyflexer, yyclexer


yylexer Members

These are the commonly used member variables and functions in the yylexer class.  For a complete list of available member variables and functions for the class look in the CLEX.H header file.

Members Variables

yydebug	Whether debug information should be output.

yydebugflush	Indicates whether debug output should be flushed when it is written.

yydebugout	The stream to which the lexical analyser sends debug output.

yyeol	Whether the previous (buffered) character was a newline character.

yyerr	The steam to which the lexical analyser sends error messages.

yyin	The stream from which the lexical analyser gets its input.

yyleng	The length of the matched string.

yylineno	The current line number.

yyout	The stream to which the lexical analyser sends its output.

yystart	The current start state.

yytext	The matched string.

yytextgrow	Specifies whether the text buffer should grow if it ever should overflow.

yyunputgrow	Specifies whether the unput buffer should grow if it ever overflows.

Construction Member Functions

yylexer	Constructs a yylexer object.

yycreate	Creates the lexical analyser, allocating any memory where necessary.

yydestroy	Destroys the lexical analyser, freeing any allocated memory.

General Member Functions

yycleanup	Frees any memory that may have been allocated through changing the size of text and unput buffers.

yylex	Performs lexical analysis for the lexical analyser.

yyreset	Resets the lexical analyser.

yysettextsize	Sets the size of the lexical analyser’s text buffer and state buffer.

yysetunputsize	Sets the size of the lexical analyser’s unput buffer.

Service Member Functions

yygetchar	Called to get the next character for the lexical analyser.

yyinput	Called to get the next, possibly buffered, character for the lexical analyser.

yyoutput	Called to output an unmatched character.

yytextoverflow	Called when the text buffer overflows.

yyunput	Called to back up a character.

yyunputoverflow	Called when the unput buffer overflows.

yywrap	Called when EOF is encountered by the lexical analyser.

Action Member Functions

yybegin	Places the lexical analyser in the requested start state.

yyecho	Echoes the matched string.

yyless	Backs up the matched string until it has the requested length.

yymore	Concatenates the next matched string on the end of the currently matched string.

yynewline	Specifies whether the previous matched character was a newline character or not.

yyreject	Forces the lexical analyser to match another different regular expression.

yyunputcount	Retrieves the number of characters stored in the unput buffer.

See Also

yylexer


yylexer::yydebug

#include <clex.h>

int yydebug;

Remarks

The yydebug member variable specifies whether debug information should be output by the lexical analyser while it is running.  If yydebug is nonzero then debug information is generated.  By default yydebug is set to zero, i.e. no debugging information is generated.  This member is only defined for the debug version of the YACC and Lex library.

See Also

yylexer::yydebugout, yydebug


yylexer::yydebugflush

#include <clex.h>

int yydebugflush;

Remarks

The yydebugflush member variable specifies whether the debug output stream should be flushed whenever debug information is written to it.  If yydebugflush is nonzero then the steam will be flushed.  By default yydebugflush is set to zero i.e. the stream will not be flushed.  Flushing the debug stream is useful if the application which is being debugged crashes.  In such circumstances, the file written to, assuming it is a file, will contain all of the debug information.  The downside though is that flushing the stream slows down the application significantly.  This variable is only defined for the debug version of the YACC and Lex library.

See Also

yylexer::yydebugout, yylexer::yydebug, yydebug, yydebugflush


yylexer::yydebugout

#include <clex.h>

FILE* yydebugout;

Remarks

The yydebugout member variable specifies the output stream to which the lexical analyser sends its debugging information.  This variable is initialised to stdout by default.  You can assign a new value to yydebugout in your source code.  This variable is only defined for the debug version of the YACC and Lex library.

In 32-bit Windows versions of the YACC and Lex library, if yydebugout is set to NULL, then debugging information is sent to the debugger.  If the application containing the parser has an associated debugger, then the debug information is sent here.  Failing that, if the system debugger is active then debug information is sent here.  Otherwise any debug information is simply lost.

See Also

yylexer::yydebug, yylexer::yydebugflush, yydebug, yydebugflush


yylexer::yyeol

#include <clex.h>

unsigned char yyeol;

Remarks

The yyeol member variable specifies whether the previous (buffered) character read by the lexical analyser was a newline \n character or not.  yyeol is nonzero if the previous character was a newline character, otherwise it is zero.  yyeol is initialised to 1 automatically, and is set to 1 by the yyreset member function.

After a string has been matched by the lexical analyser, but before the associated action has been performed, yyeol is updated ready for matching the next string.  yyeol is also updated by the yyless member function, should you use one in an action.  Note that if you backup the matched string completely, then yyeol is initialised with the yyoldeol member variable.  You can modify yyeol in your actions to suit your needs, providing of course, that you don't make a subsequent call to yyless.

See Also

yylexer::yystart, yylexer::yyreset, yylexer::yyless


yylexer::yyerr

#include <clex.h>

FILE* yyerr;

Remarks

The yyerr member variable specifies the output stream to which any error messages are sent.  This variable is initialised to the standard error stream stderr by default.  You can assign a new value to yyerr in your source code.

See Also

yylexer::yytextoverflow, yylexer::yyunputoverflow


yylexer::yyin

#include <clex.h>

FILE* yyin;

Remarks

The yyin member variable specifies the input stream from which the yyinput member function gets its characters.  This variable is initialised to the standard input stream stdin by default.  You can assign a new stream to yyin in your Lex source file.  For example:

int main(void)
{

    mylexer lexer;

    lexer.yycreate();

    lexer.yyin = fopen("mytext.txt", "r");

    if (lexer.yyin != NULL) {

        return lexer.yylex();

    }

    return 1;

}

Here an attempt is made to open the text file "mytext.txt" as the replacement input steam.  If this is successful, indicated by a non-NULL return value from fopen, then the input is processed in the usual way by a call to yylex.  Note that in this example it is not strictly necessary to call fclose in order to close the input stream.  When main exits, all open streams are closed automatically.

See Also

yylexer::yyinput, yylexer::yyout


yylexer::yyleng

#include <clex.h>

int yyleng;

Remarks

The yyleng member variable specifies the length of the matched string.  Beware of modifying yyleng in your source code, especially if you make use of the yymore, yyless and yyreject member functions.  All three functions make use of the variable.

See Also

yylexer::yytext


yylexer::yylineno

#include <clex.h>

int yylexer::yylineno;

Remarks

The yylineno member variable holds the current line number.  Line numbers are assumed to start at 1.  The line number is incremented whenever a newline character \n is input, and it is decremented whenever a newline character \n is unput during backup.

This variable is reset to 1 by the yyreset member function.

See Also

yylexer::yyinput, yylexer::yyunput, yylexer::yyreset


yylexer::yyout

#include <clex.h>

FILE* yyout;

Remarks

The yyout member variable specifies the output stream to which the yyoutput member function sends its characters. This variable is initialised to the standard output stream stdout by default.  You can assign a new stream to yyout in your Lex source file.  For example:

int main(void)
{

    mylexer lexer;

    lexer.yycreate();

    lexer.yyout = fopen("mytext.txt", "w");

    if (lexer.yyout != NULL) {

        return lexer.yylex();

    }

    return 1;

}

Here an attempt is made to open the text file "mytext.txt" as the replacement output steam.  If this is successful, indicated by a non-NULL return value from fopen, then the input is processed in the usual way by a call to yylex.  Note that in this example it is not strictly necessary to call fclose in order to close the output stream.  When main exits, all open streams are closed automatically.

See Also

yylexer::output, yylexer::yyin


yylexer::yystart

#include <clex.h>

int yystart;

Remarks

The yystart member variable holds the current start state for the lexical analyser.  The start state can be changed by using the yybegin member function.  However, sometimes it is useful to be able to use the actual start state value itself.  For instance, consider the example below which is attempting to recognise a C comment:

%name mylexer { virtual int yywrap(); }

%start COMMENT

%%

<COMMENT>"*/"   { yybegin(0); }

<COMMENT>.|\n   { /* do nothing */ }

"/*"            { yybegin(COMMENT); }

...

%%

int mylexer::yywrap(void)
{

    if (yystart == COMMENT) {

        error("unexpected end-of-file found in comment");
    }

    return 1;
}

In the overridden yywrap member function above a check is made to see whether the lexical analyser is in the COMMENT start state.  If so then it means that the lexical analyser has found the start of a comment, but has not seen a terminating */.  In these cases, an appropriate error message is generated, before returning with the usual 1 indicating no more input.  It is worth noting that this use of yywrap can be duplicated by trailing end-of-file operators <<EOF>> in your Lex rules.  For more information see Right Context Expression.

See Also

yylexer::yybegin


yylexer::yytext

#include <clex.h>

char* yytext;

Remarks

The yytext member variable holds the matched regular expression string.  The string is zero terminated, although be aware that regular expressions can include zero as a character.  The length of the matched string is specified by the yyleng member variable.

See Also

yylexer::yymore, yylexer::yyleng, yylexer::yytextgrow


yylexer::yytextgrow

#include <clex.h>

unsigned char yytextgrow;

Remarks

The yytextgrow member variable specifies whether the lexical analyser's text and state buffers should grow if they ever overflow.  If yytextgrow is nonzero then the buffers are allowed to grow.  The initial size of the text and state buffers is 100 elements unless you have defined YYTEXT_SIZE in your Lex source file.  The example below defines a text and state buffer with 1000 elements:

#define YYTEXT_SIZE 1000

By default yytextgrow is set to 1 allowing the text and state buffers to grow.

If the text and state buffers overflow and they are allowed to grow, the lexical analyser will double the size of each buffer.  If the text and state buffers have zero elements (possible if you define YYTEXT_SIZE as 0), then the lexical analyser will allocate 100 elements for each buffer instead.  All buffer changes are achieved through calls to the yysettextsize member function.

If the text and state buffers overflow and they are not allowed to grow, then the lexical analyser calls the yytextoverflow member function before calling the standard C library function exit with the argument EXIT_FAILURE.

In all of the above cases the text buffer always has one extra character.  This is to cope with the zero terminating character which is added onto the end of all matched strings.

See Also

yylexer::yysettextsize, yylexer::yytextoverflow, yylexer::yyunputgrow


yylexer::yyunputgrow

#include <clex.h>

unsigned char yyunputgrow;

Remarks

The yyunputgrow member variable specifies whether the lexical analyser's unput buffer should grow if it ever overflows.  If yyunputgrow is nonzero then the unput buffer is allowed to grow.  The initial size of the buffer is 100 elements unless you have defined YYTEXT_SIZE or YYUNPUT_SIZE in your Lex source file.  The example below defines an initial unput buffer of 1000 elements.

#define YYUNPUT_SIZE 1000

By default yyunputgrow is set to 1 allowing the unput buffer to grow.

If the unput buffer overflows and it is allowed to grow, the lexical analyser will double the size of the buffer.  If the buffer has zero elements (possible if you define YYTEXT_SIZE or YYUNPUT_SIZE as 0), then the lexical analyser will allocate a 100 element buffer instead.  All buffer changes are achieved through calls to the yysetunputsize member function.

If the unput buffer overflows and it is not allowed to grow, then the lexical analyser calls the yyunputoverflow member function before calling the standard C library function exit with the argument EXIT_FAILURE.

See Also

yylexer::yysetunputsize, yylexer::yyunputoverflow, yylexer::yytextgrow


yylexer::yylexer

#include <clex.h>

yylexer::yylexer( );

Remarks

Constructs a yylexer object.  The lexical analyser will not be ready for use until the yydestroy member function has been called.

See Also

yylexer::yycreate


yylexer::yycreate

#include <clex.h>

int yycreate( yyparser* parserptr = NULL );

Parameter	Description

parserptr	A pointer to the parser object associated with the lexical analyser, or NULL if there is none.

Return Value

Nonzero if successful, otherwise zero.  Failure occurs if the necessary memory for the initial text, state and unput buffers cannot be allocated from the heap.

Remarks

The yycreate member function creates the lexical analyser.  This function must be called before the lexical analyser can be used.  The function needs to allocate memory from the heap for the initial text, state and unput buffers.  If the function cannot allocate the necessary memory then the function fails and returns zero.

The parserptr parameter passed to the yycreate member function is assigned to the yyparserptr member variable.  This member variable is not used by any yylexer, yyflexer or yyclexer member functions.  It's primary purpose is to facilitate assigning token attributes to a parser's yylval variable.  In the yyparser class, this is pointed to by the yylvalptr member variable.  Typically, the following Code Block, or something similar, will be placed after the first %% mark in a Lex source file at the top of the Rules section.  The code will be copied into the top of the yyaction member function.  Any following actions can then, quite easily, make reference to yylval:

%{

YYSTYPE& yyval = *(YYSTYPE*)yyparserptr->yylvalptr;
%}

The lexical analyser is destroyed by calling the yymdestroylex member function.

An example of how a lexical analyser is created, used and destroyed is shown below:

#include "mylexer.h"

int analyse(void)
{

    mylexer lexer;

    if (lexer.yycreate()) {

        int n = lexer.yylex();

        lexer.yydestroy();

        return n;

   }

   return 1;
}

The lexical analyser class mylexer is declared in the include file "mylexer.h".  This would have been generated by ALex.  Note that the call to yydestroy is not really necessary, since it will be called by the destructor anyway.

See Also

yylexer::yydestroy


yylexer::yydestroy

#include <clex.h>

void yydestroy( );

Remarks

The yydestroy member function destroys the lexical analyser.  A lexical analyser is usually created first by a call to the yycreate member function.  The yydestroy member function frees up any memory allocated to the text, state and unput buffers.  The function does not attempt to free up the memory associated the yylexer object itself.  For an example of how lexical analysers are created and destroyed see yycreatelex.

The yydestroy member function frees the memory allocated to the lexical analyser by first calling the yycleanup member function.  This frees any memory allocated through calls to the yysettextsize and yysetunputsize member functions.  The function then frees the initial text, state and unput buffers allocated by the yycreate member function.

This member function is also called the yylexer destructor as part of clean up.  As such, this member function can be called multiple times, and it is not necessary to call yycreate beforehand.

See Also

yylexer::yycreate


yylexer::yycleanup

#include <clex.h>

void yycleanup( );

Remarks

The yycleanup member function frees any memory that may have been allocated through calls to the yysettextsize and yysetunputsize member functions.  These functions are either called directly by yourself, or indirectly by the lexical analyser if it ever needs to grow one of the buffers.  Do not call this function while the lexical analyser is running.

The yycleanup function will not free any memory allocated by the yycreate member function.  In order to do this you will need to call the yydestroy member function.  Note that this function in turn calls yycleanup as part of its operation.

See Also

yylexer::yysettextsize, yylexer::yysetunputsize


yylexer::yylex

#include <clex.h>

virtual int yylex( ) = 0;

Return Value

Zero if the lexical analyser has no more processing to do, otherwise user defined.

Remarks

The yylex member function causes the lexical analyser to process input.  It is specified as a pure virtual function in the yylexer base class.  Proper virtual functions are provided in the yyflexer and yyclexer derived classes.  Each one providing the necessary functionality to deal with default (fast) and compact tables respectively.

Control will remain in yylex until one of the following conditions is met:

1	The lexical analyser has run out of input.  In which case zero is returned.

2	An action executes a return statement.  In which case the expression specified in the statement is returned by yylex.

The yylex member function also calls a number of service functions.  These can be customised to change the way the lexical analyser behaves by overriding the relevant virtual function.  For a list of the available service functions see yylexer Members.

See Also

yylexer::yyreset


yylexer::yyreset

#include <clex.h>

void yyreset( );

Remarks

The yyreset member function resets the lexical analyser’s member variables back to their default values.  It effectively flushes all the buffers, sets the line count back to 1, and ensures that the states are set up correctly ready for a new stream of input.  Note that this function does not have any impact on the sizes of the text and unput buffers.  To reset these back to their initial sizes you will need to call the yycleanup member function.

It is only necessary to call this function if you are going to re-use the lexical analyser.  Note that the variables start off initialised.  It is however your responsibility to make sure that the input stream has been set-up correctly ready for the lexical analyser to receive new characters.

See Also

yylexer::yycleanup, yylexer::yywrap


yylexer::yysettextsize

#include <clex.h>

int yylexer::yysettextsize( int size );

Parameter	Description

size	The requested size of the lexical analysers text buffer and state buffer.  This must be greater than or equal to zero.  The text buffer always gains one extra element.  This is to cope with the zero terminating character that is appended to all matched strings.

Return Value

Nonzero if the lexical analysers text and state buffers were set to the specified size, otherwise zero in which case the size of the buffers remains unchanged.

Remarks

The yysettextsize member function sets the lexical analysers text and state buffers to the specified size.  You can change the size of the buffers while the lexical analyser is running.  Note that if you specify a value that is less than yyleng, then the function will fail (and return zero).  In some circumstances you may want to explicitly set yyleng to zero to ensure that this function succeeds.

This function may allocate memory from the heap if it needs to.  If the specified size is less that or equal to the size of the initial (static) text and state buffers, then it will use these.  By default they are 100 elements in size, unless of course you have redefined YYTEXT_SIZE in your Lex source file.  If it is greater than the initial text and state buffer size then it will allocate memory from the heap.

Note that this function is also used by the lexical analyser when it needs to automatically grow the text and state buffers.

See Also

yylexer::yycleanup, yylexer::yytextoverflow, yylexer::yysetunputsize, yylexer::yytextgrow, yylexer::yyleng


yylexer::yysetunputsize

#include <clex.h>

int yysetunputsize( int size );

Parameter	Description

size	The requested size of the lexical analysers unput buffer.  This must be greater than or equal to zero.

Return Value

Nonzero if the lexical analysers unput buffer was set to the specified size, otherwise zero in which case the size of the buffer remains unchanged.

Remarks

The yysetunputsize member function sets the unput buffer to the specified size.  You can change the size of the unput buffer while the lexical analyser is running.  Note that if you specify a value that is less than amount of unput buffer currently in use, then the function will fail (and return zero).

This function may allocate memory from the heap if it needs to.  If the specified size is less that or equal to the size of the initial (static) unput buffer, then it will use this.  By default this is 100 elements in size, unless of course you have redefined YYTEXT_SIZE or YYUNPUT_SIZE in your Lex source file.  If it is greater than the initial unput buffer size then it will allocate memory from the heap.

Note that this function is also used by the lexical analyser when it needs to automatically grow the unput buffer.

See Also

yylexer::yycleanup, yylexer::yyunputoverflow, yylexer::yysettextsize, yylexer::yyunputgrow


yylexer::yygetchar

#include <clex.h>

virtual int yygetchar( );

Return Value

The next (unbuffered) input character needed by the yyinput member function.  The value must be in the range 0 - 255 or -1 to indicate the end-of-file condition.

Remarks

The yygetchar member function is called by the yyinput member function to get the next (unbuffered) input character.  The default implementation of yygetchar simply gets a character from the stream specified by the yyin member variable:

int yylexer::yygetchar()
{

    return getc(yyin);
}

Override yygetchar in your ALex generated lexical analyser if you wish to change how the lexical analyser gets its input.  Note though that it is possible to re-assign a different stream to yyin.  For instance you may want to get input from a character array instead of a file.  The overridden yygetchar member function might look something like this.

%name mylexer { virtual int yygetchar();  }

...

int mylexer::yygetchar()
{

    if (*m_p == ‘\0’) {

        return -1;
    }

    return (unsigned char)*m_p++;
}

Note the cast of the contents of the member variable m_p to an unsigned char.  This eliminates any potential problems if char is signed by default.  Incidentally m_p is a char pointer.  This must be initialised to point to the start of the character array prior to the start of the lexical analysis.

The yygetchar member function may be called by the yyinput member function after it has returned a -1 (end-of-file) value.  As such if you do override yygetchar, be sure to take this into account.  In the example above, the contents of m_p are first checked for the end-of-file condition before are pointer incrementing takes place.

Beware of redefining yyparser::yygetchar directly.  Some compilers, Visual C++ for instance, do not permit member functions held in libraries to be redefined.  You always have to derive a new class from the one you want to modify, and then override the relevant member functions where necessary.  Note that this does not apply to C functions, where you can generally redefine any function held in a library that you want to.

See Also

yylexer::yyinput


yylexer::yyinput

#include <clex.h>

virtual int yyinput( );

Return Value

The next (buffered) input character needed by the lexical analyser.  The value is in the range 0 - 255 or -1 to indicate the end-of-file condition.

Remarks

The yyinput member function is called by the lexical analyser to get the next (buffered) input character.  It gets this character either from the unput buffer or by calling the yygetchar member function.

This function is also responsible for incrementing the yylineno member variable.  Whenever yyinput returns a newline character \n (either from the unput buffer or by calling the yygetchar member function) it increments the variable by 1.

Compatibility

The use of the yygetchar member function has reduced the need to redefine the yyinput member function.  However for compatibility reasons yyinput can also be defined as the macro input.  Note that the macro does not take any parameters.  In the generated lexical analyser there is a small section of code, similar to that below which then converts this macro into the function yyinput:

input

#ifdef input

int yyinput(void)

{

    return input();
}

#endif

The same also applies to the yyminput function, although not for multiple instance lexical analysers.

If you do redefine yyminput in your source code then, by default, all multiple lexical analysers will make use of the new function.  As an alternative, you can define a new function with the same prototype and assign it to the yyminput member of the yymlex_t structure.  You can do this either statically or at run time.  To do it statically, you need to define YYMINPUT as the function name in your Lex source file.  This needs to be done in the declarations section:

%{

int myinput(yymlex_t *yy)
{

    /* do something */
}
#define YYMINPUT myinput

%}

See Also

yylexer::yygetchar, yylexer::yyunput


yylexer::yyoutput

#include <clex.h>

virtual void yyoutput( int ch );

Parameter	Description

ch	The unmatched character to output.

Remarks

The yyoutput member function is called by the lexical analyser to output any unmatched characters.  The function is also called by the yyecho member function to output the matched string.  The default implementation of yyoutput simply writes the character to the stream specified by the yyout member variable:

void yylexer::yyoutput(int ch)
{

    putc(ch, yyout)
}

Override yyoutput in your ALex generated lexical analyser if you wish to change how the lexical analyser outputs unmatched characters.  Note though that it is possible to re-assign a different stream to yyout.

Compatibility

For compatibility reasons yyoutput can also be defined as the macro output.  In the generated lexical analyser there is a small section of code, similar to that below which then converts this macro into the function yyoutput:

#ifdef output

void yyoutput(int ch)

{

    output(ch);
}

#endif

The same also applies to the yymoutput function, although not for multiple instance lexical analysers.

See Also

yylexer::yyinput, yylexer::yyunput


yylexer::yytextoverflow

#include <clex.h>

virtual void yytextoverflow( );

Remarks

The yytextoverflow member function is called if the lexical analysers text buffer yytext ever overflows. Note that this also includes the accompanying state buffer.  This can be for one of three reasons:

1	The text buffer is not growable, and has simply overflowed.

2	The text buffer is growable, however the memory block requested by the lexical analyser is simply too big to fit in an internal representation.  For instance, in 16-bit DOS applications, the largest block of memory that can be allocated is 65,535 bytes in size.  Note also that when the text buffer grows, the accompanying state buffer also grows.  For a 16-bit application, a state is 2 bytes in size.

3	The text buffer is growable, however the computer has run out of memory. Note this is not very likely on modern machines.

The default implementation of yytextoverflow simply outputs the error message "lex text buffer overflow (size)\n", where size is the current size of the buffer.  The error message is sent to the stream specified by the yyerr member variable.  Override this function in your ALex generated lexical analyser if you wish to handle text buffer overflow reporting differently.  Note that when yytextoverflow returns, the calling lexical analyser function (yylex) will call the standard C library function exit immediately with the exit code EXIT_FAILURE.

See Also

yylexer::yyunputoverflow, yylexer::yytextgrow


yylexer::yyunput

#include <clex.h>

virtual void yyunput( int ch );

Parameter	Description

ch	The character to place in the unput buffer.  The character must be in the range 0-255.

Remarks

The yyunput member function is called by the lexical analyser whenever it needs to backup a character.  This can occur if the lexical analyser has read too many characters when trying to match a regular expression.  If the function cannot backup the character then it will call the yyunputoverflow member function, followed by the standard C library function exit with EXIT_FAILURE as its argument.  Note that this is unlikely to happen since, by default the yyunputgrow member variable is set to one, enabling the unput buffer to grow if it needs to.

The initial size of the unput buffer can be specified in the Lex source file by defining YYUNPUT_SIZE in the declarations section.  If you do not define YYUNPUT_SIZE, then it assumes the same value as that defined for YYTEXT_SIZE.  This in turn has a default value of 100.  Note that YYTEXT_SIZE specifies the initial size of the input buffer yytext.  The example below specifies an initial unput buffer 1000 characters in size:

%{

#define YYUNPUT_SIZE 1000

%}

This function is also responsible for decrementing the yylineno member variable.  Whenever yyunput receives a newline character \n it decrements the variable by 1.

Compatibility

It is not common to redefine the yyunput function.  For compatibility reasons though, yyunput can also be defined as the macro unput.  In the generated lexical analyser there is a small section of code, similar to that below which then converts this macro into the function yyunput:

#ifdef unput

void yyunput(int ch)
{

    unput(ch);
}

#endif

The same also applies to the yymunput function, although not for multiple instance lexical analysers.

If you do redefine yymunput in your source code then, by default, all multiple lexical analysers will make use of the new function.  As an alternative, you can define a new function with the same prototype and assign it to the yymunput member of the yymlex_t structure.  You can do this either statically or at run time.  To do it statically, you need to define YYMUNPUT as the function name in your Lex source file.  This needs to be done in the declarations section:

%{

void myunput(yymlex_t *yy, int ch)
{

    /* do something */
}
#define YYMUNPUT myunput

%}

See Also

yylexer::yyinput, yylexer::yyoutput, yylexer::yyunputgrow


yylexer::yyunputoverflow

#include <clex.h>

virtual void yyunputoverflow( );

Remarks

This yyunputoverflow member function is called if the lexical analysers unput buffer ever overflows.  This can be for one of three reasons:

1	The unput buffer is not growable, and has simply overflowed.

2	The unput buffer is growable, however the memory block requested by the lexical analyser is simply too big to fit in an internal representation.  For instance, in 16-bit DOS applications, the largest block of memory that can be allocated is 65,535 bytes in size.

3	The unput buffer is growable, however the computer has run out of memory. Note this is not very likely on modern machines.

The default implementation of yyunputoverflow simply outputs the error message "lex unput buffer overflow (size)\n", where size is the current size of the buffer.  The error message is sent to the stream specified by the yyerr member variable.  Overrride this function in your ALex generated lexical analyser if you wish to handle unput buffer overflow reporting differently.  Note that when yyunputoverflow returns, the calling lexical analyser function (yylex) will call the standard C library function exit immediately with the exit code EXIT_FAILURE.

See Also

yylexer::textoverflow, yylexer::yyunputgrow


yylexer::yywrap

#include <clex.h>

virtual int yywrap( );

Return Value

Nonzero if there is no more input for the lexical analyser to process, otherwise zero.

Remarks

The yywrap member function is called by the lexical analyser whenever it inputs an EOF as the first character when trying to match a regular expression.  The function returns a nonzero value to indicate that there is no more input for the lexical analyser to process, otherwise zero.  The default implementation of yywrap simply returns 1, i.e. there is no more input to process:

int yylexer::yywrap(void)
{

    return 1;
}

Override yywrap in your ALex generated lexical analyser if you wish to return a value of zero and process more input.  Note that it is your responsibility to ensure that this new input is available for the lexical analyser.  For instance, this may involve assigning a new stream to the yyin member variable.

The processing of more input is not the only reason for overriding yywrap.  For instance you can use the function to generate lexical analysis results.  Taking a simple example, a word count program, you can use the yywrap function to print out the number of words that have been counted:

int yylexer::yywrap()
{

    printf(“word count: %d\n”, wc);

    return 1;
}

See Also

yylexer::yyinput


yylexer::yybegin

#include <clex.h>

void yybegin( int state );

Parameter	Description

state	The new start state to place the lexical analyser in, or zero to return to the default start state.

Remarks

The yybegin member function changes the lexical analysers start state to the requested state.  Start states are introduced in a Lex source file through the use of the %start keyword.  As an alternative to passing INITIAL as a parameter to return to the default start state, it is possible to simply pass the value zero.

Compatibility

The yybegin member function is also implemented as the BEGIN macro.  The macro does not have any trailing parentheses.  An example of its use is shown below:

BEGIN aa;

See Also

yylexer::yynewline, yylexer::yystart


yylexer::yyecho

#include <clex.h>

void yyecho( );

Remarks

The yyecho member function echoes the matched string.  It does this by passing each character in the matched string to the yyoutput member function.  This is the function that is called by the lexical analyser whenever it cannot match a character.  The function simply copies the character to the stream specified by the yyout member variable.  By default, this is assigned to the standard C output stream stdout.  To redirect output elsewhere you can either assign a new stream to yyout, or override yyoutput in your ALex generated lexical analyser class.

Compatibility

The yyecho member function is also implemented as the ECHO macro.  The macro does not have any trailing parentheses.  An example of its use is shown below:

ECHO;

See Also

yylexer::yyoutput


yylexer::yyless

#include <clex.h>

void yyless( int length );

Parameter	Description

length	The new length of the matched string.  This must be greater than or equal to zero, and less than or equal to the current length of the string.

Remarks

The yyless member function changes the length of the matched regular expression string.  The new length must be greater than or equal to zero and less than or equal to the current length of the string.  Any characters that are not wanted are backed up.  They are removed one by one from the end of the string and passed to the yyunput member function.

Be careful of setting the length of the string to zero.  Unless you are planning on doing something else as well (such as changing start states) this will cause the lexical analyser to match the same expression forever:

abc     { yyless(0); /* forever */ }

An alternative to using yyless is to use the right context operator /.  The two regular expressions below, with their associated actions both match the same string:

abc/de  { ... }

abcde   { yyless(3); ... }

The choice whether to use the lookahead operator or yyless is one of taste.  It is worth noting that using yyless is slightly faster than using the lookahead operator.  Having said that the operator provides a clearer specification of the string that is actually matched.

See Also

yylexer::yymore


yylexer::yymore

#include <clex.h>

void yymore( );

Remarks

The yymore member function causes the next matched string to be concatenated on the end of the currently matched string.  The resultant string is stored in the yytext member variable.  Note also that the yyleng member variable holds the combined length of the two strings.

See Also

yylexer::yyless


yylexer::yynewline

#include <clex.h>

void yynewline( int newline );

Parameter	Description

newline	Nonzero if the previous character read by the lexical analyser was a newline character, otherwise zero.

Remarks

The yynewline member function specifies whether the last character read by the lexical analyser was a newline character or not.  The lexical analyser uses this value when it tries to match the next string.  Regular expressions starting with the leading newline operator ^ will be preferred over those which don't.

Ordinarily, the lexical analyser decides whether the last character read was a newline character or not.  It does this after a string has been matched, but before any action associated with the regular expression has been called.  This is to enable the use of the yynewline function in an action.

The yyless member function destroys the result of any call to the yynewline member function, so if you call both functions in an action, make sure that the yynewline call goes after the yyless call.

See Also

yylexer::yybegin


yylexer::yyreject

#include <clex.h>

void yyreject( );

Remarks

The yyreject member function forces the lexical analyser to re-scan the input looking for another regular expression to match.  The function has some limitations on where it can be used, however these are not generally restrictive.  If the function is called from elsewhere then it will have no effect.  The yyreject function can be:

1	Called from an action.

2	Called from some function, but program flow of control must return to an action at some point.

The yyreject member function does not take effect immediately, but rather when control returns from the action to the lexical analyser.  If a return statement is executed in an action, then the results of any call to yyreject will be lost.

The yyreject member function does not cause the input to be re-scanned as such, but rather the result of the previous match to be re-examined.  The result of any re-scanning also depends upon whether the -r option has been specified for ALex or not.  If so then it is possible for the same string to be matched by more than one regular expression.  The regular expressions following the order in which they were defined in the Lex source file.  Otherwise the length of the next matched string will always be less than the current one.

If a string of length one has been matched, and any other regular expressions matched if the -r option has been specified, then the character will be matched by the lexical analysers default rule.  This will simply call yyoutput with the character in question.

An example best illustrates how the yyreject member function behaves.  The example below matches the string "fu", and one or more letters.  In each action there is a call to the yyreject function:

%%

"fu"        { yyreject(); }

[a-zA-Z]+   { yyreject(); }

If the character stream "fubar" is now input to the resultant lexical analyser, and the -r option has been specified, the following strings are matched:

"fubar", "fuba", "fub", "fu", "fu", "f", "f", "ubar", "uba", "ub", "u", "u", "bar", "ba", "b", "b", "ar", "a", "a", "r", "r"

Notice that the string "fu" is matched twice in succession.  In the first case it is matched by the rule for "fu", and in the second case the rule for one or more letters.  The characters "f", "u", "b" etc. are also matched twice in succession.  However here the reasons are different.  In the first case they are matched by the rule for one or more letters, and in the second case the default rule.

If the -r option is not specified, then the matched strings are similar, however there is one important difference.  The string "fu" is only matched once:

"fubar", "fuba", "fub", "fu", "f", "f", "ubar", "uba", "ub", "u", "u", "bar", "ba", "b", "b", "ar", "a", "a", "r", "r"

If, on the other hand, there are no calls to the yyreject function, then the following strings are matched:

"fu", "bar"

Compatibility

The yyreject member function is also implemented as the REJECT macro.  The macro does not have any trailing parentheses.  An example of its use is shown below:

"fu"        { REJECT; }

In traditional Lex, this macro will cause an immediate jump to the relevant reject handling code.  This means that no code following the REJECT macro will be executed.  In traditional YACC, due to implementation details, this macro can only be placed in an action itself.  It cannot be placed in a function called from an action.  Note that this does not apply to ALex, where REJECT can be placed anywhere that yyreject can.


yylexer::yyunputcount
yyunputcount

#include <clex.h>

int yyunputcount( );

Return Value

The number of characters stored in the unput buffer.

Remarks

The yyunputcount member function returns the number of characters stored in the unput buffer.  The unput buffer is used to hold any excess characters that the lexical analyser may have read whilst trying to match a regular expression.

See Also

yylexer::yyinput, yylexer_yyunput


yyflexer

#include <clex.h>

class yyflexer : public yylexer;

Remarks

The yyflexer class provides the basic functionality for a default (fast) C++ lexical analyser generated by ALex.  It is derived from yylexer, which provides most of the functionality.  The yyflexer class has an implementation of the yylex pure virtual function.  It also has a number of extra member variables which hold table data used by yylex.

The yyflexer class is an abstract class so you cannot create an object from this class directly.  The pure virtual function which makes yyflexer an abstract class is yyaction.

If a default (fast) lexical analyser is generated by ALex, then the generated class is derived from yyflexer.  The generated class provides an implementation of the yyaction pure virtual function.

See Also

yylexer, yylexer Members


yyclexer

#include <clex.h>

class yyclexer : public yylexer;

Remarks

The yyclexer class provides the basic functionality for a compact C++ lexical analyser generated by ALex.  It is derived from yylexer, which provides most of the functionality.  The yyclexer class has an implementation of the yylex pure virtual function.  It also has a number of extra member variables which hold table data used by yylex.

The yyclexer class is an abstract class so you cannot create an object from this class directly.  The pure virtual function which makes yyclexer an abstract class is yyaction.

If a compact lexical analyser is generated by ALex, then the generated class is derived from yyclexer.  The generated class provides an implementation of the yyaction pure virtual function.

See Also

yylexer, yylexer Members


License

Please read the license agreement below to find out whether you need to register your copy of Parser Generator.  If so, then you will need to fill in an order form and send it to Bumble-Bee Software.  The details of how to do this are given in the ordering information help topic:

License Agreement

Ordering Information

Parser Generator Order Form


Ordering Information

The registration fee for Parser Generator is 50 US dollars or 30 UK pounds per copy.  To find out whether you need to register your copy of Parser Generator, please see the License Agreement.

You can pay for your copy of Parser Generator by one of three methods:

1	Cheque.

2	Money/Postal Order.

3	Credit Card.

In each case you will need to print out and complete a Parser Generator Order Form.  If you are paying by credit card you will need to give the card details.  That is, the credit card number, the card holders name and the expirey date of the card.  Then simply post the order form, along with the cheque or money/postal order where necessary to:

Bumble-Bee Software Limited

38 Stradbroke Grove

Buckhurst Hill

Essex

IG9 5PF

England

If posting from outside the UK it is preferable to send the letter by Airmail.  You will need to look up the cost of posting to the UK from your respective county.  Airmail letters typically have "Airmail" written in the left hand corner (or "Par Avion" if you are French) to distinguish them from surface mail.

When you have registered your copy of Parser Generator you will receive a License Agreement detailing your serial number and the number of licences you have purchased along with an invoice for your records.

For UK companies note that Bumble-Bee Software Limited is NOT currently VAT registered.  However to check on the exact status please visit the Bumble-Bee Software web page at www.bumblebeesoftware.com.

See Also

License Agreement, Parser Generator Order Form


License Agreement

If you are a student, or a member of an academic community, then you are automatically licensed to use Parser Generator.  This academic license remains in effect only whilst you remain a student, or a member of an academic community.  If during this time though, you use Parser Generator for commercial purposes then you will need to purchase a license.

Otherwise, you are licensed to use Parser Generator for a period of 30 days, starting from the date that you installed the software.  If you use the software after this evaluation period then you are required to purchase a license.  Failure to do so will place you in violation of English and international copyright laws.

One licensed copy of Parser Generator may either be used by a single person who uses the software personally on one or more computers, or installed on a single workstation used non-simultaneously by multiple people, but not both.

You may access a copy of Parser Generator through a network, provided that you have obtained licenses for the software covering all workstations that will access the software through the network.  For instance, if 8 different workstations will access Parser Generator on the network, each workstation must be licensed to use Parser Generator, regardless of whether they use Parser Generator at different times or concurrently.

A Parser Generator license costs 50 US dollars or 30 UK pounds per copy.  For information on how to purchase the license(s) you require see Ordering Information.

You may freely distribute the Parser Generator software, provided that:

1	Such distribution includes only the original archive(s) supplied by Bumble-Bee Software Limited.

2	No money is charged to the person receiving the software, beyond reasonable cost of packaging and similar.

You may not reverse engineer, de-compile, or disassemble Parser Generator.

Disclaimer of Warranty

This software and the accompanying files are provided "as is" without warranties as to performance of merchantability or any other warranties whether expressed or implied.  No warranty of fitness for a particular purpose is offered.  The entire risk arising out of use or performance of Parser Generator remains with you.  In no event shall Bumble-Bee Software Limited be liable for any damages whatsoever (including, without limitation, damages for loss of business profit, business interruption, loss of business information, or any other pecuniary loss) arising out of the use or inability to use Parser Generator.

See Also

Ordering Information, Parser Generator Order Form


Parser Generator Order Form

Name:	________________________________________________

Organisation:	________________________________________________

Address:	________________________________________________

________________________________________________

________________________________________________

City/Town:	________________________________________________

State/County:	________________________________________________

Zip Code/Postcode:	________________________________________________

Country:	________________________________________________

Telephone Number:	________________________________________________

Fax Number:	________________________________________________

E-mail Address:	________________________________________________

Date (dd/mm/yy):	___ / ___ / ___

Licenses

Unit Price:	$50 US / £30 UK

Quantity:	_____________

Total:	_____________

Method of Payment:	  Cheque

  Money/Postal Order

  Credit Card *

Credit Card Details

Card Type:	  VISA

  MasterCard

Card Number:	________________________________________________

Card Holder's Name:	________________________________________________

Expires End (mm/yy):	___ / ___

Any Comments or Suggestions:

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

__________________________________________________________________________

* Bumble-Bee Software Ltd. is currently in the middle of setting up a credit card transaction processing capability, so if you order by credit card you may find a delay occurs before the transaction takes place.