Preprocessor Token Macros

Source Insight contains its own preprocessor, which is used when parsing files. The preprocessor macros are called Token Macros. They are token substitutions that occur as Source Insight parses a file. They allow Source Insight to handle any special language keywords not known to Source Insight's parsers, and to handle special C/C++ preprocessor substitutions that would otherwise confuse it.

Preprocessor macros are normally not expanded by Source Insight

Source Insight does not expand C/C++ preprocessor macros when it parses your files. Because of this, certain preprocessor macros and constants can fool Source Insight. Therefore, token macros are used to let Source Insight selectively expand some preprocessor substitutions.

For example, the STDMETHOD(methodname) macro is used to declare a COM method function. Source Insight ships with a default C/C++ token macro file (c.tom) that has entries for this macro, and many others.

Token Macro Files

The token macros are listed in a file with a .tom extension. The global token macro file resides in the Source Insight program directory. The project-specific token macro file, if any, is stored in project's data directory. The project token macro file is combined with the global file, with the project macros taking precedence.

Token Macro Syntax

A token macro file consists of token macros, one per line. The format of a token macro is:

macroname    <no text here means macro is a no-op>

macroname    substituted text here

macroname(parameter list)    substituted text with parameter names

macroname(parameter)    text##parameter // concatenates text

; comments begin with a semicolon

 

Some examples of token macros:

MyStructure(sname)    struct sname

NoOperation

BuildName(name1, name2)    name1##name2

 

Each built-in language parser has a corresponding token macro file. The name of the token macro file for each language is summarized below:

Editing Token Macros

If you want to change the token macros, simply open the token macro file, make your changes, and save the file. Source Insight will recognize that the token macros have changed for the appropriate language. Open files are automatically re-parsed.

Save a token macro file to get Source Insight to recognize the macros

When you edit a token macro file, you must save it to disk before Source Insight will re-parse your open files. However, Source Insight will not automatically re-parse your whole project. You should make all your changes to the token macro file first, then use the Rebuild Project command to re-parse your whole project. Until your project is re-parsed, the symbol information stored in Source Insight's symbol database will not reflect the changes you made to your token macros.

Project Specific Token Macros

Each project can have its own set of token macro files. Source Insight does not create them automatically, but you can yourself. A project token macro file is saved in the project's data directory. When Source Insight parses a source file, it combines the project token macros with the global set saved in the Source Insight pro­gram directory. The project token macros take precedence over the global ones. By adding project specific token macros, you can tailor the token macro expansion for each project individually.