The two modes of TeX engines: INI mode and production mode

All TeX engines have two "modes" of operation: an "INI" mode and "production" mode—the latter is just the default (standard) mode used for the production of typeset material. INI mode is rather different and you need to use a command-line option to put a TeX engine into INI mode. You can see how to do that by issuing --help on the command line. For example, with LuaTeX if you do

luatex --help

you'll see many command-line options supported by that engine:

Usage: luatex --lua=FILE [OPTION]... [TEXNAME[.tex]] [COMMANDS]
   or: luatex --lua=FILE [OPTION]... \FIRST-LINE
   or: luatex --lua=FILE [OPTION]... &FMT ARGS
  Run LuaTeX on TEXNAME, usually creating TEXNAME.pdf.
  Any remaining COMMANDS are processed as luatex input, after TEXNAME is read.

  Alternatively, if the first non-option argument begins with a backslash,
  luatex interprets all non-option arguments as an input line.

  Alternatively, if the first non-option argument begins with a &, the
  next word is taken as the FMT to read, overriding all else.  Any
  remaining arguments are processed as above.

  If no arguments or options are specified, prompt for input.

  The following regular options are understood:

   --credits                     display credits and exit
   --debug-format                enable format debugging
   --draftmode                   switch on draft mode (generates no output PDF)
   --[no-]file-line-error        disable/enable file:line:error style messages
   --[no-]file-line-error-style  aliases of --[no-]file-line-error
   --fmt=FORMAT                  load the format file FORMAT
   --halt-on-error               stop processing at the first error
   --help                        display help and exit
   --ini                         be iniluatex, for dumping formats
   --interaction=STRING          set interaction mode (STRING=batchmode/nonstopmode/scrollmode/errorstopmode)
   --jobname=STRING              set the job name to STRING
   --kpathsea-debug=NUMBER       set path searching debugging flags according to the bits of NUMBER
   --lua=FILE                    load and execute a lua initialization script
   --[no-]mktex=FMT              disable/enable mktexFMT generation (FMT=tex/tfm)
   --nosocket                    disable the lua socket library
   --output-comment=STRING       use STRING for DVI file comment instead of date (no effect for PDF)
   --output-directory=DIR        use existing DIR as the directory to write files in
   --output-format=FORMAT        use FORMAT for job output; FORMAT is 'dvi' or 'pdf'
   --progname=STRING             set the program name to STRING
   --recorder                    enable filename recorder
   --safer                       disable easily exploitable lua commands
   --[no-]shell-escape           disable/enable system commands
   --shell-restricted            restrict system commands to a list of commands given in texmf.cnf
   --synctex=NUMBER              enable synctex (see man synctex)
   --utc                         init time to UTC
   --version                     display version and exit

Alternate behaviour models can be obtained by special switches

  --luaonly                      run a lua file, then exit
  --luaconly                     byte-compile a lua file, then exit
  --luahashchars                 the bits used by current Lua interpreter for strings hashing

See the reference manual for more information about the startup process.

Email bug reports to dev-luatex@ntg.nl.

Amongst those many options are two of relevance here:

--ini              be iniluatex, for dumping formats
--fmt=FORMAT       load the format file FORMAT

Here, LuaTeX refers to its INI mode by the name of iniluatex. So what does this iniluatex (INI mode) actually mean? A slight clue is the second command-line option shown above:

--fmt=FORMAT       load the format file FORMAT

INI mode? Format files? I'm confused...

Macro packages, such as LaTeX, are large collections of complex macros: loading and processing the raw text of all the constituent files does have a certain time and computation overhead; for example:

• reading and scanning tens of thousands of individual characters of text;
• parsing, processing and storing hundreds of macros/commands;
• initializing internal tables of fonts and hyphenation data.

All of this has to take place, and the results stored in memory, before the TeX engine is actually ready to process a user's TeX document which makes use of the macros provided by, for example, the Plain TeX or LaTeX macro collections. You can think of this as a TeX engine's bootstrapping process: essential preparations before it is ready to process your document.

Perhaps reflecting the technology of its day, TeX engines offer a shortcut to processing text files full of macro collections: loading pre-processed binary (format) files.

When placed into INI mode a TeX engine's task is not to perform typesetting but instead to:

1. load and initialize all of its primitive commands plus set up the very minimum of catcodes (and other codes) so that the TeX engine can start further processing.
2. once (1) is done it will then load and process a user-provided macro collection, referred to as a "format" (e.g., Plain TeX or the LaTeX macro collection).
3. after it has successfully completed (1) and (2) TeX will perform a "binary brain dump" of everything that has been loaded and processed into its memory and write it out into a binary format file (e.g., lualatex.fmt).

A format file is a binary file which stores a representation of TeX's "state of mind" after having initialized numerous internal data tables (including the primitives) and processed all the macro commands present in the user-supplied macro collection. TeX engines do this because it is faster to load a "pre-compiled" binary file rather than re-process the raw macros each time you want to run a TeX engine using, for example, Knuth's Plain TeX or the LaTeX macro package—or indeed any macro package designed as a "format".

When you process a document that is to be typeset by LaTeX, for example, the TeX engine does not re-read (re-process) all LaTeX's core macros but instead it reads-in a binary file called, for example, lualatex.fmt. The inner details of format files are specific to each TeX engine and each version of the macro package used to generate a .fmt file. For example, if you upgrade your LaTeX distribution (i.e., the core of LaTeX itself, not add-on packages) then you might need to generate a new .fmt file too.

Example

Within the LaTeX source code distribution you will find files with an .ini extension, for example lualatex.ini, which are used to generate .fmt files—those .ini files typically contain engine-specific initialization code and then proceed to input the core LaTeX macro files. For example, if you run LuaTeX with the following command line:

luatex --ini lualatex.ini

LuaTeX will write out a file called lualatex.fmt

At the time of writing, lualatex.ini contains the following code:

% tex-ini-files 2016-04-15: lualatex.ini
% Originally written 2008 by Karl Berry. Public domain.

\input luatexconfig.tex

\begingroup
  \catcode`\{=1 %
  \catcode`\}=2 %
  % Set up job name quoting before latex.ltx
  % Web2c pdfTeX/XeTeX quote job names containing spaces, but LuaTeX does
  % not do this at the engine level. The behaviour can be changed using
  % a callback. Originally this code was loaded via lualatexquotejobname.tex
  % but that required a hack around latex.ltx: the behaviour has been altered
  % to allow the callback route to be used directly.
  \global\everyjob{\directlua{require("lualatexquotejobname.lua")}}
\endgroup

\input latex.ltx

lualatex.ini starts by inputting a file called luatexconfig.tex which does some LuaTeX-specific initialization—as you might expect from an .ini (initialization) file! On the last line it inputs latex.ltx which causes LuaTeX to read and process the core LaTeX macros.

After LuaTeX finishes processing lualatex.ini it will write out the binary format file called lualatex.fmt. Once the LaTeX code has been compiled into a binary format, LuaTeX (or any other TeX engine) can very quickly read the .fmt file to bootstrap and initialize its internal tables of LaTeX's macro definitions, fonts and so forth—it does not have to undergo the lengthy process of reading and processing the many thousands of characters contained in the LaTeX macro package.

You can now instruct LuaTeX to typeset your LaTeX document yourfile.tex (using the lualatex format file) by issuing the command:

luatex --fmt=lualatex yourfile.tex

.fmt files are usually specific to a particular engine: e.g., LuaTeX's .fmt files are generally not portable for use with a different TeX engine: e.g., pdfTeX or XeTeX. Furthermore, if the LuaTeX executable is updated with new features it is quite possible that a .fmt file prepared with an older version of LuaTeX would not work with the new engine: you'd need to re-build (re-compile) the .fmt file.

If you try to use an incompatible .fmt file then a TeX engine will fail to load it and respond with the error:

Fatal format file error; I'm stymied

Similarly, if the core LaTeX macros are updated you might also need to rebuild the lualatex.fmt file too.

.fmt files can be prepared from any suitable macro package—not just LaTeX; for example, Knuth's original, and much simpler, "Plain" TeX macros can also be processed to generate a .fmt file. You could, if you had the time, write your own macro collection and produce a format file for that package—note that somewhere in your macro package you'd need to invoke the primitive command \dump which tells the TeX engine to do its "binary brain dump" and write out the .fmt file.

However, the vast majority of users never need to worry about the "INI" mode of TeX engines, or the process of generating .fmt files: Overleaf and the TeX Live distribution shield you from those details—unless you need to go looking for them.