Mar 21, 2022

Pre-release of LaTeX for 2022-06-01 ready for testing

We submitted the latest pre-release version for the 2022-06-01 LaTeX kernel to CTAN recently. It should have made its way into TeX Live (both 2021 and the 2022 pre-test) for everyone by now, so it’s ready for user to try.

It already contains most updates to the LaTeX format that will become part of the standard in June; thus testing the them now will help identifying packages that need upgrading and finding any bugs lurking before the June release hits the streets; see below for how to do this in a safe way.

The highlights are briefly discussed below, further details plus smaller bug fixes and improvements not covered here can be found in the draft version of the LaTeX2e News Issue 35 newsletter.

New features in the kernel (highlights)

For users

One of the most significant changes in the code is one that users (hopefully) won’t notice: the definition of UTF-8 characters in pdfTeX. We have made these engine-protected, which should make processing them in expl3 a lot easier. The way the change has been set up means we don’t expect users to actually see a difference, but we do want to hear about how it is working. So get your complicated accents out and try them all over the place with pdfTeX. (LuaTeX and XeTeX users won’t even need to worry.) One thing that goes with this change is that we have updated the definition of \MakeUppercase and \MakeLowercase, so they use Unicode-aware expl3 code ‘behind the scenes’. Again, this should be transparent to users, but it would be good to hear how it works in the wild.

More visible are a small set of new commands we are adding in this pre-release. We are moving \fpeval and \inteval from the xfp package directly into the kernel. We are also adding \ExpandArgs and \UseName, ideas again taken from expl3. These commands allow control of expansion, so you can do a range of basic expansion tasks in document commands, without needing TeX programming or using expl3. For example, you can now do

\NewDocumentCommand\newcopyedit{mO{red}}
  {\newcounter{todo#1}%
   \ExpandArgs{c}\NewDocumentCommand{#1}{s m}%
     {\stepcounter{todo#1}%
      \IfBooleanTF {##1}%
         {\todo[color=#2!10]%
            {\UseName{thetodo#1}: ##2}}%
         {\todo[inline,color=#2!10]%
            {\UseName{thetodo#1}: ##2}}%
     }%
  }

which can create a document command by name: something people have been asking for many years.

One more addition from expl3 ideas is \IfBlankTF: this can test if it has been given an entirely blank argument. Here, blank means empty or just spaces: this comes up surprisingly often.

For package authors

Package authors might well choose to use the features described above. There is though one big change aimed squarely at packages: keyval-based option handling. We have introduced a new set of commands, \DeclareKeys, \ProcessKeyOptions and \SetKeys, which can be used to create keyvals and to use them as package options. \DeclareKeys uses the same approach as the expl3 function \keys_define:nn to create keys: each key has one or more ‘properties’ which control how it behaves. We have provided a small number of common properties at this stage, for storing key values in a macro, for setting a switch and for executing some code. We have also provided a method to define the usage of keys: whether they are only available at package loading time, in the preamble or can also be used in the document body.

\DeclareKeys
  {
    draft.if    = @mypkg@draft      ,
    draft.usage = preamble          ,
    name.store  = \@mypkg@name      ,
    name.usage  = load              ,
    command.code = \showtokens{#1}
  }

Package options can then be processed using \ProcessKeyOptions, which will set keys defined by \DeclareKeys. Keys can also be set in the document body using \SetKeys. All of these new commands take an optional argument to define the key family: the namespace for the keys. This is by default the name of the current package file.

More advanced key types can be created directly using expl3 (l3keys): the keys created by \DefineKeys are standard expl3 keys.

Unlike traditional package options, loading a package using \ProcessKeyOptions multiple times with different options will not lead to a clash error. Instead, the options will simply be passed to the same key handler. This means that the package author can control how repeatedly setting the same keys is handled.

`\DocumentMetadata`

There is one more addition to the kernel for use at the top of the document: \DocumentMetadata. It is the trigger to tell LaTeX that the current document is new/modern document that should use concepts not available in traditional LaTeX2e documents, for example, everything developed for the PDF tagging project.

This new command has to come first in a LaTeX document, before \documentclass. It is used, as the name suggests, to set metadata about the entire document, but while the PDF tagging project is being worked on, it is also used to enable new code from this project (see the discussion of latex-lab below).

Even if used with an empty argument, it will currently already show some effect, by loading the new pdfmanagement that offers a reliable and consistent interfaces for packages (such as hyperref, media9, etc.) that need to set or query PDF resources, so that they can coexist without overwriting their settings in random ways.

The hyperref package, for example, will switch to a new driver with extended possibilities, if the \DocumentMetadata declaration was used, you can see this with the following simple document, that does shows the new default link colors of the new driver.

\DocumentMetadata{}

\documentclass{article}
\usepackage{hyperref}

\begin{document}
\tableofcontents
\section{A link}  \url{https://www.latex-project.org}
\end{document}

The LaTeX-lab bundle

This is a new space to explore ideas that we think may get added to LaTeX. These experimental ideas are only activated if the right key is present in the argument to \DocumentMetadata, i.e., they do not affect users who do not wish to experiment with new features. For example, the key testphase can be used to enable code that has been developed for the PDF tagging project. We currently are in testphase II of the tagging project, and that can be activated using

\DocumentMetadata{testphase = phase-II}

This then loads tagpdf and additional support code from latex-lab and produces documents that have paragraph text and footnotes tagged; further document elements, e.g., headings and lists will follow soon as part of this phase.

The latex-lab space may also hold code that we are in the process of moving to the kernel, but that we first want to tests without the danger of disruption, for example, at the moment it hosts the new management code for independent marks, which will either in June of by the end of the year move to the format.

It is possible that some ideas from the lab will be dropped or modified: anyone using them should be mindful of that. However, we expect that most will eventually move to the kernel. Tagging will always require \DocumentMetadata to be present, but over time the need to activate it with the testphase key will be removed.

Outlook

We expect to produce another pre-release around the end of April, which will give us time to finish the release by June. Please help with the testing

Processing your documents with the pre-release is straightforward. All you have to do is to replace the invocation command by appending -dev to the executable, e.g., on the command line you would run

pdflatex-dev myfile or
lualatex-dev myfile or
xelatex-dev myfile

instead of using pdflatex, lualatex or xelatex. If you use an integrated editing environment, then it depends on the system how to configure it to use an alternative format; but in any case the necessary modification should be straightforward.

Enjoy — Joseph & Frank

LaTeX News