Default FreeSerif font not found by xelatex due to specifying Extension

[apteryks]

Describe the bug

When specifying the latex engine as latex_engine = 'xelatex', the generated .tex file has the following font configuration:

\setmainfont{FreeSerif}[
  Extension      = .otf,
  UprightFont    = *,
  ItalicFont     = *Italic,
  BoldFont       = *Bold,
  BoldItalicFont = *BoldItalic
]
\setsansfont{FreeSans}[
  Extension      = .otf,
  UprightFont    = *,
  ItalicFont     = *Oblique,
  BoldFont       = *Bold,
  BoldItalicFont = *BoldOblique,
]
\setmonofont{FreeMono}[
  Extension      = .otf,
  UprightFont    = *,
  ItalicFont     = *Oblique,
  BoldFont       = *Bold,
  BoldItalicFont = *BoldOblique,
]

I was puzzled to encounter the error:

! Package fontspec Error: The font "FreeSerif" cannot be found.

Although the font-gnu-freefont package was installed on my system and visible via fc-list:

fc-list | grep 'FreeSerif.*otf'
/gnu/store/pdj0hwrri2riw5av1k0dsvaxfda891yw-profile/share/fonts/opentype/FreeSerifBold.otf: FreeSerif:style=Bold,получерен,negreta,tučné,fed,Fett,Negrita,Lihavoitu,Gras,Félkövér,Grassetto,Vet,Halvfet,Pogrubiony,Negrito,gros,Обычный,Fet,ตัวหนา,Kalın,huruf tebal,жирний,Polkrepko,treknraksts,pusjuodis,ضخیم,đậm,Lodia,धृष्ट
/gnu/store/pdj0hwrri2riw5av1k0dsvaxfda891yw-profile/share/fonts/opentype/FreeSerif.otf: FreeSerif:style=Regular,нормален,normal,obyčejné,Mittel,µεσαία,Normaali,Normál,Normale,Gemiddeld,odmiana zwykła,Обычный,Normálne,ปกติ,menengah,прямій,Navadno,vidējs,normalusis,عادی,vừa,Arrunta,सामान्य
/gnu/store/pdj0hwrri2riw5av1k0dsvaxfda891yw-profile/share/fonts/opentype/FreeSerifItalic.otf: FreeSerif:style=Italic,курсивен,cursiva,kurzíva,kursiv,Λειψίας,Kursivoitu,Italique,Dőlt,Corsivo,Cursief,kursywa,Itálico,cursiv,Курсив,ตัวเอียง,İtalik,kursif,Ležeče,kursīvs,kursivas,nghiêng,Etzana,तिरछा
/gnu/store/pdj0hwrri2riw5av1k0dsvaxfda891yw-profile/share/fonts/opentype/FreeSerifBoldItalic.otf: FreeSerif:style=Bold Italic,получерен курсивен,negreta cursiva,tučné kurzíva,fed kursiv,Fett-Kursiv,Negrita Cursiva,Lihavoitu Kursivoi,Gras Italique,Félkövér dőlt,Grassetto Corsivo,Vet Cursief,Halvfet Kursiv,Pogrubiona kursywa,Negrito Itálico,gros cursiv,Обычный Курсив,Tučná kurzíva,Fet Kursiv,ตัวเอียงหนา,Kalın İtalik,huruf tebal kursif,жирний курсив,Polkrepko Pežeče,treknais kursīvs,pusjuodis kursyvas,nghiêng đậm,Lodi etzana,धृष्ट-

After asking in #latex on libera.chat (IRC), it was diagnosed to be because when specifying Extension, the font is looked by its file name in the TEXMF tree by kpathsea which fails here because the font is not installed to the TEXMF tree. Removing the Extension lines above, the font is correctly found by xelatex through fontconfig.

I'm not sure what the best course of action is here. Could dropping the Extension line work for everyone (including those using the texlive package for the GNU FreeFont rather than as a "regular" font package) ?

How to Reproduce

It depends of your distribution, but on Debian you'd need to have the fonts-freefont-otf package installed and not have the texlive-fonts-extra-links package installed to trigger the issue.

$ git clone https://github.com/ipython/ipython
$ cd ipython
$ cd docs
$ echo "latex_engine = 'xelatex'" >> source/conf.py
$ pip install -r requirements.txt
$ make pdf

Expected behavior

The fonts should be found by xelatex.

Your project

N/A

Screenshots

No response

OS

GNU/Linux

Python version

3.9.9

Sphinx version

4.2.0

Sphinx extensions

No response

Extra tools

No response

Additional context

No response

[cadr]

Is there a reason we need to specify the font extension at all? Maybe I'm doing something silly, on RHEL8 it doesn't look like they have the OTF version of these fonts, but rather at TTF. If I remove the line in the produced .tex file that specifies the extension everything works well for me.

[madjxatw]

Is there a reason we need to specify the font extension at all? Maybe I'm doing something silly, on RHEL8 it doesn't look like they have the OTF version of these fonts, but rather at TTF. If I remove the line in the produced .tex file that specifies the extension everything works well for me.

Same issue here, is there anyway to remove the extension entry.

[picnixz]

I haven't checked but I assume that this is to always load the same fonts. If you have a .ttf and an .otf file, I am not sure which one will be used by default.

I think @jfbu may have better insight since he's maintaining latex-related parts.

[jfbu]

The fontspec documentation has always been very confusing to me regarding font selection. Hurdles include that XeLaTeX and LuaLaTeX behave differently, and that XeLaTeX behaves differently depending on whether on Linux, Windows, or macos. The current usage of Extension=.otf ensured things worked on TeXLive based installations. (@picnixz the .ttf or .otf potential induced differences were not the rationale, as I expected -- perhaps mistakenly -- for the rendering to be the same anyhow).

The problem for maintenance is difficulty to access resources for testing, e.g. at this time I can only access macos, at some periods of the year I can additionally test on Ubuntu, and generally speaking I do not have any access whatsoever to a Windows machine.

As far as I understand the fontspec documentation, section 2.2 "By file name", it could be that replacing Extension=.otf by Path would work here. If we can drop it altogether all the better but I am afraid then that it could fail on systems where FreeSerif is not installed system-wide but only in the TeXLive tree.

I thus propose we replace Extension=.otf by Path.

Sorry for late about this, feedback welcome whether this works before I make a PR. Or someone, please make a suitable PR, thanks.

[jfbu]

Also, it could be that by adding this to your conf.py

import sphinx.builders.latex
sphinx.builders.latex.constants.XELATEX_DEFAULT_FONTPKG = r'''
\setmainfont{FreeSerif}[
  UprightFont    = *,
  ItalicFont     = *Italic,
  BoldFont       = *Bold,
  BoldItalicFont = *BoldItalic
]
\setsansfont{FreeSans}[
  UprightFont    = *,
  ItalicFont     = *Oblique,
  BoldFont       = *Bold,
  BoldItalicFont = *BoldOblique,
]
\setmonofont{FreeMono}[
  UprightFont    = *,
  ItalicFont     = *Oblique,
  BoldFont       = *Bold,
  BoldItalicFont = *BoldOblique,
]
'''

you can monkey-patch the current config and get rid of Extension=.otf.

Again, sorry for late answering.

[jfbu]

And I completely forgot to mention that you can also use the official interface via latex_elements['fontpkg'] in conf.py, like this:

latex_elements = {
    'fontpkg': '''
\setmainfont{FreeSerif}[
  UprightFont    = *,
  ItalicFont     = *Italic,
  BoldFont       = *Bold,
  BoldItalicFont = *BoldItalic
]
\setsansfont{FreeSans}[
  UprightFont    = *,
  ItalicFont     = *Oblique,
  BoldFont       = *Bold,
  BoldItalicFont = *BoldOblique,
]
\setmonofont{FreeMono}[
  UprightFont    = *,
  ItalicFont     = *Oblique,
  BoldFont       = *Bold,
  BoldItalicFont = *BoldOblique,
]
''',
 }

again to get rid of the Extension=.otf or replace it by something else.

[picnixz]

the .ttf or .otf potential induced differences were not the rationale, as I expected -- perhaps mistakenly -- for the rendering to be the same anyhow

AFAIK, there are some subtle differences related to the underlying OS as well. IIRC, OTF and TTF differ by the way they render glyphs and you might have some slight differences (e.g., darkness levels). I learned that OTF is usually better when you physically print your document because they use cubic Bézier whereas TTF uses quadratic Bézier curves.

---

With https://github.com/sphinx-doc/sphinx/issues/10347#issuecomment-1647984313, I think we can avoid having a PR on our side (and avoid the needs to test). But maybe we should write some documentation for that trick.