Perform font fallback

[nulano]

Fixes #4808.

Add a new type, ImageFont.FreeTypeFontFamily(font1, font2, ..., layout_engine=layout_engine), that can be used with ImageDraw.text*(...) functions performing font fallback. Font fallback is done per cluster with Raqm layout (similar to Chromium) and per codepoint with basic layout.

This PR is far from complete, several TODOs:

• [ ] Font families have only a minimal API so far, e.g. retrieving metrics or setting font variations should be supported
• [ ] Maybe add a wrapper similar to ImageFont.truetype(...), perhaps ImageFont.truetype_family(...)?
• [ ] Lots of tests
• [ ] Documentation

I would like to get some feedback, both on the API and the implementation, before working on the TODOs above. A dev build for Windows is available from the artifact here: https://github.com/nulano/Pillow/actions/runs/8583137967

A few examples (click to expand):

All examples use this helper block:

from PIL import Image, ImageDraw, ImageFont

im = Image.new("RGBA", (500, 200), "white")
draw = ImageDraw.Draw(im)
def line(y, string, font, name, **kwargs):
  draw.text((10, y), name, fill="black", font=font, **kwargs)
  draw.text((300, y), string, fill="black", font=font, **kwargs)

example()

im.show()

Combining Latin, symbols, and an emoji:

def example():
  s = "smile ⊗ 😀"

  times = ImageFont.truetype("times.ttf", 24)
  segoe_ui_emoji = ImageFont.truetype("seguiemj.ttf", 24)
  segoe_ui_symbol = ImageFont.truetype("seguisym.ttf", 24)
  family = ImageFont.FreeTypeFontFamily(times, segoe_ui_emoji, segoe_ui_symbol)

  line(30, s, times, "Times New Roman", anchor="ls", embedded_color=True)
  line(80, s, segoe_ui_emoji, "Segoe UI Emoji", anchor="ls", embedded_color=True)
  line(130, s, segoe_ui_symbol, "Segoe UI Symbol", anchor="ls", embedded_color=True)
  line(180, s, family, "Font Family", anchor="ls", embedded_color=True)

Combining Arabic, Greek, Latin, and a symbol:

def example():
  s = "ية↦α,abc"

  scriptin = ImageFont.truetype(r"C:\Users\Nulano\AppData\Local\Microsoft\Windows\Fonts\SCRIPTIN.ttf", 24)
  segoe_ui = ImageFont.truetype("segoeui.ttf", 24)
  segoe_ui_symbol = ImageFont.truetype("seguisym.ttf", 24)
  family = ImageFont.FreeTypeFontFamily(scriptin, segoe_ui, segoe_ui_symbol)

  line(30, s, scriptin, "Scriptina", direction="ltr", anchor="ls")
  line(80, s, segoe_ui, "Segoe UI", direction="ltr", anchor="ls")
  line(130, s, segoe_ui_symbol, "Segoe UI Symbol", direction="ltr", anchor="ls")
  line(180, s, family, "Font Family", direction="ltr", anchor="ls")

Combining characters are treated as part of a single cluster (with Raqm layout):

def example():
  import unicodedata

  s = " ̌,ῶ,ω̃,ώ,ώ, ́,á,č,č"
  for c in s:
    print(unicodedata.name(c))

  le = ImageFont.Layout.RAQM  # or ImageFont.Layout.BASIC
  scriptin = ImageFont.truetype(r"C:\Users\Nulano\AppData\Local\Microsoft\Windows\Fonts\SCRIPTIN.ttf", 24, layout_engine=le)
  dubai = ImageFont.truetype(r"DUBAI-REGULAR.TTF", 24, layout_engine=le)
  gentium = ImageFont.truetype(r"C:\Users\Nulano\AppData\Local\Microsoft\Windows\Fonts\GentiumPlus-Regular.ttf", 24, layout_engine=le)
  family = ImageFont.FreeTypeFontFamily(scriptin, dubai, gentium, layout_engine=le)

  line(30, s, scriptin, "Scriptina", anchor="ls")
  line(80, s, dubai, "Dubai", anchor="ls")
  line(130, s, gentium, "GentiumPlus", anchor="ls")
  line(180, s, family, "Font Family", anchor="ls")

Raqm layout:

Basic layout:

The string s contains:

SPACE
COMBINING CARON
COMMA
GREEK SMALL LETTER OMEGA WITH PERISPOMENI
COMMA
GREEK SMALL LETTER OMEGA
COMBINING TILDE
COMMA
GREEK SMALL LETTER OMEGA WITH TONOS
COMMA
GREEK SMALL LETTER OMEGA
COMBINING ACUTE ACCENT
COMMA
SPACE
COMBINING ACUTE ACCENT
COMMA
LATIN SMALL LETTER A
COMBINING ACUTE ACCENT
COMMA
LATIN SMALL LETTER C WITH CARON
COMMA
LATIN SMALL LETTER C
COMBINING CARON

[nissansz]

How to download the module or the build with below module?

AttributeError: module 'PIL.ImageFont' has no attribute 'FreeTypeFontFamily'

[nulano]

You can use a source install from the branch https://github.com/nulano/Pillow/archive/refs/heads/font-fallback.zip using the installation instructions: https://pillow.readthedocs.io/en/stable/installation.html#building-from-source

The previous Windows dev build has expired so I've re-run the CI for this branch here: https://github.com/nulano/Pillow/actions/runs/5568846705 You can download the dist-font-fallback-build.zip artifact from there and install the relevant wheel for your version of Python. (I've had to mark the test step as ignored because some crash test files seem to have been moved since.)

[nissansz]

Thank you. I tried. It works now. Will it be updated to new pillow version 10.0 too?

[nissansz]

If I want to use below method to draw text for each char. is it slower than fontfamily?

Iterate all chacters in a string, judge each character to see whether it is in the char list of a desire font, if not in the list, specify the backup font to draw the character, then continue

[nissansz]

If a character is not in char list of a font, it can be displayed by a backup font.

But some fonts are strange, the backupfont did not work too.

[mic-user]

Hi. Is there any progress?

[aclark4life]

Hi. Is there any progress?

@Mitchell-kw-Lee Looks like this is still a WIP, are you able to test the branch and report results? That may help …

[nulano]

This needs a pretty large rebase before it can cleanly merge with the main branch.

Currently, I see https://github.com/python-pillow/Pillow/pull/6926#discussion_r1099570782 as the biggest unresolved issue with this PR. I think I have an idea that could work, but I've not had time to work on this.

However, if you are able to test this PR as is (even though it is made for an older version of Pillow) and report whether it works / causes issues, it could perhaps be helpful.

[mic-user]

@nulano @aclark4life Errr, actually have no idea that the test process. Zero knowledge of git cowork test and manual installation stuff. And there is running environment that may impact own work. Well...seems some most of part works as the post at https://github.com/python-pillow/Pillow/issues/4808#issuecomment-1414075732. So, my suggestion is that, if possible, release current code base to public, and see and wait the bug report(include from me) that may helpful variety of sample code by report can check widely. Holding and being abandoned the prior effort/code are so shame which have potential..... sorry, just giving an idea is all i can do for now.

[nulano]

As I wrote above, this branch has a non-trivial merge conflict I haven't had time to resolve, even though I would like to get to it at some point.

If you'd like (keep in mind this is based on Pillow from a year ago), I can re-run the CI to generate new Windows wheels for easier installation. If you are on Linux/macOS, you'll need to figure out source installation (hint: https://pillow.readthedocs.io/en/stable/installation.html#building-from-source)

[mic-user]

@nulano Ah? You are going to give me aa package that can be installed by a command 'python -m pip install pillow==TESTVERSION --upgrade --user' or so? Then lets do it. I'll test it by applying piilow code in my code. And I can do feedback.(I'm running on Windows 11 64bit BUT Python 3.11.7- 32BIT, beware please.) Before we proceed, I need the syntax example to do that set a custom single font path for main usage and I will have another font for fallbacking(It should be https://github.com/python-pillow/Pillow/issues/4808#issuecomment-1414075732, right?). And those are Korean(part of CJK) and an open source symbol font. Send me the installation code

[nulano]

I've rerun the CI: https://github.com/nulano/Pillow/actions/runs/7765749734 You can download dist-font-fallback-build from the list of artifacts at the bottom, unzip it, and install with python -m pip install Pillow-9.5.0.dev0-cp311-cp311-win32.whl --user.

Example usage is at the top of this page (click on A few examples (click to expand) in the top comment) or https://github.com/python-pillow/Pillow/issues/4808#issuecomment-1414075732.

[mic-user]

@nulano okay, after fix version conflict, I have got the installation on my system now. BUT will comback early next week. Give me sometime.

[mic-user]

@nulano When I call the FreeTypeFontFamily I got the error <class 'AttributeError'>'FreeTypeFontFamily' object has no attribute 'getsize'. (Fortunately, there is no 'No FreeTypeFontFamily method found' stuff. Do I need to do something prerequsition part like ...remove some library or so? What should I do for now?

[radarhere]

FreeTypeFontFamily doesn't have getsize() - but that's not a method that needs to be added later, it's a method that has been transitioned out of handling fonts in the latest Pillow versions. I suggest you use getbbox() instead. You can read more at https://pillow.readthedocs.io/en/stable/deprecations.html#font-size-and-offset-methods

[mic-user]

@radarhere I do not catch. Or you don't catch the whole situation. The error message occured from the method FreeTypeFontFamily that something happended inside.

[mic-user]

The error above came from the line _font_family = ImageFont.FreeTypeFontFamily(_font, _backup_font) <<-- HERE

[nulano]

_font_family = ImageFont.FreeTypeFontFamily(_font, _backup_font) <<-- HERE

This function does not call getsize or any similar function, so I don't see how that could be possible. Please post the full stack trace.

[mic-user]

@nulano Erm, wrong line not that position but in the draw.text(). But here you go the stack trace.

Traceback (most recent call last):
  File "test.py", line 111, in xxxxxxxxx
    drawHandle.text(xy=(-_print_x, -_print_y),
  File "xxxxxxxxxxxxxxxxxx\Python\Python311-32\site-packages\PIL\ImageDraw.py", line 424, in text
    return self.multiline_text(
           ^^^^^^^^^^^^^^^^^^^^
  File "xxxxxxxxxxxxxxxxxx\Python\Python311-32\site-packages\PIL\ImageDraw.py", line 554, in multiline_text
    line_spacing = self._multiline_spacing(font, spacing, stroke_width)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "xxxxxxxxxxxxxxxxxx\Python\Python311-32\site-packages\PIL\ImageDraw.py", line 397, in _multiline_spacing
    self.textsize(
  File "xxxxxxxxxxxxxxxxxx\Python\Python311-32\site-packages\PIL\ImageDraw.py", line 633, in textsize
    return font.getsize(
           ^^^^^^^^^^^^
AttributeError: 'FreeTypeFontFamily' object has no attribute 'getsize'

[nulano]

Ah right, looks like multiline text is not working for this branch. If you want to use multiline text, you'll have to wait for someone to rebase this branch to main (the current _multiline_spacing function uses getbbox instead of getsize and I didn't implement getsize for FreeTypeFontFamily in anticipation of that change).

[mic-user]

@nulano Questions.

• Is there the 'Someone' already on it? or?
• 'Wait' mean.....how long?

[aclark4life]

@nulano Questions.

* Is there the 'Someone' already on it? or?

* 'Wait' mean.....how long?

@Mitchell-kw-Lee You can follow related issues and provide comments, questions and feedback just as you did here to find the answers to those questions. Pillow is released quarterly, and your fix may or may not be in the next release, based on the answers to those questions. I suspect we already know most of the involved parties from the discussion in this thread, and I don't think we can give any meaningful answer in this case to the the question of how long you'll have to wait for a new feature, developed in large part by volunteers, or underpaid developers. Sometimes we can answer! But I don't see anything definitive here yet … more than @nulano has already provided at least.

[nissansz]

fontfamily (font1 (include some chars)+font2(include most chars)) cannot show some characters

font1: see attachment.

font2: Arial Unicode MS.ttf or any other fonts str = 'をも資资儲储議议歷历權权個个TextInAaBbCcDdEeFfGgHhIi family PILLOW'

ballpen.zip

    # 写入字体效果的文字
    image = Image.new('RGB', (image_width, image_height), color='white')
    draw = ImageDraw.Draw(image)

     font0 = ImageFont.truetype(font_file, size=image_height-2)
     font1 = ImageFont.truetype(r'C:/F/fonts/tianshiyanti2.0.ttf', size=image_height-2)

        font_family = ImageFont.FreeTypeFontFamily(font0, font1)
        draw.text((0, 0), text+'   family PILLOW', fill='black', font=font_family)

[nulano]

Your code is incomplete. What is font_file? What is C:/F/fonts/tianshiyanti2.0.ttf? What is text? Please provide a complete example. See https://stackoverflow.com/help/minimal-reproducible-example

[nulano]

@mic-user I have rebased this PR to include Pillow 10.2.0 changes, multiline text seems to work now. You should be able to get a built wheel from https://github.com/nulano/Pillow/actions/runs/8583137967

[nissansz]

@mic-user I have rebased this PR to include Pillow 10.3.0 changes, multiline text seems to work now. You should be able to get a built wheel from https://github.com/nulano/Pillow/actions/runs/8583137967

Cannot see download link for windows. Does this wheel 10.3.0 support fontfamily?

[nissansz]

fontfamily (font1 (include some chars)+font2(include most chars)) cannot show some characters

font_file0: ballpen.zip.

font_file1: tianshiyanti2.0.ttf.zip

str = 'をも資资儲储議议歷历權权個个TextInAaBbCcDdEeFfGgHhIi family PILLOW'

image = Image.new('RGB', (image_width=1000, image_height=25), color='white')
draw = ImageDraw.Draw(image)

font0 = ImageFont.truetype(font_file0, size=20)
font1 = ImageFont.truetype(font_file1, size=20)

font_family = ImageFont.FreeTypeFontFamily(font0, font1)
draw.text((0, 0), textstr, fill='black', font=font_family)

[radarhere]

Cannot see download link for windows. Does this wheel 10.3.0 support fontfamily?

If you scroll down on that page, you will see dist-windows-x86, dist-windows-ARM64 and dist-windows-AMD64.

Yes, those wheels would support FreeTypeFontFamily.

[nissansz]

Cannot see download link for windows. Does this wheel 10.3.0 support fontfamily?

If you scroll down on that page, you will see dist-windows-x86, dist-windows-ARM64 and dist-windows-AMD64.

Yes, those wheels would support FreeTypeFontFamily.

I can download from your above link, but I don't see such link on list. How to find above links? Any new function for fontfamily in new version?

What is difference between [dist-windows-x86] and [dist-windows-AMD64]?

(https://github.com/nulano/Pillow/actions/runs/8583137967/artifacts/1391023653).

[nulano]

What is difference between [dist-windows-x86] and [dist-windows-AMD64]?

dist-windows-x86 is 32-bit x86 dist-windows-AMD64 is 64-bit x86. dist-windows-ARM64 is 64-bit ARM.

Any new function for fontfamily in new version?

No, except multiline text seems to work now.

I can download from your above link, but I don't see such link on list. How to find above links?

Go to https://github.com/nulano/Pillow/actions/runs/8583137967 and scroll down: