E-Mail-Templates: Field value for asset field contains references to all images in Volume and crashes SendMail jobs

[mnlmaier]

Bug description

Hi guys, here's another bug which took me literal days and an accidental variable dumping to figure out. On a production site, we are facing the problem of constant SendMail job crashes because of timeouts. This only happened if asset fields were part of the form.

The assets will not be attached directly to the email. Instead, they are placed in an Asset Container form_uploads and the links to the respective files will be rendered into the email template.

Now, back to the crash. Locally and in our Staging Environments, everything was working as expected. On live though, no emails were sent. Turns out, when Assets are passed into the mail template, references to all images which are stored in the asset volume are passed into the template as well. This very large dataset obviously crashes the job, since we have about 20k assets just happily hanging around in the file system.

I don't know if this is intended, but I would guess (and hope) it's not :) Sure, purging the Volume solves the problem, but it's only masking the problem and not solving the issue itself.

How to reproduce

• Create a public asset volume for form uploads
• Create a form with an asset field
• Setup an Email Template
• Render links to the uploaded files into the template
• Submit a form
• Check the Mail queue
• See it working fine
• Populate the Asset Container with about 20k images (depending on server power)
• Submit the form again
• See it fail

Logs

No response

Environment

Environment
Application Name: ..
Laravel Version: 10.48.11
PHP Version: 8.3.1
Composer Version: 2.7.7
Environment: production
Debug Mode: OFF
URL: ...
Maintenance Mode: OFF

Cache
Config: CACHED
Events: NOT CACHED
Routes: NOT CACHED
Views: CACHED

Drivers
Broadcasting: log
Cache: statamic
Database: mysql
Logs: stack / single, flare
Mail: smtp
Queue: redis
Session: file

Statamic
Addons: 7
Antlers: runtime
Sites: 4 (...)
Stache Watcher: Enabled
Static Caching: full
Version: 4.58.2 PRO

Statamic Addons
aryehraber/statamic-color-extractor: 1.4.0
aryehraber/statamic-uuid: 2.3.0
ddm-studio/cookie-byte: 1.5.0
handmadeweb/statamic-laravel-packages: 1.0.5
jonassiewertsen/statamic-documentation: 1.10.0
statamic-rad-pack/shopify: 3.3.1
swiftmade/statamic-clear-assets: 2.0.1

Installation

Starter Kit using via CLI

Additional details

No response

[duncanmcclean]

Hi guys, here's another bug which took me literal days and an accidental variable dumping to figure out. On a production site, we are facing the problem of constant SendMail job crashes because of timeouts. This only happened if asset fields were part of the form.

How big are the files being uploaded via the form?

[mnlmaier]

Hi @duncanmcclean, not really that large. Mostly just images, ranging around 0,5-10mb. Occasional videos, largest file is 45mb. Uploading works as espected, too, no problems in the CMS. Only the rendering of the mail template is what's causing issues.

I have now moved most of the files out of the volume, and Mails are delivered as expected. So I guess it's really the quantity of the files.

[duncanmcclean]

This very large dataset obviously crashes the job, since we have about 20k assets just happily hanging around in the file system.

Ah, just re-read your issue now and realised that's a lot of assets. 🤔

When you have a lot of assets, the Stache implementation for assets isn't the most performant. Usually, at that point we'd recommend looking at moving assets into the database using the Eloquent Driver.

The Eloquent Driver will keep an index of the files in the database, which makes it much quicker for querying.

Alternatively, you could "up" the timeout for your queue. If you're using Forge or Ploi, I believe there's a setting for the timeout where you configure the queue worker.

[mnlmaier]

@duncanmcclean sure, i understand that upping the timeout or moving the asset index somewhere else might prevent the obvious timeout issue. i don't think it fixes the underlying problem, though — why would i need a reference to every asset in the container while rendering the urls of a handful of uploaded assets (in my example, i needed to render links to 4 uploaded images) in the form notification?

that's about 20k references passed into the template to render 4 images, in other words, about 5000 times the asset references i actually need 🤕

i'm trying to figure out a way around this, but tbh, i don't really know where to start :/

[duncanmcclean]

Yeah, it shouldn't be referencing all assets, only the ones for the form submission. 🤔

Are you using a custom email template? If so, are you able to share it?

[mnlmaier]

@duncanmcclean Sure, I've been using Rob's Peak Form Template for ages. It's basically following this structure: https://github.com/studio1902/statamic-peak/blob/main/resources/views/email/form_owner.antlers.html — with a slight adjustment:

{{ fields }}
{{ if value && handle !== 'consent' }}
<tr>
    <td style='vertical-align:top; font-family:"Helvetica", sans-serif; padding:5px; border-bottom:#e3e3da 1px solid; font-size:14px' valign="top">
        <strong>{{ trans key="{display}" locale="{site:locale}" }}:</strong><br>
        <pre style="white-space:pre-line; font-family:inherit; margin:0">
            {{ if fieldtype == 'assets' }}
                <ul>{{ value }}
                    <li><a href="{{ url }}" target="_blank">{{ permalink }}</a></li>{{ /value }}
                </ul>
            {{ else }}
                {{ if (value | is_array) }}
                    <ul>
                        {{ value }}
                            <li>{{ value }}</li>
                        {{ /value }}
                    </ul>
                {{ else }}
                    {{ value }}
                {{ /if }}
            {{ /if }}
        </pre>
    </td>
</tr>
{{ /if }}
{{ /fields }}

Let me know if you need more info :)

[duncanmcclean]

Hmm, that looks fine to me...

It might not be possible, but are you able to provide a .zip of the assets container the asset field is using? You can send it to us privately via [email protected].

[github-actions[bot]]

This issue has not had recent activity and has been marked as stale — by me, a robot. Simply reply to keep it open and send me away. If you do nothing, I will close it in a week. I have no feelings, so whatever you do is fine by me.

[jbrhel]

Hello everyone. We have a similar issue related to handling emails that reference Globals. The task of sending an email randomly times out.

At first, we thought the problem was with loading an asset and then resizing it using Glide (a photo of a customer care team member in the footer). So, we switched to using a direct link to S3 instead of calling Glide. However, this didn’t solve the issue, and some emails still weren’t sent.

It was only when we removed any reference to our Globals from the view that the issue stopped.

The blueprint definition of the problematic Globals also includes an Asset field. We have thousands of images in the assets.

Let me know if I can assist with a more detailed report. Thank you

[duncanmcclean]

Are you able to provide access to the asset container? Either S3 credentials or a .zip. You can send it to us privately via [email protected].

[github-actions[bot]]

This issue has not had recent activity and has been marked as stale — by me, a robot. Simply reply to keep it open and send me away. If you do nothing, I will close it in a week. I have no feelings, so whatever you do is fine by me.