Gemma 3n Model Stuck on Loading with GPU Accelerator—No Token Output

[seanmrnda]

Description:
Hi team,

I’m experiencing an issue with the Edge Gallery when running the Gemma 3n model. After downloading the model locally and setting the accelerator to GPU, the interface remains stuck on “loading” and does not produce any tokens in response to input. I have waited several minutes with no change.

Steps to Reproduce:

1. Download Gemma 3n model locally via the Edge Gallery.
2. Set the accelerator to GPU.
3. Attempt to generate tokens with any input prompt.

Expected Behavior:
The model should load successfully and generate tokens in response to input.

Actual Behavior:
The loading indicator remains on screen indefinitely. No output or tokens are generated.

Environment:

• Model: Gemma 3n
• Accelerator: GPU
• Platform/OS: HyperOs 2.0
• Edge Gallery version: 1.0.3

Additional Information:

• No error message is displayed.
• The problem persists after restarting the application.

[Bzucko]

Same issue here on Asus Zenfone 8, android 13, gpu adreno 660, snapdragon 888 (SM8350), Gemma 3n 4B, edge 1.0.3 UPDATE: the model loads after 2-3min.(top right icons are clickable again), so it is working, but the response after several hundreds of tokens start to generate gibberish.

[Jake36921]

Same issue on my S22 (Snapdragon 8 Gen 1, Adreno 730, android 15, 8Gb ram), using Gemma 3n4B, edge 1.0.3 It crashes after a while, most likely a memory leak on my end.

[ic4-y]

Yeah, I'd say I am experiencing the same, however with the Gemma 3n 1B model. Moto G32 (Snapdragon 680, Adreno 610, 6 GB RAM) with CalyxOS (Android 15). Also had the crash experience just a moment ago.

[gururise]

Same issue - Google Pixel 8 ( Android 15 )

[masc-it]

for me (samsung s25+) it either crashes or generates gibberish. on CPU works fine.

[sleyv]

At first I thought it was busted, but after like 3 minutes it finally loaded up and started running fine on the GPU. But when I switch back to the CPU, it suddenly gets super slow for some reason, even though it wasn’t like that before.

Poco f6

[masc-it]

update: I've tried it on a Moto g84, Android 13, 16GB ram, it works on gpu as well.

On the Samsung, Android 15, Snapdragon 8 Elite, either crashes or outputs garbage.

I'd like to provide crash logs or something like that, but I see no way of doing that.

Also, it would be so nice that before loading the model, which takes some time, the app would first ask the device to use. everytime I need to wait for the defaul loading (CPU) and then switch to GPU. double wait time.

[nonetrix]

Same hers Samsung S25+

Edit: turns out I am just impatient, works "fine". However, it just repeats itself which is another issue. On the bright side however: quite fast 40 tokens a second with 4B model, compared to like 13 on CPU. Model loading speed is definitely main issue, shouldn't be that much slower since they share same memory pretty sure

[alex-x-x]

Works like this on GPU on OnePlus 13:

On CPU works fine.

[dpknag]

Hello, Thank you, @seanmrnda, for reporting this issue, and thank you everyone for contributing to this thread with your important feedback. We have logged this as a bug and have passed it to the team.

Thanks again for your help in improving the app!

[dpknag]

Hi all,

As we continue our investigation into this issue, we are requesting a full Bug Report. This will be a great help to our engineers in diagnosing the root cause.

We've put together a step-by-step guide on how to capture and share one here: https://github.com/google-ai-edge/gallery/blob/main/Bug_Reporting_Guide.md

Thank you for your ongoing help and patience.

[masc-it]

@dpknag quite convoluted process, will try. What about an in app reporting logic? you should log all model loading failures by default. given the well known android hardware fragmentation, that s the most likely problem to happen, along with unsupported ops.

[Bzucko]

@dpknag here is at least log grabbed using: adb logcat >logcat.txt I started logging before I run gallery app and I stopped few seconds after gemma started to generates gibberish. Asus Zenfone 8, android 13 rooted, gpu adreno 660, snapdragon 888 (SM8350), Gemma 3n 4B, edge 1.0.3

logcat.txt

[dpknag]

Hello everyone,

Thank you all for your detailed reports and valuable feedback on the GPU performance issues. Your contributions have been crucial in helping us address these problems.

@masc-it We appreciate your feedback on the bug reporting process. Your suggestion for a more streamlined, in-app logic is a great point, and we have logged it and will keep tracking the request. @Bzucko Thank you for taking the time to provide the logcat file.

We have released a new version of the app (v1.0.7) and please update to the latest version and share your feedback. Release Page: https://github.com/google-ai-edge/gallery/releases/tag/1.0.7 Play Store: https://play.google.com/store/apps/details?id=com.google.ai.edge.gallery

We are closing this issue now. If continue to experience these problems after updating to v1.0.7, please leave a new comment, or reopen the issue.

Thank you again for helping us improve the app.