Error: Unsupported model type: gemma3n

[seungjun-green]

If I try to use gemma-3n model using swift-mlx-examples package I get following error:

Error: Unsupported model type: gemma3n

I installed the package and run the code with below simple code. It all worked well with different version of model such as SmolLM-135M-Instruct-4bit but keep failing when I try to use mlx-community/gemma-3n-E2B-it-lm-bf16 or mlx-community/gemma-3n-E4B-it-lm-bf16

import SwiftUI
import MLXVLM
import MLXLMCommon

struct ContentView: View {
    @State private var isLoading = false

    var body: some View {
        VStack(spacing: 20) {
            if isLoading {
                ProgressView("Processing...")
            }

            Button("Process") {
                Task {
                    await process()
                }
            }
            .padding()
        }
        .padding()
    }

    func process() async {
        isLoading = true
        defer { isLoading = false }

        do {
            let model = try await loadModel(id: "mlx-community/SmolLM-135M-Instruct-4bit")
            let session = ChatSession(model)
            
            print(try await session.respond(to: "What are two things to see in San Francisco?"))

        } catch {
            print("Error: \(error.localizedDescription)")
        }
    }
}

[davidkoski]

I wonder if you are using the tag of mlx-swift-examples (mlx-libraries) or main? This code isn't in a tag yet.

[seungjun-green]

I was using the tag version not the main. Using main the gemma3n works very well. Sorry for asking stupid question. Thanks for the great work!

[davidkoski]

No worries -- it isn't always clear what to do!

[seungjun-green]

Sorry for the confusion. Last time I tested gemma3n in a wrong way(passed other model name to the id argument). When tested correctly, it does not produce Error: Unsupported model type: gemma3n error but generating the response never stops. Could you check this issue please?

[shashankb-cc]

Sorry for the confusion. Last time I tested gemma3n in a wrong way(passed other model name to the id argument). When tested correctly, it does not produce Error: Unsupported model type: gemma3n error but generating the response never stops. Could you check this issue please?

Hi mate, could you please provide me with a flow for running an LLM locally within our own app? I’ve been trying but haven’t been able to get it working. Could you share part of your code or explain how you did it? It would be really helpful.

[seungjun-green]

Sorry for my late reply, Here is an example iOS app I built which runs gemma-3n models on iPhone. https://github.com/seungjun-green/FoodLen