Import Recipes from Images

[NoDiskNoFun]

Is your feature request related to a problem? Please describe. I have recipes in the form of jpeg files, which i can currently only import by tipping them in manually.

Describe the solution you'd like It would be nice to integrate some form of OCR, Image to Text recognition. There are some open source projects out there, maybe one could "steal" some code of them to make this feature possible. Here is an example --> https://github.com/tesseract-ocr/tesseract

Describe alternatives you've considered There are websites out there which do provide OCR, maybe one could integrate these instead? Example --> https://www.onlineocr.net (first google hit, there are many more out there)

I am no programmer and i do not know how hard it is to implement such feature. I just wanted to share this idea here.

[mrleadhead]

This sounds like a fantastic idea! I can totally relate—my family has recipes they’d never sit down to type out themselves. OCR seems like a great starting point for digitizing the handwritten recipes, but transforming that output into a structured schema might be the real challenge. Recipes often come in many different formats, and OCR might not always capture sections in the right order.

To tackle this, we’d likely need to leverage an LLM to process and organize the data, but LLMs can sometimes produce inconsistent results. Running this within the app would likely demand a lot of resources, so integrating with an API—whether it’s cloud-based or locally hosted—might be the best option.

That said, I feel like this could be better tackled as its own dedicated project. I really love the concept, though, and it would be awesome to hear any other creative ideas for how we could make it work!

Another idea I had would be a drag and drop solution where OCR scans the document then the user organizes the output in a nice GUI.

[NoDiskNoFun]

Hey, thanks for your reply. After i posted my idea here i did manually what could be done with a nice GUI. I edited the pictures in GIMP and threw theses slices in some online OCR.

With a GUI it could work like this:

Open the imagefile with some kind of editor which is able to cut out slices of it Every field in the recipe (Picture, Ingrediens, and every step for the cooking process, ...) could be marked by the user accordingly These slices will be thrown in some OCR and the text gets filled in the space it needs to be (except the picture of cause :D)

With this i think most of the recipes availabe from books or handwrittings could be imported easily, but i know some that wont (especially from old cookbooks, as they sort there recipes in a wierd way sometimes)

[Eisbergsalat]

That is a great Idea.

Unfortunately the prerequisite "text to recipe" got marked as "not planned" #2100

I actually tried to create a skript (with the help of AI, so it isnt really robust)

The main funtion beeing:

• Watch for changes in the recipe folder.
• if there is a new *.md file that got the last word "fertig" (german for finished) that file will be converted to a recipe.json and gets moved to a new folder with the name of the recipe.
• And it creates a new Formatvorlage.md (german for style sheet) to show the user how the new recipe.md has to look like.

But the script is at the moment neither fully functionally (nutrients suddenly being steps to do... ) Nor is it fully compatible with most other setups -> You need your data internally fully exposed. (What I am doing only for the develepmoent of some scripts) And You NEED to execute it as root.........

But in hopes of rekindling this fire for new Import-Possibilities I will share what I have so far:

P.s. the picture is just a picture of "BEISPIEL" meaning example for testing purpose. I was not really bothering with it, because it doesnt work anyway and there were more pressing things in this script ... -draft

#!/bin/bash

WATCH_DIR="/mnt/ncdata/__groupfolders/1"
TEMPLATE_FILE="$WATCH_DIR/Formatvorlage.md"

# Erstellt die Formatvorlage, falls sie nicht existiert
if [ ! -f "$TEMPLATE_FILE" ]; then
    cat <<EOL > "$TEMPLATE_FILE"
# Rezeptname: Beispielrezept
Kategorie: Hauptgericht
Portionen: 2
Zubereitungszeit: PT0H30M0S
Gesamtzeit: PT1H0M0S
Bild: "https://as2.ftcdn.net/jpg/00/21/25/91/500_F_21259145_WhSLnmTzhJjMbVUZjZKt38kWwiraYMcv.jpg"
Schlagwörter: Beispiel, Test
Werkzeuge:
- Messer
- Schneidebrett

Zutaten:
- 200g Mehl
- 1 Ei
- 100ml Milch

Anleitung:
1. Mische das Mehl mit dem Ei.
2. Füge die Milch hinzu und verrühre alles gut.

Nährwerte (optional):
- Kalorien: 500 kcal
- Eiweiß: 20g
- Fett: 10g
- Kohlenhydrate: 70g

fertig
EOL
    echo "Formatvorlage wurde erstellt: $TEMPLATE_FILE"
fi

# Überwacht den Rezepte-Ordner auf Änderungen an .md-Dateien
inotifywait -m -e close_write --format "%w%f" "$WATCH_DIR" | while read FILE; do
    if [[ "$FILE" == *.md ]]; then
        # Prüft, ob das letzte Wort der Datei "fertig" ist
        LAST_WORD=$(tail -n 1 "$FILE" | awk '{print $NF}')
        if [ "$LAST_WORD" == "fertig" ]; then
            echo "Rezeptdatei erkannt: $FILE"

            # Rezept-Name aus der ersten Zeile extrahieren
            RECIPE_NAME=$(sed -n 's/^# Rezeptname: //p' "$FILE" | head -n 1)
            RECIPE_DIR="$WATCH_DIR/newRecipe"
            JSON_FILE="$RECIPE_DIR/recipe.json"

            # Überprüfen, ob der Ordner bereits existiert
            if [ -d "$RECIPE_DIR" ]; then
                echo "Fehler: Der Ordner '$RECIPE_DIR' existiert bereits!"
                continue
            fi

            mkdir "$RECIPE_DIR"
            # Daten extrahieren
            CATEGORY=$(sed -n 's/^Kategorie: //p' "$FILE")
            SERVINGS=$(sed -n 's/^Portionen: //p' "$FILE")
            PREP_TIME=$(sed -n 's/^Zubereitungszeit: //p' "$FILE")
            TOTAL_TIME=$(sed -n 's/^Gesamtzeit: //p' "$FILE")
            IMAGE_URL="https://as2.ftcdn.net/jpg/00/21/25/91/500_F_21259145_WhSLnmTzhJjMbVUZjZKt38kWwiraYMcv.jpg"
            KEYWORDS=$(sed -n 's/^Schlagwörter: //p' "$FILE")
            ID=$RANDOM
            DATE_CREATED=$(date -u +"%Y-%m-%dT%H:%M:%S+00:00")
            DATE_MODIFIED=$DATE_CREATED

            # Zutaten und Schritte extrahieren
            INGREDIENTS=$(awk '/^Zutaten:/ {flag=1; next} /^Anleitung:/ {flag=0} flag' "$FILE" | sed 's/^- //' | jq -R -s -c 'split("\n")[:-1]')
            INSTRUCTIONS=$(awk '/^Anleitung:/ {flag=1; next} /^Nährwerte:/ {flag=0} flag' "$FILE" | sed 's/^[0-9]\. //' | jq -R -s -c 'split("\n")[:-1]')
            TOOLS=$(awk '/^Werkzeuge:/ {flag=1; next} /^Zutaten:/ {flag=0} flag' "$FILE" | sed 's/^- //' | jq -R -s -c 'split("\n")[:-1]')

            # Nährwerte extrahieren
            CALORIES=$(sed -n 's/^- Kalorien: //p' "$FILE")
            PROTEIN=$(sed -n 's/^- Eiweiß: //p' "$FILE")
            FAT=$(sed -n 's/^- Fett: //p' "$FILE")
            CARBS=$(sed -n 's/^- Kohlenhydrate: //p' "$FILE")

            # JSON-Datei schreiben
            jq -n \
                --arg name "$RECIPE_NAME" \
                --arg category "$CATEGORY" \
                --arg servings "$SERVINGS" \
                --arg prepTime "$PREP_TIME" \
                --arg totalTime "$TOTAL_TIME" \
                --arg imageUrl "$IMAGE_URL" \
                --arg keywords "$KEYWORDS" \
                --arg id "$ID" \
                --arg dateCreated "$DATE_CREATED" \
                --arg dateModified "$DATE_MODIFIED" \
                --argjson ingredients "$INGREDIENTS" \
                --argjson instructions "$INSTRUCTIONS" \
                --argjson tools "$TOOLS" \
                --arg calories "$CALORIES" \
                --arg protein "$PROTEIN" \
                --arg fat "$FAT" \
                --arg carbs "$CARBS" \
                '{
                    "@context": "http://schema.org",
                    "@type": "Recipe",
                    "name": $name,
                    "recipeCategory": $category,
                    "recipeYield": $servings,
                    "prepTime": $prepTime,
                    "totalTime": $totalTime,
                    "imageUrl": $imageUrl,
                    "keywords": $keywords,
                    "id": $id,
                    "dateCreated": $dateCreated,
                    "dateModified": $dateModified,
                    "recipeIngredient": $ingredients,
                    "recipeInstructions": $instructions,
                    "tool": $tools,
                    "nutrition": {
                        "@type": "NutritionInformation",

                        "calories": $calories,
                        "protein": $protein,
                        "fat": $fat,
                        "carbs": $carbs
                    }
                }' > "$JSON_FILE"

            # Übergabe des Rezeptordners an www-data
            chown -R www-data:www-data "$RECIPE_DIR"
            mv ./1/newRecipe/ ./1/$RECIPE_NAME/

            echo "Rezept gespeichert in: $JSON_FILE"
            echo "Besitzrechte wurden an Nextcloud übergeben."
        fi
    fi
done

[lukas-holzner]

AI has improved a lot in the past Year. I have just tried out to converting a recipe using Gemini 2.5 Flash. With the Prompt "Convert the recipe in the image to JSON using the schema.org recipe schema.". This works very reliably. I created the Folder and recipe.json by hand and it detected the recipe perfectly.

I think it would be awesome to integrate it into the existing nextcloud assistant to have some cross plattform support. This might also make it possible, to import recipes from websites, that don't offer the propper formatting (A Prompt like "Couldn't parse the recipe directly. Do you want to use the Nextcloud Assistant to convert the recipe?" and than open the converted recipe in the editing mode to have the user check it).