Whole project with Playground and local model use

[gitknu]

Hello, sorry for stupid question, but can you upload the project with Playground it seelf and the manual how to run model locally, just by picking downloaded GGUF file? I would like to run offline webserver and try running models on isolated machine.

Thank you for understanding and such a great job you`ve done!

[rahuldshetty]

Hello @gitknu,

You can find the source code for the playground and other examples over here: https://github.com/rahuldshetty/ggml.js-examples

You just need to provide the relative path to the model file in your local machine.

E.g code:

<script type="module">
        import {LLM} from "../dist/llm.js";

        let submitButton = document.getElementById('submitBtn');
        let outputElement = document.getElementById('output');

        const on_loaded = () => {
            submitButton.disabled = false;
            submitButton.innerText = "Generate";
        }

        const write_result = (line) => {
            outputElement.textContent  += line + "\n";
        }

        const run_model = () => {
            const text = document.getElementById("textInput").value;
            submitButton.innerText = "Running...";
            submitButton.disabled = true;
            outputElement.textContent = ""; // clead old content
            app.run({
                prompt: text,
                max_token_len: 64,
                top_k: 1,
            });
        }

        const run_complete = () => {
            submitButton.innerText = "Generate";
            submitButton.disabled = false;
        }
        
        const app = new LLM(
            'LLAMA',
            '../models/llama2_xs_460m_experimental_evol_instruct.q4_k_m.gguf',  
            on_loaded,
            write_result,
            run_complete
        );
        
        app.load_worker();
        submitButton.addEventListener("click", run_model);

    </script>

This works with local live-server: https://marketplace.visualstudio.com/items?itemName=ritwickdey.LiveServer