Add Deepseek Janus Pro 1B/7B for Vision AND Image Generation

[SuperPat45]

Janus-Pro is a novel autoregressive framework that unifies multimodal understanding and generation.

https://huggingface.co/deepseek-ai/Janus-Pro-7B

[Hello-World-Traveler]

That is a very huge improvement for 7B.

[Hello-World-Traveler]

@mudler can this be added to the list for image generation?

[Hello-World-Traveler]

Added this myself but hit errors.

:19AM DBG Stopping all backends except 'Janus-Pro-7B'

7:19AM INF BackendLoader starting backend=diffusers modelID=Janus-Pro-7B o.model=Janus-Pro-7B-LM.Q4_K_M.gguf

7:19AM DBG Loading model in memory from file: /models/Janus-Pro-7B-LM.Q4_K_M.gguf

7:19AM DBG Loading Model Janus-Pro-7B with gRPC (file: /models/Janus-Pro-7B-LM.Q4_K_M.gguf) (backend: diffusers): {backendString:diffusers model:Janus-Pro-7B-LM.Q4_K_M.gguf modelID:Janus-Pro-7B assetDir:/tmp/localai/backend_data context:{emptyCtx:{}} gRPCOptions:0xc0a0cb1348 externalBackends:map[bark:/build/backend/python/bark/run.sh coqui:/build/backend/python/coqui/run.sh diffusers:/build/backend/python/diffusers/run.sh exllama2:/build/backend/python/exllama2/run.sh faster-whisper:/build/backend/python/faster-whisper/run.sh kokoro:/build/backend/python/kokoro/run.sh rerankers:/build/backend/python/rerankers/run.sh transformers:/build/backend/python/transformers/run.sh vllm:/build/backend/python/vllm/run.sh] grpcAttempts:20 grpcAttemptsDelay:2 parallelRequests:false}

7:19AM DBG Loading external backend: /build/backend/python/diffusers/run.sh

7:19AM DBG external backend is a uri

7:19AM DBG Wait for the service to start up

7:19AM DBG Options: ContextSize:16384 Seed:179870228 NBatch:512 F16Memory:true MMap:true NGPULayers:99999999 Threads:10 PipelineType:"StableDiffusionPipeline" SchedulerType:"k_dpmpp_2m" CUDA:true Options:"gpu"

7:20AM INF Success ip=127.0.0.1 latency="28.183µs" method=GET status=200 url=/readyz

7:20AM ERR failed starting/connecting to the gRPC service error="rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp: address /build/backend/python/diffusers/run.sh: missing port in address\""

7:20AM DBG GRPC Service NOT ready

7:20AM ERR Server error error="failed to load model with internal loader: grpc service not ready" ip=192.168.68.55 latency=40.248424031s method=POST status=500 url=/v1/images/generations

      

[github-actions[bot]]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.