dokploy icon indicating copy to clipboard operation
dokploy copied to clipboard

feat!: Network Management System v1

Open chichi13 opened this issue 3 months ago • 38 comments

What is this PR about?

Breaking Change

This PR introduces a complete redesign of network management in Dokploy, replacing the legacy isolatedDeployment feature with a flexible custom network system.


Overview

This PR implements comprehensive Docker network management as a core feature of Dokploy. Users can now create, manage, and assign custom Docker networks to any resource type (applications, compose services, databases) with full control over network configuration, IPAM settings, and driver selection.


Major Changes

Network Management System

  • Global Network Management UI: New /dashboard/networks page for organization-wide network administration
  • Resource-Level Network Assignment: Integrated network management into all resource types (applications, compose, databases)
  • Project Organization: Added dedicated networks tab to project views for better organization

Database Schema

New network Table

  • Multiple network drivers: bridge and overlay
  • Advanced IPAM configuration: subnet, gateway, IP range
  • Network encryption support for overlay networks
  • Organization scope
  • Docker network ID tracking for synchronization

Resource Schema Updates

  • Added customNetworkIds field to all resource types:
    • application
    • compose
    • postgres, mysql, mariadb, mongo, redis
  • Supports multiple network assignments per resource

Network Features

Automatic Traefik Integration

  • Traefik automatically connects to custom networks when domains are added to resources
  • Automatic disconnection when networks are no longer in use by resources with domains
  • Ensures proper routing without manual network configuration

Driver Validation

  • Enforces overlay networks for Docker Swarm services
  • Enforces overlay networks for compose stacks
  • Bridge networks allowed for standard docker-compose mode

Network Synchronization (based on labels)

  • Sync functionality to detect discrepancies between Docker and database state
  • Identifies missing networks (in DB but not in Docker)
  • Identifies orphaned networks (in Docker but not in DB)
  • Import orphaned networks with proper label validation

Migration and Backward Compatibility

Migration 0117: Add Network table and add customNetworkIds field

Migration 0118: Convert isolatedDeployment to customNetworkIds

This migration automatically converts all existing compose services using isolatedDeployment=true to the new network system:

  1. Scans all compose services with isolatedDeployment=true
  2. Creates network records using the compose appName as the network name
  3. Assigns appropriate driver (overlay for remote servers, bridge for local)
  4. Links networks to compose services via customNetworkIds
  5. Sets isolatedDeployment=false on migrated services

Context: Compose services with isolated deployment already have Docker networks created with name = compose.appName. The migration imports these existing networks into the database and establishes proper tracking.

Migration 0119: Remove isolatedDeployment Field

Removes the obsolete isolatedDeployment boolean field from the database schema after successful migration.


Testing

New Test Coverage

  • Network Service Tests (network-service.test.ts):

    • CRUD operations
    • Resource assignment/removal
    • Traefik integration
    • Sync and import functionality
  • Compose Network Integration (compose-networks.test.ts):

    • Compose-specific network behavior
    • Stack vs docker-compose mode validation
  • Schema Validation (schema.test.ts):

    • Network configuration validation
    • IPAM parameter validation
    • Driver validation

Breaking Changes

Removed: isolatedDeployment Field

The isolatedDeployment boolean field has been removed from the compose table schema.

Migration: All existing data is automatically migrated via migration 0118. No manual intervention required for Dokploy users.

Checklist

Before submitting this PR, please make sure that:

  • [x] You created a dedicated branch based on the canary branch.
  • [x] You have read the suggestions in the CONTRIBUTING.md file https://github.com/Dokploy/dokploy/blob/canary/CONTRIBUTING.md#pull-request
  • [x] You have tested this PR in your local instance.

Issues related

closes #2798 closes #1495

Screenshots

image image image

chichi13 avatar Oct 12 '25 10:10 chichi13

If you have any seed.sql or backup.sql file to test migrations other than mine, I'd be happy to use it. If you want, you can also run this branch on a duplicate instance of your dokploy production instance.

  • This is v1 implementation, I would love to add a topology schema in an other PR, so we can see exactly what are the networks connected to Traefik for example.
  • Now that I'm familiar with the codebase, I can see lots of interesting PRs that I could work on.
  • As for the documentation, I'm waiting for your go-ahead to start working on it.

chichi13 avatar Oct 12 '25 10:10 chichi13

After several hours of testing, I can't find any more bugs. I look forward to hearing your feedback!

chichi13 avatar Oct 12 '25 13:10 chichi13

Looks good ! Thank you for the PR :)

s1nyx avatar Oct 12 '25 18:10 s1nyx

You can try this PR by pulling this image docker pull achichi13/dokploy:network-pr (which is this PR). Don't try this on production server.

For my part, my production server is running on this version and I haven't had any problems so far.

chichi13 avatar Oct 16 '25 07:10 chichi13

Hi @chichi13. First of all thanks you for taking initiative on implementing this feature. I started testing your PR from docker image you provided. Could you please clarify if Traefik instance should automatically connect to newly created network, or that is what should be manually configured?

KarpachMarko avatar Oct 16 '25 15:10 KarpachMarko

Thanks for testing @KarpachMarko

  1. You should create a network manually first.
  2. Then, you should add it to the application, compose, database (either with or without a domain).
  3. When attaching a domain to an application with a custom network you should redeploy the app. It's not automatically redeployed.

Is it answering your question?

chichi13 avatar Oct 16 '25 15:10 chichi13

Thanks for quick response. I tested again following your steps but same issue. Here is what I did:

  • Created internal bridge network
  • Create new project with docker compose service
  • Assigned this internal network to created service
  • Set up domain name for the service
  • Redeployed the service

As a result no new file created in "Traefik File System"

Am I missing something? Note: I have traefik running as standalone container. Not as docker service

KarpachMarko avatar Oct 16 '25 15:10 KarpachMarko

Okay, maybe I should clarify what "internal" is. It's used essentially for container (like databases) which don't need internet access. By enabling "internal", adding a domain won't work:

image

You should then, attach it to a non internal network. I'll try to figure out how I can manage those internal networks.

chichi13 avatar Oct 16 '25 15:10 chichi13

@chichi13 I found minor UI issue on network management page. There is duplicated header showing app navigation Screenshot 2025-10-16 at 19 53 23

KarpachMarko avatar Oct 16 '25 16:10 KarpachMarko

Indeed, will fix that, thanks!

chichi13 avatar Oct 16 '25 18:10 chichi13

Hi,

I cannot choose where the network is created, i have tested with a remote servers, and the network is always created of the dokploy host, but my application are on a remote servers

Elvinra avatar Oct 17 '25 09:10 Elvinra

Hi,

I'll check that this afternoon, thanks!

chichi13 avatar Oct 17 '25 11:10 chichi13

And you're totally right, I didn't take that case into account. I'm about to commit a fix!

Thanks for this test

chichi13 avatar Oct 17 '25 12:10 chichi13

Okay so, now:

  • When adding a domain to an application / compose which is attached to an internal network only:
image image
  • Remote server are now working well. @Elvinra could you try again please? Unfortunately, I was unable to test deploying a domain on a remote server. If you can test it, that would be great.

  • UI Fix on network page

When attaching a network or deploying a domain, you should redeploy the application. I haven't changed the original functionality with dokploy-network.

You can force pull the same image. I've updated it: docker pull achichi13/dokploy:network-pr

chichi13 avatar Oct 17 '25 13:10 chichi13

Great it's working with remote servers.

I made some test, i think something don't feel right. with/without remote server

So i'm using a compose for starting everything except the database.

I've created a network "db-prod" link to the database. I setup my compose stack webserver, processing queue, and database migration with the network db-prod. When i add a domain for the service "webserver", traefik have a new network "db-prod", but it should the service "webserver" to have the network "dokploy-network"

Elvinra avatar Oct 17 '25 14:10 Elvinra

@Elvinra is the network "db-prod" internal? If not Traefik will connect to both which can be improved.

But I don't really know how to do it:

  • 3 networks attached
  • 1 domain
  • Which network Traefik should choose? Should I put a dropdown in the domain modal to select only one network?

chichi13 avatar Oct 17 '25 14:10 chichi13

"db-prod" is not internal, dont know what internal should do ^^

Traefik have already a network attached "dokploy-network", why not use this one. And attach "dokploy-network" to the service specified in the domain ?

Elvinra avatar Oct 17 '25 14:10 Elvinra

internal(internal) network are used essentially for container (like databases) which don't need internet access. Internal network won't have internet access + can't be published.

Attaching every application to dokploy-network is a security issue. You can check the issue #2798 if you want more details.

chichi13 avatar Oct 17 '25 15:10 chichi13

Ok i see now.

The select box inside the "Domain" modal should be the better thing to do.

The select box should list every network who are not internal By default if none or empty are selected, it's "dokploy-network" for not breaking anything

and on deploy both traefik + the service selected must have the network register.

edit: I made some other test with an internal network "db-internal"

the db network is internal now, and my stack are attached with 2 networks,

  • db-internal
  • mystack-network (not internal)

Only if i recreate the domain, "mystack-network" is attached to Traefik

Elvinra avatar Oct 17 '25 15:10 Elvinra

I'll check another day for the improvements.

the db network is internal now, and my stack are attached with 2 networks,

Did you redeploy the app?

chichi13 avatar Oct 17 '25 15:10 chichi13

I'll check another day for the improvements.

the db network is internal now, and my stack are attached with 2 networks,

Did you redeploy the app?

yes at each step

Elvinra avatar Oct 17 '25 15:10 Elvinra

@Elvinra so, I've refactored how domain/traefik works, now you don't need to redeploy when you attach a domain. You can specify to which domain you want to attach. If you have 2 domains, both domains should be attached to the same network.

I would be happy to have a feedback from @Siumauricio to find out if there's a chance this PR will be merged (given the time I've spent on it, I'd really like it to be).

chichi13 avatar Oct 24 '25 08:10 chichi13

You can force pull the same image. I've updated it: docker pull achichi13/dokploy:network-pr

New migration file added:

ALTER TABLE "domain" ADD COLUMN "networkId" text;--> statement-breakpoint
ALTER TABLE "domain" ADD CONSTRAINT "domain_networkId_network_networkId_fk" FOREIGN KEY ("networkId") REFERENCES "public"."network"("networkId") ON DELETE set null ON UPDATE no action;

After analysis, one feature is still missing, but I don't currently have time to prioritize dokploy:

  • Adding networks for “preview deployments”

I've made a good start on preview deployments, but it's taking longer than expected. There are some small subtleties that need to be dealt with.

chichi13 avatar Oct 24 '25 10:10 chichi13

I believe you're doing a great job with this repository, Antoine. Do you think you could help me implement it in my swarm? If there's any cost, we can discuss it.

luisrosasx avatar Oct 24 '25 12:10 luisrosasx

I finally added the feature for “preview deployments.” One last mini SQL migration:

ALTER TABLE "application" ADD COLUMN "previewNetworkIds" text[];

It is now possible to define a custom network for preview deployments. This allows you to avoid running the preview deployment on the same network as the production app (which I recommend). I tested it on my production environment and everything seems to be working very well. I suggest creating one or more networks for previews.

As usual, the image has been updated. You can pull it with docker pull achichi13/dokploy:network-pr. Please let me know if you find any bugs or have any suggestions for improvements.

chichi13 avatar Oct 24 '25 13:10 chichi13

Hello,

I just pull the last version with a fresh docker, and i have an error

[
    {
        "error": {
            "json": {
                "message": "column projects_environments_applications_domains.networkId does not exist",
                "code": -32603,
                "data": {
                    "code": "INTERNAL_SERVER_ERROR",
                    "httpStatus": 500,
                    "path": "project.all",
                    "zodError": null
                }
            }
        }
    }
]

Elvinra avatar Oct 24 '25 13:10 Elvinra

@Elvinra apply the SQL Migration:

docker exec -it $(docker ps -q --filter name=dokploy-postgres) psql -U dokploy

Then, execute migration 120/121:

ALTER TABLE "domain" ADD COLUMN "networkId" text;--> statement-breakpoint
ALTER TABLE "domain" ADD CONSTRAINT "domain_networkId_network_networkId_fk" FOREIGN KEY ("networkId") REFERENCES "public"."network"("networkId") ON DELETE set null ON UPDATE no action;
ALTER TABLE "application" ADD COLUMN "previewNetworkIds" text[];

It should now work :)

chichi13 avatar Oct 24 '25 13:10 chichi13

But you said "fresh docker".

Did you delete the old database volume?

chichi13 avatar Oct 24 '25 13:10 chichi13

But you said "fresh docker".

Did you delete the old database volume?

I boot a new VM from zero

  • install docker
  • curl -sSL https://dokploy.com/install.sh | sh
  • setup login
  • docker pull achichi13/dokploy:network-pr
  • docker service update --image achichi13/dokploy:network-pr dokploy

Running the migration in psql directly works, but why the migration wasn't launch at the update ?

Elvinra avatar Oct 24 '25 13:10 Elvinra

I've updated the migration files + the docker image. I feel like I did the migration with drizzle incorrectly (I'm not very familiar with this tool).

chichi13 avatar Oct 24 '25 14:10 chichi13