Download Sync Helper creates `inotify` watchers with `disableDownload` set to true

[plisy]

What happened? I started sync on large web client app (1000s of files). Webpack complained it reached the limit of file watchers.

I ran

find /proc/*/fd/* -type l -lname 'anon_inode:inotify' -exec sh -c 'cat $(dirname {})/../cmdline; echo ""' \; 2>/dev/null

to determine what else is taking file watchers and got devspacehelper sync downstream as the only process.

I tried a few things

• set downloadExcludePaths to ["/"] (which should match everything),
• set disableDownload to true on the file sync,

What did you expect to happen instead?

Since I set disableDownload to true on the file sync, I expected devspacehelper to take no file watchers.

How can we reproduce the bug? (as minimally and precisely as possible)

Set disableDownload on dev.sync to true.

My devspace.yaml:

version: v1beta11

# `vars` specifies variables which may be used as ${VAR_NAME} in devspace.yaml
vars:
- name: APP
  value: sample
- name: NAMESPACE
  value: sample

# `dev` only applies when you run `devspace dev`
dev:
  # `dev.sync` configures a file sync between our Pods in k8s and your local project files
  sync:
  - labelSelector:
      app: ${APP}
    namespace: ${NAMESPACE}
    waitInitialSync: true
    
    disableDownload: true
    downloadExcludePaths:
    - /
    uploadExcludePaths:
    - .git
    - node_modules/
    
  # `dev.terminal` tells DevSpace to open a terminal as a last step during `devspace dev`
  terminal:
    labelSelector:
      app: ${APP}
    namespace: ${NAMESPACE}
    # With this optional `command` we can tell DevSpace to run a script when opening the terminal
    # This is often useful to display help info for new users or perform initial tasks (e.g. installing dependencies)
    # DevSpace has generated an example ./devspace_start.sh file in your local project - Feel free to customize it!
    command:
    - ./bin/devspace_start.sh

  # Since our Helm charts and manifests deployments are often optimized for production,
  # DevSpace let's you swap out Pods dynamically to get a better dev environment
  replacePods:
  - labelSelector:
      app: ${APP}
    namespace: ${NAMESPACE}
    replaceImage: loftsh/typescript
    # Besides replacing the container image, let's also apply some patches to the `spec` of our Pod
    # We are overwriting `command` + `args` for the first container in our selected Pod, so it starts with `sleep 9999999`
    # Using `sleep 9999999` as PID 1 (instead of the regular ENTRYPOINT), allows you to start the application manually
    patches:
    - op: replace
      path: spec.containers[0].command
      value:
        - sleep
    - op: replace
      path: spec.containers[0].args
      value:
        - "9999999"
    - op: remove
      path: spec.containers[0].securityContext

Local Environment:

• DevSpace Version: 5.18.1
• Operating System: mac
• ARCH of the OS: ARM64 Kubernetes Cluster:
• Cloud Provider: google (loft vcluster)
• Kubernetes Version:

Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.0", GitCommit:"ab69524f795c42094a6630298ff53f3c3ebab7f4", GitTreeState:"clean", BuildDate:"2021-12-07T18:08:39Z", GoVersion:"go1.17.3", Compiler:"gc", Platform:"darwin/arm64"}
Server Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.8+k3s1", GitCommit:"5d0a12a6168bf2140216913ea9e5df6f81a26ec6", GitTreeState:"clean", BuildDate:"2021-06-21T20:44:53Z", GoVersion:"go1.15.12", Compiler:"gc", Platform:"linux/amd64"}

Anything else we need to know?

/kind bug

[plisy]

Workaround: Enable polling on dev.sync - it prevents the helper from getting file watches.

See docs for more info.

dev:
  sync:
  - # ...
    polling: true # <<-- Here, this does the trick.
    disableDownload: true # <<-- This doesn't. For now.
    # ...

[FabianKramm]

@plisy thanks for creating this issue! We might want to set polling: true automatically if disableDownload is set to true which should prevent this issue

[plisy]

From what I noticed, you might also not open the download stream and/or skip starting the server, but I didn't get to figure out whether that would have some unwanted side effects.

Another option might be passing the flag just like polling and handling it on the helper side.

Based on docs, polling might bring high CPU usage on container side, so I was hoping this would be a short term hack.

[FabianKramm]

@plisy if you disable downloading, DevSpace needs to remote side only initally for comparing files, but polling will be used only once and then will not poll again after initial sync ran through, which should not consume any CPU anymore, we could also think about closing the download stream fully after initially syncing.

[FabianKramm]

closing this for now, if this becomes a problem, please feel free to reopen