Resources are exhausted due to the threads and processes not released when a connection is not alive

[yuyang733]

I use pyftpdlib to finish my ftp server program. But I find a serious issue on pyftpdlib that the machine resource are exhausted after multi clients connect it repeatedly. I read the source code and found that the threads and processes which is created to handle every connection can be wait only when the server close.

It result that the server running for a long time, the resource would be exhausted and service is not available finally.

[giampaolo]

Please paste the code you're using.

[yuyang733]

Here is my code:

# ...
    if masquerade_address is not None:
        handler.masquerade_address = masquerade_address

    handler.passive_ports = passive_ports

    server = FTPServer(("0.0.0.0", port), handler)

    try:
        server.serve_forever()
    finally:
        server.close_all()
# ...

And the following is the source code for pyftpd，which can cause zombie processes or threads:

    def serve_forever(self, timeout=1.0, blocking=True, handle_exit=True):
        self._exit.clear()
        if handle_exit:
            log = handle_exit and blocking
            if log:
                self._log_start()
            try:
                self.ioloop.loop(timeout, blocking)
            except (KeyboardInterrupt, SystemExit):
                pass
            if blocking:
                if log:
                    logger.info(
                        ">>> shutting down FTP server (%s active workers) <<<",
                        self._map_len())
                self.close_all()
        else:
            self.ioloop.loop(timeout, blocking)
    def close_all(self):
        tasks = self._active_tasks[:]
        # this must be set after getting active tasks as it causes
        # thread objects to get out of the list too soon
        self._exit.set()
        if tasks and hasattr(tasks[0], 'terminate'):
            # we're dealing with subprocesses
            for t in tasks:
                try:
                    if not _BSD:
                        t.terminate()
                    else:
                        # XXX - On FreeBSD using SIGTERM doesn't work
                        # as the process hangs on kqueue.control() or
                        # select.select(). Use SIGKILL instead.
                        os.kill(t.pid, signal.SIGKILL)
                except OSError as err:
                    if err.errno != errno.ESRCH:
                        raise

        self._wait_for_tasks(tasks)
        del self._active_tasks[:]
        FTPServer.close_all(self)

    def _wait_for_tasks(self, tasks):
        """Wait for threads or subprocesses to terminate."""
        warn = logger.warning
        for t in tasks:
            t.join(self.join_timeout)
            if t.is_alive():
                # Thread or process is still alive. If it's a process
                # attempt to send SIGKILL as last resort.
                # Set timeout to None so that we will exit immediately
                # in case also other threads/processes are hanging.
                self.join_timeout = None
                if hasattr(t, 'terminate'):
                    msg = "could not terminate process %r" % t
                    if not _BSD:
                        warn(msg + "; sending SIGKILL as last resort")
                        try:
                            logger.info("")
                            os.kill(t.pid, signal.SIGKILL)
                        except OSError as err:
                            if err.errno != errno.ESRCH:
                                raise
                    else:
                        warn(msg)
                else:
                    warn("thread %r didn't terminate; ignoring it", t)

[yuyang733]

I am using version 1.5.3

[giampaolo]

I suppose this should fix it https://github.com/giampaolo/pyftpdlib/commit/37236bfdaf51cd9d0c5759560a391771765a2ee8. There is currently no unit test for this though.