docker-firefox-syncserver
docker-firefox-syncserver copied to clipboard
crazy-max
forwarded_allow_ips variable not applied to gunicorn launch
Behaviour
As my Nginx server with the reverse proxy is on another machine than the docker, I'm in need of the "forwarded_allow_ips" setting set to * (or the IP of my Nginx server), but I can't get the setting to be applied to Gunicorn. It always keeps the default of 127.0.0.1 - see line from the debug log:
firefox_syncserver | forwarded_allow_ips: ['127.0.0.1']
Steps to reproduce this issue
- docker pull
- set
FF_SYNCSERVER_FORWARDED_ALLOW_IPS = * - run docker
Expected behaviour
gunicorn should display set forward_allow_ips variable instead of the default. This would prevent gunicorn from mistrusting the https protocol from the proxy and allowing the syncserver to properly work.
Actual behaviour
gunicorn always uses default forwarded_allow_ips = 127.0.0.1 and per the note from the official syncserver docs:
Note
If you see errors about a mismatch between public_url and application_url, you may need to tell gunicorn that it should trust the X-Forwarded-Proto header being sent by nginx. Add the following to the gunicorn configuration in syncserver.ini:
forwarded_allow_ips = *
this prevents the sync server from actually syncing as the sync process then attempts to use http:// instead of the secured one, which fails as my server isn't set to accept insecure requests.
Configuration
- Docker version (type
docker --version) : 20.10.17, build 100c701 - Docker compose version if applicable (type
docker-compose --version) : 1.29.2 - Platform (Debian 9, Ubuntu 18.04, ...) : Debian Bullseye
- System info (type
uname -a) : 5.15.43-sunxi64 #22.05.1 SMP Sat May 28 08:25:20 UTC 2022 aarch64 GNU/Linux - docker-compose.yml:
version: "3.2"
services:
firefox-syncserver:
image: crazymax/firefox-syncserver
container_name: firefox_syncserver
ports:
- target: 5000
published: 5000
protocol: tcp
volumes:
- "./data:/data"
env_file:
- "./firefox-syncserver.env"
restart: always
firefox-syncserver.env:
TZ=Europe/Paris
PUID=1000
PGID=1000
FF_SYNCSERVER_PUBLIC_URL=https://[myserver]/moz-sync
FF_SYNCSERVER_SECRET=[mysecret]
FF_SYNCSERVER_ALLOW_NEW_USERS=true
FF_SYNCSERVER_FORCE_WSGI_ENVIRON=false
FF_SYNCSERVER_FORWARDED_ALLOW_IPS=*
FF_SYNCSERVER_LOGLEVEL=debug
contents of /opt/syncserver/syncserver.ini from within running docker container:
bash-5.0# cat /opt/syncserver/syncserver.ini
[server:main]
use = egg:gunicorn
host = 0.0.0.0
port = 5000
workers = 1
timeout = 30
loglevel = debug
[app:main]
use = egg:syncserver
[syncserver]
# This must be edited to point to the public URL of your server,
# i.e. the URL as seen by Firefox.
public_url = https://[myserver]/moz-sync
# This defines the database in which to store all server data.
sqluri = sqlite:///data/syncserver.db
# This is a secret key used for signing authentication tokens.
# It should be long and randomly-generated.
# The following command will give a suitable value on *nix systems:
#
# head -c 20 /dev/urandom | sha1sum
#
# If not specified then the server will generate a temporary one at startup.
secret = d0a07aea2425a116f688aed63bd60e1278fcc512
# Set this to "false" to disable new-user signups on the server.
# Only request by existing accounts will be honoured.
allow_new_users = true
# Set this to "true" to work around a mismatch between public_url and
# the application URL as seen by python, which can happen in certain reverse-
# proxy hosting setups. It will overwrite the WSGI environ dict with the
# details from public_url. This could have security implications if e.g.
# you tell the app that it's on HTTPS but it's really on HTTP, so it should
# only be used as a last resort and after careful checking of server config.
force_wsgi_environ = false
# Uncomment and edit the following to use a local BrowserID verifier
# rather than posting assertions to the mozilla-hosted verifier.
# Audiences should be set to your public_url without a trailing slash.
#[browserid]
#backend = tokenserver.verifiers.LocalVerifier
#audiences = https://localhost:5000
# If you are running Nginx on a different host than the ff sync server the ff snyc server have to trust the X-Forwarded-* headers sent by Nginx.
forwarded_allow_ips = *
Include all necessary configuration files : docker-compose.yml, .env, ...
Docker info
Client:
Context: default
Debug Mode: false
Plugins:
app: Docker App (Docker Inc., v0.9.1-beta3)
buildx: Docker Buildx (Docker Inc., v0.8.2-docker)
compose: Docker Compose (Docker Inc., v2.6.0)
Server:
Containers: 9
Running: 8
Paused: 0
Stopped: 1
Images: 7
Server Version: 20.10.17
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 0197261a30bf81f1ee8e6a4dd2dea0ef95d67ccb
runc version: v1.1.3-0-g6724737
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: default
cgroupns
Kernel Version: 5.15.43-sunxi64
Operating System: Debian GNU/Linux 11 (bullseye)
OSType: linux
Architecture: aarch64
CPUs: 4
Total Memory: 1.943GiB
Name: pine64
ID: KWYO:YKZ5:RMQF:347Q:QUYU:XDYJ:36VS:ZUVK:SI57:2EIF:F4FX:W5XW
Docker Root Dir: /var/lib/docker
Debug Mode: false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
Logs
Attaching to firefox_syncserver
firefox_syncserver | Setting timezone to Europe/Paris...
firefox_syncserver | Checking prerequisites...
firefox_syncserver | Generating configuration...
firefox_syncserver | Fixing perms...
firefox_syncserver | [2022-09-08 11:00:32 +0000] [1] [DEBUG] Current configuration:
firefox_syncserver | secure_scheme_headers: {'X-FORWARDED-PROTOCOL': 'ssl', 'X-FORWARDED-PROTO': 'https', 'X-FORWARDED-SSL': 'on'}
firefox_syncserver | proxy_protocol: False
firefox_syncserver | worker_connections: 1000
firefox_syncserver | statsd_host: None
firefox_syncserver | max_requests_jitter: 0
firefox_syncserver | post_fork: <function post_fork at 0xffff9cfc2f50>
firefox_syncserver | pythonpath: None
firefox_syncserver | enable_stdio_inheritance: False
firefox_syncserver | worker_class: sync
firefox_syncserver | ssl_version: 3
firefox_syncserver | suppress_ragged_eofs: True
firefox_syncserver | syslog: False
firefox_syncserver | syslog_facility: user
firefox_syncserver | when_ready: <function when_ready at 0xffff9cfc2c50>
firefox_syncserver | pre_fork: <function pre_fork at 0xffff9cfc2dd0>
firefox_syncserver | cert_reqs: 0
firefox_syncserver | preload_app: False
firefox_syncserver | keepalive: 2
firefox_syncserver | accesslog: None
firefox_syncserver | group: 1000
firefox_syncserver | graceful_timeout: 30
firefox_syncserver | do_handshake_on_connect: False
firefox_syncserver | spew: False
firefox_syncserver | workers: 1
firefox_syncserver | proc_name: None
firefox_syncserver | sendfile: None
firefox_syncserver | pidfile: None
firefox_syncserver | umask: 0
firefox_syncserver | on_reload: <function on_reload at 0xffff9cfc2ad0>
firefox_syncserver | pre_exec: <function pre_exec at 0xffff9cfc85d0>
firefox_syncserver | worker_tmp_dir: None
firefox_syncserver | post_worker_init: <function post_worker_init at 0xffff9cfc8150>
firefox_syncserver | limit_request_fields: 100
firefox_syncserver | on_exit: <function on_exit at 0xffff9cfc8cd0>
firefox_syncserver | config: None
firefox_syncserver | logconfig: None
firefox_syncserver | check_config: False
firefox_syncserver | statsd_prefix:
firefox_syncserver | proxy_allow_ips: ['127.0.0.1']
firefox_syncserver | pre_request: <function pre_request at 0xffff9cfc8750>
firefox_syncserver | post_request: <function post_request at 0xffff9cfc8850>
firefox_syncserver | user: 1000
firefox_syncserver | forwarded_allow_ips: ['127.0.0.1']
firefox_syncserver | worker_int: <function worker_int at 0xffff9cfc82d0>
firefox_syncserver | threads: 1
firefox_syncserver | max_requests: 0
firefox_syncserver | limit_request_line: 4094
firefox_syncserver | access_log_format: %(h)s %(l)s %(u)s %(t)s "%(r)s" %(s)s %(b)s "%(f)s" "%(a)s"
firefox_syncserver | certfile: None
firefox_syncserver | worker_exit: <function worker_exit at 0xffff9cfc89d0>
firefox_syncserver | chdir: /
firefox_syncserver | paste: /opt/syncserver/syncserver.ini
firefox_syncserver | default_proc_name: /opt/syncserver/syncserver.ini
firefox_syncserver | errorlog: -
firefox_syncserver | loglevel: debug
firefox_syncserver | capture_output: False
firefox_syncserver | syslog_addr: udp://localhost:514
firefox_syncserver | syslog_prefix: None
firefox_syncserver | daemon: False
firefox_syncserver | ciphers: TLSv1
firefox_syncserver | on_starting: <function on_starting at 0xffff9cfc2950>
firefox_syncserver | worker_abort: <function worker_abort at 0xffff9cfc8450>
firefox_syncserver | bind: ['0.0.0.0:5000']
firefox_syncserver | raw_env: []
firefox_syncserver | reload: False
firefox_syncserver | limit_request_field_size: 8190
firefox_syncserver | nworkers_changed: <function nworkers_changed at 0xffff9cfc8b50>
firefox_syncserver | timeout: 30
firefox_syncserver | ca_certs: None
firefox_syncserver | django_settings: None
firefox_syncserver | tmp_upload_dir: None
firefox_syncserver | keyfile: None
firefox_syncserver | backlog: 2048
firefox_syncserver | logger_class: gunicorn.glogging.Logger
firefox_syncserver | [2022-09-08 11:00:32 +0000] [1] [INFO] Starting gunicorn 19.6.0
firefox_syncserver | [2022-09-08 11:00:32 +0000] [1] [DEBUG] Arbiter booted
firefox_syncserver | [2022-09-08 11:00:32 +0000] [1] [INFO] Listening at: http://0.0.0.0:5000 (1)
firefox_syncserver | [2022-09-08 11:00:32 +0000] [1] [INFO] Using worker: sync
firefox_syncserver | [2022-09-08 11:00:32 +0000] [16] [INFO] Booting worker with pid: 16
firefox_syncserver | [2022-09-08 11:00:32 +0000] [1] [DEBUG] 1 workers
firefox_syncserver | /usr/local/lib/python2.7/site-packages/pyramid/path.py:341: PkgResourcesDeprecationWarning: Parameters to load are deprecated. Call .resolve and .require separately.
firefox_syncserver | 'x=%s' % value).load(False)
firefox_syncserver | /usr/local/lib/python2.7/site-packages/jwt/utils.py:8: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release.
firefox_syncserver | from cryptography.hazmat.primitives.asymmetric.utils import (
nginx reverse proxy config:
location /moz-sync {
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
proxy_redirect off;
proxy_read_timeout 120;
proxy_connect_timeout 10;
proxy_pass http://[DockerMachineIP]:5000/;
}
I noticed one potential issue in the docker container. While it's running I opened a shell and looked at the app/docker-entrypoint.sh file, which starts off with:
#!/bin/sh
cd $(dirname $0)
case "$1" in
server)
export SYNCSERVER_SQLURI="${SYNCSERVER_SQLURI:-sqlite:///tmp/syncserver.db}"
exec gunicorn \
--bind ${HOST-0.0.0.0}:${PORT-5000} \
--forwarded-allow-ips="${SYNCSERVER_FORWARDED_ALLOW_IPS:-127.0.0.1,172.17.0.1}" \
syncserver.wsgi_app
;;
It lists this variable SYNCSERVER_FORWARDED_ALLOW_IPS while I believe the variable name should actually be FF_SYNCSERVER_FORWARDED_ALLOW_IPS.
That said, I'm not versed in docker and wasn't actually sure how to make changes to that file to test this as each time the docker is stopped and restarted, the change seems to revert.