openvstorage
Update our core dependencies
Problem description
While looking around for certain bugs/annoyances in our libraries, I discovered that we are 4.5 years behind on updates on our core dependencies:
Front-end
- Knockout:
- Installed: 3.3.0
- Candidate: 3.4.2
- Durandal:
- Installed: 2.0.1
- Candidate: 2.2.0
- JQuery:
- Installed: 1.9.1
- Candidate: 2.1
Back-end
- Celery:
- Installed: 3.1.23-5
- Candidate: 4.2
- RabbitMQ:
- Installed: 3.5.7
- Candidate: 3.6.12
- Django Rest Framework (API):
- Installed: 2.3.12
- Candidate: 3.7.1
- Python:
- Installed: 2.7
- Candidate: 3.6.2
Needless to say, being this behind is handicapping us in terms of feature and bug fixes
Update progress
- [x] JQuery (https://github.com/openvstorage/framework/issues/1840)
- [x] Durandal (https://github.com/openvstorage/framework/issues/1341)
- [x] Knockout (https://github.com/openvstorage/framework/issues/1341)
- [ ] Celery
- [ ] RabbitMQ (https://github.com/openvstorage/framework/issues/1497)
- [ ] DJango Rest Framework
- [ ] Python (https://github.com/openvstorage/home/issues/42)
Reworking certain GUI aspects for https://github.com/openvstorage/framework/issues/1656
Picked up the following tickets:
- https://github.com/openvstorage/framework/issues/1341
- https://github.com/openvstorage/framework/issues/1840
@JeffreyDevloo can you add an estimate for the update of Celery, rabbitmq and Django . What would be the impact on the code? Maybe we can update to the latest minor version of the one we are currently on instead of the latest latest?
Update to Python 3 would be out of scope.
@wimpers I'd estimate every update to be at least a weeks worth of testing/bugfixing The impact of Django is rather high due to all decorators needing a visit The impact of both Celery and Rabbitmq is currently unknown to me
Gunicorn currently has a bug in deleting temporary files on the filesystem which are already gone from the directory metadata. These files are filling up the disk and can only be deleted when you restart the webapp-api.
PR Gunicorn: https://github.com/benoitc/gunicorn/pull/1428
This PR got merged in December 2017 but the latest Gunicorn release is from March 2017 so it is not yet possible to implement a fixed release.
I've also noticed that the ubuntu 16.04 is also running behind. It is at version 19.4.5 while the latest release is 19.7.1 (implemented in ubuntu 18.04).
root@DEV-3N-199-181:~# lsof |grep -i deleted | grep gunicorn
gunicorn 7130 root 6u REG 253,1 0 262242 /tmp/wgunicorn-WvrCzS (deleted)
gunicorn 7130 root 7u REG 253,1 0 279988 /tmp/wgunicorn-Lb2C_G (deleted)
gunicorn 7130 root 8u REG 253,1 0 281825 /tmp/wgunicorn-nZmI6u (deleted)
gunicorn 7130 root 9u REG 253,1 0 281826 /tmp/wgunicorn-OB4bdy (deleted)
gunicorn 7130 root 10u REG 253,1 0 281842 /tmp/wgunicorn-KlODhS (deleted)
gunicorn 7156 root 6u REG 253,1 0 262242 /tmp/wgunicorn-WvrCzS (deleted)
gunicorn 7156 20618 root 6u REG 253,1 0 262242 /tmp/wgunicorn-WvrCzS (deleted)
gunicorn 7156 20648 root 6u REG 253,1 0 262242 /tmp/wgunicorn-WvrCzS (deleted)
gunicorn 7156 27429 root 6u REG 253,1 0 262242 /tmp/wgunicorn-WvrCzS (deleted)
gunicorn 7158 root 6u REG 253,1 0 262242 /tmp/wgunicorn-WvrCzS (deleted)
root@DEV-3N-199-181:~# lsof |grep -i deleted | grep gunicorn -c
73
Rabbitmq has a bug https://github.com/rabbitmq/rabbitmq-server/issues/714 between master and slaves (deadlocks). This has been fixed in 3.6.x. Today GIG ended up with this scenario. See ticket http://support.openvstorage.com/browse/OVSSUP-429
Digital Energy had the same rabbitmq issue as described in the comment above.
DE had another instance of this http://tickets.openvstorage.com/browse/OVSSUP-453
Note that 18.04 comes with RabbitMQ 3.6.10 so maybe better to with that version instead of 3.612 or 3.7.X.
I'm running RabbitMQ 3.7.4-1 on my test environment.
root@LC1SRV0008:~# apt-cache policy rabbitmq-server
rabbitmq-server:
Installed: 3.7.4-1
Candidate: 3.7.4-1
Version table:
*** 3.7.4-1 500
See ansible role how to setup: https://github.com/openvstorage/setups/tree/qa/openvstorage/qa/VIR-18N-50.10/roles/externalRabbitMQ