owncloud
Unit tests failing on Ubuntu 20.04 with PHP 7.4
Yesterday the owncloudci/php:7.4 docker images was updated to use Ubuntu 20.04 as the base image (was previously Ubuntu 18.04). See PR https://github.com/owncloud-ci/php/pull/128
Now some unit tests are failing. See demonstration in PR #38346
Maybe Ubuntu 20.04 is missing some things that were installed by default in Ubuntu 18.04, or...?
Investigate and fix (or revert the docker image change for now)
An example in the nightly CI - files_primary_s3 running the core unit tests with PHP 7.4 https://drone.owncloud.com/owncloud/files_primary_s3/2077/8/7
There were 8 failures:
1) Test\LargeFileHelperGetFileSizeTest::testGetFileSizeViaCurl with data set #0 ('/var/www/owncloud/server/test...em.txt', 446)
Failed asserting that null is identical to 446.
/var/www/owncloud/server/tests/lib/LargeFileHelperGetFileSizeTest.php:53
2) Test\LargeFileHelperGetFileSizeTest::testGetFileSizeViaCurl with data set #1 ('/var/www/owncloud/server/test...2).txt', 446)
Failed asserting that null is identical to 446.
/var/www/owncloud/server/tests/lib/LargeFileHelperGetFileSizeTest.php:53
3) Test\Preview\BitmapTest::testGetThumbnail with data set #0 (-68, -77)
Failed asserting that 0.77 matches expected 1.41.
/var/www/owncloud/server/tests/lib/Preview/Provider.php:155
/var/www/owncloud/server/tests/lib/Preview/Provider.php:104
4) Test\Preview\BitmapTest::testGetThumbnail with data set #1 (48, 24)
Failed asserting that 0.77 matches expected 1.41.
/var/www/owncloud/server/tests/lib/Preview/Provider.php:155
/var/www/owncloud/server/tests/lib/Preview/Provider.php:104
5) Test\Preview\BitmapTest::testGetThumbnail with data set #2 (-95, 73)
Failed asserting that 0.77 matches expected 1.41.
/var/www/owncloud/server/tests/lib/Preview/Provider.php:155
/var/www/owncloud/server/tests/lib/Preview/Provider.php:104
6) Test\Preview\BitmapTest::testGetThumbnail with data set #3 (49, -34)
Failed asserting that 0.77 matches expected 1.41.
/var/www/owncloud/server/tests/lib/Preview/Provider.php:155
/var/www/owncloud/server/tests/lib/Preview/Provider.php:104
7) Tests\Core\Templates\TemplatesTest::test403
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">\n
-<html><body><ul><li class="error">\n
- Access forbidden<br><p class="hint"></p>\n
-</li></ul></body></html>\n
+<html><body><ul>\n
+<li class="error">\n
+ Access forbidden<br>\n
+ <p class="hint"></p>\n
+ </li>\n
+</ul></body></html>\n
'
/var/www/owncloud/server/tests/lib/TestCase.php:506
/var/www/owncloud/server/tests/lib/TestCase.php:484
/var/www/owncloud/server/tests/Core/Templates/TemplatesTest.php:9
8) Tests\Core\Templates\TemplatesTest::test404
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
'<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN" "http://www.w3.org/TR/REC-html40/loose.dtd">\n
-<html><body><ul><li class="error">\n
- File not found<br><p class="hint">The specified document has not been found on the server.</p>\n
-<p class="hint"><a href="/owncloud/">You can click here to return to ownCloud.</a></p>\n
- </li></ul></body></html>\n
+<html><body><ul>\n
+<li class="error">\n
+ File not found<br>\n
+ <p class="hint">The specified document has not been found on the server.</p>\n
+ <p class="hint"><a href="/owncloud/">You can click here to return to ownCloud.</a></p>\n
+ </li>\n
+ </ul></body></html>\n
'
/var/www/owncloud/server/tests/lib/TestCase.php:506
/var/www/owncloud/server/tests/lib/TestCase.php:484
/var/www/owncloud/server/tests/Core/Templates/TemplatesTest.php:16
--
I have reverted the changes to owncloudci/php:7.4 so that existing CI can be "great again" - PR https://github.com/owncloud-ci/php/pull/129
PR https://github.com/owncloud-ci/php/pull/130 has created owncloudci/php:7.4-ubuntu20.04 docker image.
I will use that to test the phpunit, Behat, etc tests and get it passing properly before putting Ubuntu 20.04 into the "real" owncloudci/php:7.4 docker image.
Test\LargeFileHelperGetFileSizeTest::testGetFileSizeViaCurl seems to be a php_curl regression:
# curl -I file:///home/deo/public_html/oc-tmp/tests/data/lorem.txt
Content-Length: 446
Accept-ranges: bytes
Last-Modified: Mon, 17 Jun 2013 07:25:11 GMT
Headers are here. But the script
<?php
$ch = \curl_init("file:///home/deo/public_html/oc-tmp/tests/data/lorem.txt");
\curl_setopt_array(
$ch,
[
CURLOPT_NOBODY => true,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HEADER => true
]
);
$data = \curl_exec($ch);
\curl_close($ch);
var_dump($data);
returns no headers:
string(0) ""
I'm pretty sure file protocol is enabled - after removing CURLOPT_NOBODY => true, the file content is dumped.
and it does return headers when the resource is changed from
file:///home/deo/public_html/oc-tmp/tests/data/lorem.txt into e.g. http://localhost/oc/core/js/login.js?v=988646cb965e528fd57397e8b9b4468f
So CLI curl returns headers for the file protocol while php curl extendsion doesn't.
@phil-davis this one fails for me even with php 7.3 when running locally, so seems like the problem is with ubuntu 20.04.
Note: this needs doing in the next month or two, because it will be good to run all CI with Ubuntu 20.04 before PHP 8.* gets progress.
So CLI curl returns headers for the file protocol while php curl extension doesn't.
I found this which reports the same problem:
https://stackoverflow.com/questions/59709625/curlfile-scheme-not-returns-headers-in-php-7-3
I am starting an Ubuntu 18.04 VM so that I can try it with PHP 7.4 etc.
Update:
This cURL code to get the headers works on Ubuntu 18.04 with any of the PHP that I tried - 7.2 7.3 and 7.4
Exactly the same code fails to return any header information on Ubuntu 20.04 with any version of PHP - I tried 5.6 7.0 7.1 7.2 7.3 7.4 8.0 8.1. So I guess that there is something different about libcurl.
On my systems: Ubuntu 18.04
$ curl-config --version
libcurl 7.58.0
Ubuntu 20.04
$ curl-config --version
libcurl 7.68.0
This issue looks like where the regressions was reported: https://github.com/curl/curl/issues/3083 And fixed in commit https://github.com/curl/curl/commit/e50a2002bd450a4800a165d2874ed79c95b33a07 But that commit seems to be included in curl 7.62 - so 7.68 should have the fix - what is going on???
I have confirmed that CURLOPT_HEADER for file:// works on Ubuntu 18.04 (in a VM that I installed)
It does not work on Ubuntu 20.04 or Ubuntu 22.04.
I found a way to use CURLOPT_HEADERFUNCTION and that works on all Ubuntu 18.04 20.04 and 22.04 - see PR #40065
PRs #40054 #40059 and #40065 fix all the issues so that the code, tests and CI all pass on both Ubuntu 18.04 and Ubuntu 20.04.
After those are reviewed and merged, then I will make another PR to run PHP tests with 7.4-ubuntu20.04, and it should pass.
Currently blocked waiting for core 10.10.0 final release. Then I can merge the waiting PRs and continue...
@phil-davis owncloud core 10.10.0 has been released. can we move forward on this?
Now blocked by #40054 - carddav and caldav tests need python2, but Ubuntu 22.04 has python3 by default.
@xoxys has said that he will update those test suites so that they run with python3: https://github.com/owncloud/core/pull/40054#issuecomment-1122167023
The PHP unit tests in master now pass on Ubuntu 22.04
PR #40054 was merged. It explicitly uses python2 in the caldav/carddav test scripts. We will stay with that for now.
The next step is testing with Ubuntu 22.04 - see PR #40123
After that is passing, I will think about what to do for regular CI. Maybe we will wait until a few other infrastructure things have happened - for example, the guzzle v7 update and dropping PHP 7.3 support. Then we can decide what to do about the test matrix for PHP 7.4.
closing. see https://github.com/owncloud/files_primary_s3/issues/613#issuecomment-1278846592
