dataverse icon indicating copy to clipboard operation
dataverse copied to clipboard

Use StandardCharsets instead of charset names

Open bencomp opened this issue 2 years ago • 1 comments

What this PR does / why we need it: This fixes a (potential) resource leak as described in #10053 and it replaces all uses of "utf-8", "us-ascii" and "iso-8859-1" (mostly in upper case though) to get a charset with StandardCharsets.X. The constants in StandardCharsets are supported on all Java implementations (since its introduction), so using them in String.getBytes(..) will not throw an UnsupportedEncodingException. This may simplify the code.

Which issue(s) this PR closes:

Closes #10076 Closes #10053

Special notes for your reviewer: Apologies for mixing these two issues in one PR.

Suggestions on how to test this: See that the test suites pass.

Does this PR introduce a user interface change? If mockups are available, please link/include them here: No

Is there a release notes update needed for this change?: No

Additional documentation: Also linked from #10076, Sonarcloud's explanation for using StandardCharsets may be helpful.

bencomp avatar Oct 31 '23 11:10 bencomp

Coverage Status

coverage: 20.661% (+0.002%) from 20.659% when pulling 641bb32216874b45dbecd0e6f4e6b44e224b6bd9 on bencomp:10076-standardcharsets into d113d7c358147b9caa7e4b148807dd28eb32c9d9 on IQSS:develop.

coveralls avatar Jun 18 '24 14:06 coveralls

Closing due to inactivity. Please reopen if this issue is still relevant.

stevenwinship avatar Jul 08 '24 19:07 stevenwinship

I didn't have time earlier; thanks for reviewing. Could you reopen this PR?

bencomp avatar Jul 09 '24 14:07 bencomp

Coverage Status

coverage: 20.752% (+0.004%) from 20.748% when pulling b37a7f80fe76a27ee3d6003327b1b3f9fe89e16e on bencomp:10076-standardcharsets into 0966ca9e1c1854410e87e89b15eca82930fc8257 on IQSS:develop.

coveralls avatar Jul 09 '24 14:07 coveralls

Could you take a look at this failing test? testDatasetThumbnail – edu.harvard.iq.dataverse.api.SearchIT

stevenwinship avatar Jul 11 '24 18:07 stevenwinship

Could you show me the output of the failed test?

bencomp avatar Jul 11 '24 21:07 bencomp

I’ll send it when I can. You should be able to view it from the PR

Get Outlook for iOShttps://aka.ms/o0ukef


From: Ben Companjen @.> Sent: Thursday, July 11, 2024 5:15:09 PM To: IQSS/dataverse @.> Cc: Winship, Steven @.>; Assign @.> Subject: Re: [IQSS/dataverse] Use StandardCharsets instead of charset names (PR #10077)

Could you show me the output of the failed test?

— Reply to this email directly, view it on GitHubhttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_IQSS_dataverse_pull_10077-23issuecomment-2D2223951477&d=DwMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=6i2oYMjEVIkJXmF6jh7cbc9JSEHUF10ousKf6CtNSeM&m=Dq17w4LPVTxWbnLDGhnDSfmzh5CeJlJszaMHaJVQfCeECKLGYZWlyDFH5L1kUmHJ&s=6Qy3mQh90smng0KjnyQHl7oBEA40nYSThUwWuXyEg4U&e=, or unsubscribehttps://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AJPMLJKYRA57IQP7WIMVL23ZL3YV3AVCNFSM6AAAAABJQDXP4WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRTHE2TCNBXG4&d=DwMCaQ&c=WO-RGvefibhHBZq3fL85hQ&r=6i2oYMjEVIkJXmF6jh7cbc9JSEHUF10ousKf6CtNSeM&m=Dq17w4LPVTxWbnLDGhnDSfmzh5CeJlJszaMHaJVQfCeECKLGYZWlyDFH5L1kUmHJ&s=Dn707FRyXi5g9fP4GTDDA406feu2rRyzUOTcKagM_pk&e=. You are receiving this because you were assigned.Message ID: @.***>

stevenwinship avatar Jul 11 '24 21:07 stevenwinship

Error

1 expectation failed. JSON path data.message doesn't match. Expected: Thumbnail is now data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADAAAAAQCAYAAABQrvyxAAAB6ElEQVR4XtWWsY3CQBBF6YASaACJEhAFIEqgAkROQkJEAHRACZRACQREJJBgmYsQyd35JDSnZ/TX69WCrLvA4ktf7O7szs6fGds0Gg/YmzKHvSskIly38XhsrVarpHY6ndr9frcsy6zdboeZqJOFgOv16gyDwcB2u12+vl6v3Xq/37fz+WydTid0VBcLAYyHw6GbI2C73bo5otgzGo1sv9+XHFEh7e12u46aKxm6T+f8NRLo2yrycZiWaTab+RiEGRZOp1M+v91uNplMnB0BEMQECPLl+8VO20aCq8LCoTKgTBMQzhlzgcB8sVjY8Xh0jtirMeKpxmazKQXIGjbgC9bduiMS5CuWy+g78Me6GCCQoNI0DZ05Uk0Yrr9iWPWKfC6AtiKLjP02iAng2SHDqgR7VAXmjPkly8vl0vnXWNXRPv1W4PMW8m2xFkqSxO1T4ASsajHHl1pFLaXgeJ60DyhJ/gNfgX97iPkezGYzZ5cAMqpAmDOWAIAICQAkjbuBzmEHkWBjLLcOGRPC16hEUQ3eQr4jtZC+GQoaH8yxcY4kSSTExhns/2oh4H/I5Iw19SkkULLf6/VCR3WxECCQvfCvBCIAfycOh4OtViubz+e10gn4yb4qM/v+zKvycbnYJU1qpRPwzvwFfFwt0uprmeYAAAAASUVORK5CYII= Actual: null Stacktrace

java.lang.AssertionError: 1 expectation failed. JSON path data.message doesn't match. Expected: Thumbnail is now data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADAAAAAQCAYAAABQrvyxAAAB6ElEQVR4XtWWsY3CQBBF6YASaACJEhAFIEqgAkROQkJEAHRACZRACQREJJBgmYsQyd35JDSnZ/TX69WCrLvA4ktf7O7szs6fGds0Gg/YmzKHvSskIly38XhsrVarpHY6ndr9frcsy6zdboeZqJOFgOv16gyDwcB2u12+vl6v3Xq/37fz+WydTid0VBcLAYyHw6GbI2C73bo5otgzGo1sv9+XHFEh7e12u46aKxm6T+f8NRLo2yrycZiWaTab+RiEGRZOp1M+v91uNplMnB0BEMQECPLl+8VO20aCq8LCoTKgTBMQzhlzgcB8sVjY8Xh0jtirMeKpxmazKQXIGjbgC9bduiMS5CuWy+g78Me6GCCQoNI0DZ05Uk0Yrr9iWPWKfC6AtiKLjP02iAng2SHDqgR7VAXmjPkly8vl0vnXWNXRPv1W4PMW8m2xFkqSxO1T4ASsajHHl1pFLaXgeJ60DyhJ/gNfgX97iPkezGYzZ5cAMqpAmDOWAIAICQAkjbuBzmEHkWBjLLcOGRPC16hEUQ3eQr4jtZC+GQoaH8yxcY4kSSTExhns/2oh4H/I5Iw19SkkULLf6/VCR3WxECCQvfCvBCIAfycOh4OtViubz+e10gn4yb4qM/v+zKvycbnYJU1qpRPwzvwFfFwt0uprmeYAAAAASUVORK5CYII= Actual: null at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480) at org.codehaus.groovy.reflection.CachedConstructor.invoke(CachedConstructor.java:73) at org.codehaus.groovy.runtime.callsite.ConstructorSite$ConstructorSiteNoUnwrapNoCoerce.callConstructor(ConstructorSite.java:108) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallConstructor(CallSiteArray.java:57) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:263) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callConstructor(AbstractCallSite.java:277) at io.restassured.internal.ResponseSpecificationImpl$HamcrestAssertionClosure.validate(ResponseSpecificationImpl.groovy:512) at io.restassured.internal.ResponseSpecificationImpl$HamcrestAssertionClosure$validate$1.call(Unknown Source) at io.restassured.internal.ResponseSpecificationImpl.validateResponseIfRequired(ResponseSpecificationImpl.groovy:696) at io.restassured.internal.ResponseSpecificationImpl.this$2$validateResponseIfRequired(ResponseSpecificationImpl.groovy) at jdk.internal.reflect.GeneratedMethodAccessor144.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.codehaus.groovy.runtime.callsite.PlainObjectMetaMethodSite.doInvoke(PlainObjectMetaMethodSite.java:43) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:198) at org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:62) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:185) at io.restassured.internal.ResponseSpecificationImpl.body(ResponseSpecificationImpl.groovy:270) at io.restassured.specification.ResponseSpecification$body$1.callCurrent(Unknown Source) at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:49) at io.restassured.specification.ResponseSpecification$body$1.callCurrent(Unknown Source) at io.restassured.internal.ResponseSpecificationImpl.body(ResponseSpecificationImpl.groovy:117) at io.restassured.internal.ValidatableResponseOptionsImpl.body(ValidatableResponseOptionsImpl.java:244) at edu.harvard.iq.dataverse.api.SearchIT.testDatasetThumbnail(SearchIT.java:611) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) Standard Output

{ "status": "OK", "data": { "user": { "id": 127, "userName": "user67c53d9e" }, "authenticatedUser": { "id": 127, "identifier": "@user67c53d9e", "displayName": "user67c53d9e user67c53d9e", "firstName": "user67c53d9e", "lastName": "user67c53d9e", "email": "[email protected]", "superuser": false, "deactivated": false, "persistentUserId": "user67c53d9e", "createdTime": "2024-07-09T15:00:22Z", "lastLoginTime": "2024-07-09T15:00:22Z", "authenticationProviderId": "builtin" }, "apiToken": "e53e7d72-af67-4ae2-a2dd-b0e38f9ea46e" } } { "status": "OK", "data": { "id": 237, "alias": "dveade0eee", "name": "dveade0eee", "dataverseContacts": [ { "displayOrder": 0, "contactEmail": "[email protected]" } ], "permissionRoot": true, "dataverseType": "UNCATEGORIZED", "ownerId": 1, "creationDate": "2024-07-09T15:00:22Z", "isReleased": false } } { "status": "OK", "data": { "id": 238, "persistentId": "doi:10.5072/FK2/X3GTB1" } } {"status":"OK","data":{"createTime":"2024-07-09T15:00:22.956","hasStaleIndex":false,"hasStalePermissionIndex":false,"lastUpdateTime":"2024-07-09T15:00:22.956","lastIndexTime":"2024-07-09T15:00:23.055","lastPermissionUpdateTime":"2024-07-09T15:00:22.956","lastPermissionIndexTime":"2024-07-09T15:00:23.055","globalIdCreateTime":"2024-07-09T15:00:22.956"}} false Waited 1.0 seconds { "status": "OK", "data": { "q": "id:dataset_238_draft", "total_count": 1, "start": 0, "spelling_alternatives": {

    },
    "items": [
        {
            "name": "Darwin's Finches",
            "type": "dataset",
            "url": "https://doi.org/10.5072/FK2/X3GTB1",
            "global_id": "doi:10.5072/FK2/X3GTB1",
            "description": "Darwin's finches (also known as the Galápagos finches) are a group of about fifteen species of passerine birds.",
            "publisher": "dveade0eee",
            "citationHtml": "Finch, Fiona, 2024, \"Darwin's Finches\", <a href=\"https://doi.org/10.5072/FK2/X3GTB1\" target=\"_blank\">https://doi.org/10.5072/FK2/X3GTB1</a>, Root, DRAFT VERSION",
            "identifier_of_dataverse": "dveade0eee",
            "name_of_dataverse": "dveade0eee",
            "citation": "Finch, Fiona, 2024, \"Darwin's Finches\", https://doi.org/10.5072/FK2/X3GTB1, Root, DRAFT VERSION",
            "storageIdentifier": "file://10.5072/FK2/X3GTB1",
            "subjects": [
                "Medicine, Health and Life Sciences"
            ],
            "fileCount": 0,
            "versionId": 91,
            "versionState": "DRAFT",
            "createdAt": "2024-07-09T15:00:22Z",
            "updatedAt": "2024-07-09T15:00:22Z",
            "contacts": [
                {
                    "name": "Finch, Fiona",
                    "affiliation": ""
                }
            ],
            "authors": [
                "Finch, Fiona"
            ]
        }
    ],
    "count_in_response": 1
}

} { "status": "OK", "data": { "id": 238, "identifier": "FK2/X3GTB1", "persistentUrl": "https://doi.org/10.5072/FK2/X3GTB1", "protocol": "doi", "authority": "10.5072", "publisher": "Root", "storageIdentifier": "file://10.5072/FK2/X3GTB1", "latestVersion": { "id": 91, "datasetId": 238, "datasetPersistentId": "doi:10.5072/FK2/X3GTB1", "storageIdentifier": "file://10.5072/FK2/X3GTB1", "versionState": "DRAFT", "latestVersionPublishingState": "DRAFT", "lastUpdateTime": "2024-07-09T15:00:22Z", "createTime": "2024-07-09T15:00:22Z", "license": { "name": "CC0 1.0", "uri": "http://creativecommons.org/publicdomain/zero/1.0", "iconUri": "https://licensebuttons.net/p/zero/1.0/88x31.png" }, "fileAccessRequest": true, "metadataBlocks": { "citation": { "displayName": "Citation Metadata", "name": "citation", "fields": [ { "typeName": "title", "multiple": false, "typeClass": "primitive", "value": "Darwin's Finches" }, { "typeName": "author", "multiple": true, "typeClass": "compound", "value": [ { "authorName": { "typeName": "authorName", "multiple": false, "typeClass": "primitive", "value": "Finch, Fiona" }, "authorAffiliation": { "typeName": "authorAffiliation", "multiple": false, "typeClass": "primitive", "value": "Birds Inc." } } ] }, { "typeName": "datasetContact", "multiple": true, "typeClass": "compound", "value": [ { "datasetContactName": { "typeName": "datasetContactName", "multiple": false, "typeClass": "primitive", "value": "Finch, Fiona" }, "datasetContactEmail": { "typeName": "datasetContactEmail", "multiple": false, "typeClass": "primitive", "value": "[email protected]" } } ] }, { "typeName": "dsDescription", "multiple": true, "typeClass": "compound", "value": [ { "dsDescriptionValue": { "typeName": "dsDescriptionValue", "multiple": false, "typeClass": "primitive", "value": "Darwin's finches (also known as the Galápagos finches) are a group of about fifteen species of passerine birds." } } ] }, { "typeName": "subject", "multiple": true, "typeClass": "controlledVocabulary", "value": [ "Medicine, Health and Life Sciences" ] } ] } }, "files": [

        ]
    }
}

} { "status": "OK", "data": { "user": { "id": 128, "userName": "user48039bde" }, "authenticatedUser": { "id": 128, "identifier": "@user48039bde", "displayName": "user48039bde user48039bde", "firstName": "user48039bde", "lastName": "user48039bde", "email": "[email protected]", "superuser": false, "deactivated": false, "persistentUserId": "user48039bde", "createdTime": "2024-07-09T15:00:25Z", "lastLoginTime": "2024-07-09T15:00:25Z", "authenticationProviderId": "builtin" }, "apiToken": "250cdbf3-a824-455a-87ad-e3fbe630dcc7" } } { "status": "OK", "data": { "isUseGenericThumbnail": false, "datasetLogoPresent": false } } { "status": "ERROR", "message": "Thumbnail not available" } { "status": "ERROR", "message": "You are not permitted to list dataset thumbnail candidates." } { "status": "OK", "data": [

]

} { "status": "ERROR", "message": "Thumbnail not available" } <bibliographicCitation xmlns="http://purl.org/dc/terms/">Finch, Fiona, 2024, "Darwin's Finches", https://doi.org/10.5072/FK2/X3GTB1, Root, DRAFT VERSION</bibliographicCitation> http://ec2-3-234-217-136.compute-1.amazonaws.com/dvn/api/data-deposit/v1.1/swordv2/edit/study/doi:10.5072/FK2/X3GTB1

no treatment information available datafileId: 239 {"status":"OK","data":{"createTime":"2024-07-09T15:00:22.956","hasStaleIndex":false,"hasStalePermissionIndex":false,"lastUpdateTime":"2024-07-09T15:00:26.225","lastIndexTime":"2024-07-09T15:00:26.643","lastPermissionUpdateTime":"2024-07-09T15:00:22.956","lastPermissionIndexTime":"2024-07-09T15:00:26.642","globalIdCreateTime":"2024-07-09T15:00:22.956"}} false Waited 1.0 seconds { "status": "OK", "data": { "q": "id:dataset_238_draft", "total_count": 1, "start": 0, "spelling_alternatives": {
    },
    "items": [
        {
            "name": "Darwin's Finches",
            "type": "dataset",
            "url": "https://doi.org/10.5072/FK2/X3GTB1",
            "global_id": "doi:10.5072/FK2/X3GTB1",
            "description": "Darwin's finches (also known as the Galápagos finches) are a group of about fifteen species of passerine birds.",
            "publisher": "dveade0eee",
            "citationHtml": "Finch, Fiona, 2024, \"Darwin's Finches\", <a href=\"https://doi.org/10.5072/FK2/X3GTB1\" target=\"_blank\">https://doi.org/10.5072/FK2/X3GTB1</a>, Root, DRAFT VERSION",
            "identifier_of_dataverse": "dveade0eee",
            "name_of_dataverse": "dveade0eee",
            "citation": "Finch, Fiona, 2024, \"Darwin's Finches\", https://doi.org/10.5072/FK2/X3GTB1, Root, DRAFT VERSION",
            "storageIdentifier": "file://10.5072/FK2/X3GTB1",
            "subjects": [
                "Medicine, Health and Life Sciences"
            ],
            "fileCount": 1,
            "versionId": 91,
            "versionState": "DRAFT",
            "createdAt": "2024-07-09T15:00:22Z",
            "updatedAt": "2024-07-09T15:00:26Z",
            "contacts": [
                {
                    "name": "Finch, Fiona",
                    "affiliation": ""
                }
            ],
            "authors": [
                "Finch, Fiona"
            ]
        }
    ],
    "count_in_response": 1
}

} getThumbnail2: { "status": "OK", "data": { "isUseGenericThumbnail": false, "datasetLogoPresent": false } } http://ec2-3-234-217-136.compute-1.amazonaws.com/dvn/api/data-deposit/v1.1/swordv2/edit/dataverse/dveade0eee

no treatment information available { "status": "OK", "data": [ { "lockType": "finalizePublication", "date": "Tue Jul 09 15:00:28 UTC 2024", "user": "user67c53d9e", "dataset": "doi:10.5072/FK2/X3GTB1", "message": "Publishing the dataset; Validating Datafiles Asynchronously" } ] } { "status": "OK", "data": [ { "lockType": "finalizePublication", "date": "Tue Jul 09 15:00:28 UTC 2024", "user": "user67c53d9e", "dataset": "doi:10.5072/FK2/X3GTB1", "message": "Publishing the dataset; Validating Datafiles Asynchronously" } ] } { "status": "OK", "data": { "id": 238, "identifier": "FK2/X3GTB1", "persistentUrl": "https://doi.org/10.5072/FK2/X3GTB1", "protocol": "doi", "authority": "10.5072", "publisher": "Root", "storageIdentifier": "file://10.5072/FK2/X3GTB1" } } getThumbnail3: { "status": "OK", "data": { "isUseGenericThumbnail": false, "datasetLogoPresent": false, "datasetThumbnailBase64image": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADAAAAAaCAYAAADxNd/XAAAEjklEQVR4Xu2Va0xTZxjHDcOzUjy2oWW6Gdg0WbIlusRpTMwUglO3IYIdujkYyNXKKCrITVDBGwIOBBRvUxA6E8AIQx27CHNB4rKQaDDyyREgyxI1Ln5AQTTy29smkvhi9dR9cGb+vjR5zvP/n/7fy3PG8YIzTi68aLwM8Lx5GeB58/8NcPKnDLnkNvfu3aO2vhh73T5OnrVQWJbM5StdctsTeaYA/X91c6Rpmlx2iwcPHhCdNonmjoXUnAmk5UI4Na0zCbJMoL2jVW53yTMFyKuYSmW9P6tzDPIjTdy5c4fqUwuxbteTWTKZErs/u6onYyt+lU0lvgSEvsK5jp9l2WNxO8C20nAyi3xI3q5SUu1HTKr7Ibq7u4lI01FsN7C7ZiqX/4gmNltHaJwHp8/PIGmrytwlCr39vbJ0DG4HWJtnYut+Hw42TCExV0d2wRy55akEhHmQkGsgbLXC+YsrKDvhR8flQJrPz6S83kBqoUrSFgO2ApVv6w7J8kdwO0BUioG0nT40tU9jvfgNjzNSdjhZbnNJRPwHbNynsnarGesmlbOdH3KwcSr2sz7srp3Ege9eI6XQi5gsbxatVDhm/0a2eATNATovtZG+Ywo2sb0bSsVZrVTJO2AUL9KRkvue3O6SdRtiCV+jsPeEmYxSAxUnzRQcM9HY/i4WqyeFdl+yin1YJxan7ISe9N2TnNPKFZoD3Lr1N6lb/NhzzJ9Dp820/P4OueW+LFjmxY6iVLndJaFxXswP8yQxX093XwT2H/xpv7iI9q5AjjTO5rcri0ncomdXjYngCIX3Az3p7++XbUbRHMCaZWZdnpkvUhTCEjz53KaQJs5qsvgj8RmqmCwDsuSxzP1YYb64oJsrfIXWIIaBIoKsIj53PBe6F3P1zzRWrh9PZIaCZY0XS6IUZsyZLNuMoinA0ZpSPl1lIm69WZx7M8W1Pth26rGJsxoSoydgqcIvv56RZWMYGhrCEj/ROcUsiXoSslWsOTq2H/Xl3KUguq5GCO/Xic1VCIlXWJGsZ6G4Bytjg2WrUTQFWPXVTD6zGgkRq7HAohAe78WsIE/mBSssX60nMkUlJs0oy8Zw9+5dZgcaxIj0oEiM4OgNOqpazGSWquTsH8/G/SqnOmazYIWH2GUvMWpVLAkTuX37tmw1ylMDhEdPZ5HFiC3PSP5eXwqqTHxtN2HNV9lx1EBmuUqimCb5FX6ERU2gueW4bPEIPT09JOWbxBAwclBc4KRt3th/nO48Sku+dIxXHVHZCqU1bxMUKhYndqnzq+2KpwZ4SGTiPJLTQyg6MIXDDW8ya564aJVvUFblT+QaI4NDg7LksYyMjFB7/DAxSR/R/H0D8xcbSd/8CRFWb7HLJgKXKRQeeotTLU3cuHFDlo9BcwCZuvTlculf0Xk8Ry5pQnMAx8o5tnJwcJDe3l7qM5Zz/fp1hoeHn7jFrnjo59Bfu3aNTnsWfX19Tn9H3fFcC24FcHxQbt68SWtrK9XV1XR1dTkv2P379zW/8CGOfoduYGDA6VNVVUVbW5vT3/EerX6aA/xXeRngefMPZUq14FqxpKIAAAAASUVORK5CYII=", "dataFileId": "239" } } �PNG &amp#26; &amp#0;&amp#0;&amp#0; IHDR&amp#0;&amp#0;&amp#0;0&amp#0;&amp#0;&amp#0;&amp#26;&amp#8;&amp#6;&amp#0;&amp#0;&amp#0;�5��&amp#0;&amp#0;&amp#4;�IDATx^�kLSg&amp#24;� óRN��PX���+]r�&amp#19;y�&amp#0;�us�i�\v�&amp#7;&amp#15;&amp#30;&amp#16;�6�掅Ԝ ��B85�3 �L���Unw�3&amp#5;ȫ�Je�?�s&amp#12;�#Mܹs��S&amp#11;�nדY2�&amp#18;�?��'c+~�M%�&amp#4;��¹��e�cq;���p2�|HޮRR�GL��!����H�Ql7��f*���&6[Gh�&amp#7;��� i���% ����t&amp#12;n&amp#7;X�gb�~&amp#31;&amp#14;6L!1WGv�&amp#28;��&amp#4;�y��k l����+(;�G��@��Ϥ��@j�J�&amp#22;&amp#3;�&amp#2;�o�&amp#14;��Gp;@T����>4�Oc�� �3Rv8YnsID�&amp#7;lܧ�v�&amp#25;�&���&amp#31;r�q*��>쮝ā�^#�Ћ�,o&amp#22;�T8f�F�x&amp#4;�&amp#1;:/���c 6��&amp#27;J�Y�T�;`&amp#20;/ґ�����u&amp#27;b _������R&amp#3;&amp#21;'�&amp#20;&amp#28;3���.&amp#22;�'�v_��}X'&amp#22;�섞�ݓ���&amp#21;�&amp#3;ܺ�7�[��s̟C�ʹ��&amp#14;��,X�Ŏ�T��%�q^�&amp#15;�$1_Ow_&amp#4;�&amp#31;�i�����@�4��+�IܢgW���&amp#8;��&amp#3;=���mF�&amp#28;��ef]��/R&amp#20;�&amp#18;w]e"n�Y�{3ŵ>�v걉�&amp#26;&amp#18;�'`��/���ec&amp#24;&amp#26;&amp#26;�&amp#18;?�9�,�z&amp#18;�U�9:�&amp#31;��ܥ ��F&amp#8;�׉�U&amp#8;�WX��g��&amp#7;+c�e�Q4&amp#5;X��L>�&amp#26; &amp#17;����&amp#16;&amp#30;�Ŭ O�&amp#5;+,_�'2E%&�(��p��]f&amp#7;&amp#26;Ĉ��H��� :�Z�d����&amp#31;���*�:f�`���e/1jU, &amp#19;�}��l5�S&amp#3;�GOg�ň-�H�^_ �L|m7a�W�q�@f�J��&�&amp#21;~�EM���l�&amp#8;===$��&amp#16;0rP\�m����N��PX���+]r�&amp#19;y�&amp#0;�us�i�\v�&amp#7;&amp#15;&amp#30;&amp#16;�6�掅Ԝ ��B85�3 �L���Unw�3&amp#5;ȫ�Je�?�s&amp#12;�#Mܹs��S&amp#11;�nדY2�&amp#18;�?��'c+~�M%�&amp#4;��¹��e�cq;���p2�|HޮRR�GL��!����H�Ql7��f*���&6[Gh�&amp#7;��� i���% ����t&amp#12;n&amp#7;X�gb�~&amp#31;&amp#14;6L!1WGv�&amp#28;��&amp#4;�y��k l����+(;�G��@��Ϥ��@j�J�&amp#22;&amp#3;�&amp#2;�o�&amp#14;��Gp;@T����>4�Oc�� �3Rv8YnsID�&amp#7;lܧ�v�&amp#25;�&���&amp#31;r�q*��>쮝ā�^#�Ћ�,o&amp#22;�T8f�F�x&amp#4;�&amp#1;:/���c 6��&amp#27;J�Y�T�;`&amp#20;/ґ�����u&amp#27;b _������R&amp#3;&amp#21;'�&amp#20;&amp#28;3���.&amp#22;�'�v_��}X'&amp#22;�섞�ݓ���&amp#21;�&amp#3;ܺ�7�[��s̟C�ʹ��&amp#14;��,X�Ŏ�T��%�q^�&amp#15;�$1_Ow_&amp#4;�&amp#31;�i�����@�4��+�IܢgW���&amp#8;��&amp#3;=���mF�&amp#28;��ef]��/R&amp#20;�&amp#18;w]e"n�Y�{3ŵ>�v걉�&amp#26;&amp#18;�'`��/���ec&amp#24;&amp#26;&amp#26;�&amp#18;?�9�,�z&amp#18;�U�9:�&amp#31;��ܥ ��F&amp#8;�׉�U&amp#8;�WX��g��&amp#7;+c�e�Q4&amp#5;X��L>�&amp#26; &amp#17;����&amp#16;&amp#30;�Ŭ O�&amp#5;+,_�'2E%&�(��p��]f&amp#7;&amp#26;Ĉ��H��� :�Z�d����&amp#31;���*�:f�`���e/1jU, &amp#19;�}��l5�S&amp#3;�GOg�ň-�H�^_ �L|m7a�W�q�@f�J��&�&amp#21;~�EM���l�&amp#8;===$��&amp#16;0rP\�m���� }, "items": [ { "name": "Darwin's Finches", "type": "dataset", "url": "https://doi.org/10.5072/FK2/X3GTB1", "global_id": "doi:10.5072/FK2/X3GTB1", "description": "Darwin's finches (also known as the Galápagos finches) are a group of about fifteen species of passerine birds.", "publisher": "dveade0eee", "citationHtml": "Finch, Fiona, 2024, \"Darwin's Finches\", <a href=\"https://doi.org/10.5072/FK2/X3GTB1\" target=\"_blank\">https://doi.org/10.5072/FK2/X3GTB1</a>, Root, DRAFT VERSION", "identifier_of_dataverse": "dveade0eee", "name_of_dataverse": "dveade0eee", "citation": "Finch, Fiona, 2024, \"Darwin's Finches\", https://doi.org/10.5072/FK2/X3GTB1, Root, DRAFT VERSION", "storageIdentifier": "file://10.5072/FK2/X3GTB1", "subjects": [ "Medicine, Health and Life Sciences" ], "fileCount": 2, "versionId": 92, "versionState": "DRAFT", "createdAt": "2024-07-09T15:00:22Z", "updatedAt": "2024-07-09T15:00:32Z", "contacts": [ { "name": "Finch, Fiona", "affiliation": "" } ], "authors": [ "Finch, Fiona" ] } ], "count_in_response": 1 }

} { "status": "OK", "data": [ { "base64image": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADAAAAAwCAYAAABXAvmHAAACV0lEQVR4Xu2Wv2obQRDG9Qh6BJUq1bgX8WvYLt2kEeQBVMWtCj+AGnfGkgikCwhcBIXYqIqME4PkP5UElhIiSGGQ9U34LnOjXdkBo70z+4NlZmdm9/bbXZ2usMg5BRvIG1FAaKKA0EQBoYkCQhMFhOZ1CDg+6Ugbjyc2nyl6O2VpmtQJfP9xJUIAbVa4PTm0ISElACfA08giT55AHnAKoDKXQs263Etz2XhrQ4JdQ3ICTOgCLcgKxJ2E/zD/tZLXY0ZHB4kPBu/3kv7PwRfx2bfPsugYfa8Aa32+thDzdX9rJU6rmwb9fm1b7Pi0lcpp9A+Zc3gFgD+TO+eidd+1QF+fuw+hNmfrXegcfa8Aa32+jdm4tvr66Ho7hw8rHIgAHh/wWfo27orZPK0VQB/XjjHW6Dxi+jka72sUbwEIA/pe8x5yMuZcoAbX0IdekBZB5qOLVN+KA14BobFiiI1vTMD//rvbhRIb35gAYoXYPln3OtVsXADQi7Yfjzpnd9v2QRABL4kI+D2fp4L8Ih2OrqX/bfDvbeA78lCIAL2o6WwmVn9Wdz58TGJZ43VcoYvz3uJm+Qd12zpc3J99ShcU/moslUpip9NpKu4CtaxbR7vdFrturqeQkb3dsrTPy1/5uzfmPeuYnDGbq1arYrvd7qJer4tfq9XERy1asVhcGa/ngXjW9vv9JI+GTcF4TTKSAshwOEx2sdFopB5SqVSkD0swcbPZFF/vPuownlAMLBej57FgM7h4F6vb6wGTQFTWeLaArBIFhCYKCE0UEJooIDRRQGhyL+ARlkVm269A600AAAAASUVORK5CYII=", "dataFileId": 240 }, { "base64image": "data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAADAAAAAaCAYAAADxNd/XAAAEjklEQVR4Xu2Va0xTZxjHDcOzUjy2oWW6Gdg0WbIlusRpTMwUglO3IYIdujkYyNXKKCrITVDBGwIOBBRvUxA6E8AIQx27CHNB4rKQaDDyyREgyxI1Ln5AQTTy29smkvhi9dR9cGb+vjR5zvP/n/7fy3PG8YIzTi68aLwM8Lx5GeB58/8NcPKnDLnkNvfu3aO2vhh73T5OnrVQWJbM5StdctsTeaYA/X91c6Rpmlx2iwcPHhCdNonmjoXUnAmk5UI4Na0zCbJMoL2jVW53yTMFyKuYSmW9P6tzDPIjTdy5c4fqUwuxbteTWTKZErs/u6onYyt+lU0lvgSEvsK5jp9l2WNxO8C20nAyi3xI3q5SUu1HTKr7Ibq7u4lI01FsN7C7ZiqX/4gmNltHaJwHp8/PIGmrytwlCr39vbJ0DG4HWJtnYut+Hw42TCExV0d2wRy55akEhHmQkGsgbLXC+YsrKDvhR8flQJrPz6S83kBqoUrSFgO2ApVv6w7J8kdwO0BUioG0nT40tU9jvfgNjzNSdjhZbnNJRPwHbNynsnarGesmlbOdH3KwcSr2sz7srp3Ege9eI6XQi5gsbxatVDhm/0a2eATNATovtZG+Ywo2sb0bSsVZrVTJO2AUL9KRkvue3O6SdRtiCV+jsPeEmYxSAxUnzRQcM9HY/i4WqyeFdl+yin1YJxan7ISe9N2TnNPKFZoD3Lr1N6lb/NhzzJ9Dp820/P4OueW+LFjmxY6iVLndJaFxXswP8yQxX093XwT2H/xpv7iI9q5AjjTO5rcri0ncomdXjYngCIX3Az3p7++XbUbRHMCaZWZdnpkvUhTCEjz53KaQJs5qsvgj8RmqmCwDsuSxzP1YYb64oJsrfIXWIIaBIoKsIj53PBe6F3P1zzRWrh9PZIaCZY0XS6IUZsyZLNuMoinA0ZpSPl1lIm69WZx7M8W1Pth26rGJsxoSoydgqcIvv56RZWMYGhrCEj/ROcUsiXoSslWsOTq2H/Xl3KUguq5GCO/Xic1VCIlXWJGsZ6G4Bytjg2WrUTQFWPXVTD6zGgkRq7HAohAe78WsIE/mBSssX60nMkUlJs0oy8Zw9+5dZgcaxIj0oEiM4OgNOqpazGSWquTsH8/G/SqnOmazYIWH2GUvMWpVLAkTuX37tmw1ylMDhEdPZ5HFiC3PSP5eXwqqTHxtN2HNV9lx1EBmuUqimCb5FX6ERU2gueW4bPEIPT09JOWbxBAwclBc4KRt3th/nO48Sku+dIxXHVHZCqU1bxMUKhYndqnzq+2KpwZ4SGTiPJLTQyg6MIXDDW8ya564aJVvUFblT+QaI4NDg7LksYyMjFB7/DAxSR/R/H0D8xcbSd/8CRFWb7HLJgKXKRQeeotTLU3cuHFDlo9BcwCZuvTlculf0Xk8Ry5pQnMAx8o5tnJwcJDe3l7qM5Zz/fp1hoeHn7jFrnjo59Bfu3aNTnsWfX19Tn9H3fFcC24FcHxQbt68SWtrK9XV1XR1dTkv2P379zW/8CGOfoduYGDA6VNVVUVbW5vT3/EerX6aA/xXeRngefMPZUq14FqxpKIAAAAASUVORK5CYII=", "dataFileId": 239 } ] } { "status": "ERROR", "message": "File is larger than maximum size: 500000" } { "status": "ERROR", "message": "Command edu.harvard.iq.dataverse.engine.command.impl.UpdateDatasetThumbnailCommand@3bca05e5 failed: Exception thrown from bean: java.lang.NullPointerException: Cannot invoke "edu.harvard.iq.dataverse.Dataset.setThumbnailFile(edu.harvard.iq.dataverse.DataFile)" because "dataset" is null" } Standard Error

Jul 09, 2024 3:00:22 PM edu.harvard.iq.dataverse.api.SearchIT testDatasetThumbnail INFO: BEGIN testDatasetThumbnail Jul 09, 2024 3:00:22 PM edu.harvard.iq.dataverse.api.UtilIT createRandomUser INFO: Creating random test user user67c53d9e Jul 09, 2024 3:00:22 PM edu.harvard.iq.dataverse.api.UtilIT getUsernameFromResponse INFO: Username found in create user response: user67c53d9e Jul 09, 2024 3:00:22 PM edu.harvard.iq.dataverse.api.UtilIT getApiTokenFromResponse INFO: API token found in create user response: e53e7d72-af67-4ae2-a2dd-b0e38f9ea46e Jul 09, 2024 3:00:22 PM edu.harvard.iq.dataverse.api.UtilIT getAliasFromResponse INFO: Alias found in create dataverse response: dveade0eee Jul 09, 2024 3:00:23 PM edu.harvard.iq.dataverse.api.UtilIT getDatasetIdFromResponse INFO: Id found in create dataset response: 238 Jul 09, 2024 3:00:24 PM edu.harvard.iq.dataverse.api.UtilIT createRandomUser INFO: Creating random test user user48039bde Jul 09, 2024 3:00:25 PM edu.harvard.iq.dataverse.api.UtilIT getUsernameFromResponse INFO: Username found in create user response: user48039bde Jul 09, 2024 3:00:25 PM edu.harvard.iq.dataverse.api.UtilIT getApiTokenFromResponse INFO: API token found in create user response: 250cdbf3-a824-455a-87ad-e3fbe630dcc7 Jul 09, 2024 3:00:25 PM edu.harvard.iq.dataverse.api.SearchIT testDatasetThumbnail INFO: Dataset created, no thumbnail expected: Jul 09, 2024 3:00:26 PM edu.harvard.iq.dataverse.api.UtilIT uploadFile INFO: Number of bytes to upload: 8460 Jul 09, 2024 3:00:26 PM edu.harvard.iq.dataverse.api.SearchIT testDatasetThumbnail INFO: DataFile uploaded, should automatically become the thumbnail: Jul 09, 2024 3:00:32 PM edu.harvard.iq.dataverse.api.SearchIT testDatasetThumbnail INFO: Second DataFile has been uploaded and switched to as the thumbnail:

stevenwinship avatar Jul 11 '24 22:07 stevenwinship

FWIW: The test fails at https://github.com/IQSS/dataverse/blob/5ba74e8a8e75fecec7f4b477abb920453212c53e/src/main/java/edu/harvard/iq/dataverse/DatasetServiceBean.java#L852-L853 (on the server side) - perhaps some of the changes to catching Exceptions in this PR can trigger that?

qqmyers avatar Jul 15 '24 15:07 qqmyers

Thanks for the output, @stevenwinship and that pointer, @qqmyers – it's a big verbose test (perhaps it can be broken up into smaller ones at some point?). I fixed my mistake of creating a FileInputStream for writing instead of a FileOutputStream and hope this solves the failure.

Can you see if the test passes?

bencomp avatar Jul 15 '24 20:07 bencomp

Seems to still have the same test failure: ``` Caused by: java.lang.NullPointerException: Cannot invoke "edu.harvard.iq.dataverse.Dataset.setThumbnailFile(edu.harvard.iq.dataverse.DataFile)" because "dataset" is null at edu.harvard.iq.dataverse.DatasetServiceBean.setNonDatasetFileAsThumbnail(DatasetServiceBean.java:853)

qqmyers avatar Jul 17 '24 15:07 qqmyers

After some investigation, local testing and adjusting log messages and log levels, I now understand less of what persistDatasetLogoToStorageAndCreateThumbnails does. That method returns null when it catches an IOException in the block starting on line 335 in DatasetUtil:

try {
    dest.transferFrom(src, 0, src.size());
} catch (IOException ex) {
    logger.severe("dest.transferFrom failed: " + ex.getMessage());
    return null;
}

Looking closer, it appears that this copies/transfers from and to itself. src is a FileInputStream's FileChannel created from tmpFile, dest is a FileOutputStream's FileChannel created from tmpFile. The log message is not helpful: dest.transferFrom failed: null.

I did change that part of the code to use try-with-resources, so I will revert that and see what happens.

bencomp avatar Jul 22 '24 10:07 bencomp

While I don't yet understand this transfer, I suddenly dawned on me that the operations needed to be run in the same code block, because of the try-with-resources construct that closes things at the end of the block.

The test now passes on my machine (in Docker) :)

bencomp avatar Jul 22 '24 10:07 bencomp

Hey @bencomp. sorry, can I bother you to update your branch from dev again? thanks!

sekmiller avatar Aug 13 '24 13:08 sekmiller