Provide a setting to use a different REST url during SSR execution

[atarix83]

References

Add references/links to any related issues or PRs. These may include:

• Fixes #1485
• Requires DSpace/DSpace#9856

Description

This pull request provide the possibility to use a different DSpace REST url during the SSR

Instructions for Reviewers

List of changes in this PR:

• Added a new setting ssrBaseUrl where to specify a different DSpace REST url to use during SSR
• Added a new interceptor which replaces the base url used to contact the REST server, if a different DSpace REST url is provided
• Changed the behaviour of the thumbnail component. The thumbnail component now renders exclusively in the browser. This change is necessary to prevent issues during server-side rendering (SSR). When a page containing a thumbnail is rendered on the server, the HTML content delivered to the browser would attempt to download the thumbnail using the URL specified in the ssrBaseUrl property. If this URL was not publicly accessible, it would result in an error. By rendering the thumbnail only in the browser, we avoid this problem.
• An additional change is required with the ServerHardRedirectService. In case the url to redirect contains the ssrBaseUrl it's reinstated with the public base url. This is necessary otherwise download bitstream page doesn't work.

Include guidance for how to test or review your PR. Unfortunately it is difficult to test this PR. The best would be to deploy it in a production like environment and configure the internal base url property. This might be done on both Angular and REST side, like in the example below where public base url is https://dspace-rest.org/server and internal base url is http://localhost:8080/server.

• Angular: in your config.prod.yml file set the ssrBaseUrl property, e.g :

rest:
  ssl: true
  host: dspace-rest.org
  port: 443
  nameSpace: /server
  ssrBaseUrl: http://localhost:8080/server

• REST: in your local.cfg file set the dspace.server.ssr.url property, e.g. :

# Public URL of DSpace backend ('server' webapp). May require a port number if not using standard ports (80 or 443)
# DO NOT end it with '/'.
# This is where REST API and all enabled server modules (OAI-PMH, SWORD, SWORDv2, RDF, etc) will respond.
# NOTE: This URL must be accessible to all DSpace users (should not use 'localhost' in Production)
# and is usually "synced" with the "rest" section in the DSpace User Interface's config.*.yml.
# It corresponds to the URL that you would type into your browser to access the REST API.
dspace.server.url = https://dspace-rest.org/server

# Additional URL of DSpace backend which could be used by DSpace frontend during SSR execution.
# May require a port number if not using standard ports (80 or 443)
# DO NOT end it with '/'.
dspace.server.ssr.url = http://localhost:8080/server

In order to test it locally you could add an alias hostname for localhost ip to be used as public url

Checklist

• [x] My PR is created against the main branch of code (unless it is a backport or is fixing an issue specific to an older branch).
• [x] My PR is small in size (e.g. less than 1,000 lines of code, not including comments & specs/tests), or I have provided reasons as to why that's not possible.
• [x] My PR passes ESLint validation using npm run lint
• [x] My PR doesn't introduce circular dependencies (verified via npm run check-circ-deps)
• [x] My PR includes TypeDoc comments for all new (or modified) public methods and classes. It also includes TypeDoc for large or complex private methods.
• [x] My PR passes all specs/tests and includes new/updated specs or tests based on the Code Testing Guide.
• [x] My PR aligns with Accessibility guidelines if it makes changes to the user interface.
• [x] My PR uses i18n (internationalization) keys instead of hardcoded English text, to allow for translations.
• [x] My PR includes details on how to test it. I've provided clear instructions to reviewers on how to successfully test this fix or feature.
• [x] If my PR includes new libraries/dependencies (in package.json), I've made sure their licenses align with the DSpace BSD License based on the Licensing of Contributions documentation.
• [x] If my PR includes new features or configurations, I've provided basic technical documentation in the PR itself.
• [x] If my PR fixes an issue ticket, I've linked them together.

[tdonohue]

@atarix83 : While I realize this is still in "draft", I'm assigning this to @artlowel and I for review. @artlowel , I know you have a lot on your plate, but this seems like a high priority fix that would be good to get your or your team's feedback on.

[github-actions[bot]]

Hi @atarix83, Conflicts have been detected against the base branch. Please resolve these conflicts as soon as you can. Thanks!

[atarix83]

@tdonohue @ybnd

I've improved my implementation trying to address your feedback. Here the main changes:

• added a new config property to disable transfer state (see here). By default even if the property is enabled when a different SSR base url is set the state is not transferred from server to client application.
• added a new config to allow url replacement in the state transferred to the client application. default to false.

[github-actions[bot]]

Hi @atarix83, Conflicts have been detected against the base branch. Please resolve these conflicts as soon as you can. Thanks!

[atarix83]

@tdonohue

Authentication doesn't work. This might be a side effect of an error I'm seeing on main, but here's what I'm trying.

I had difficult to find the problem, but i think i found it. Could you please try again running this branch https://github.com/4Science/DSpace/tree/task/main/DURACOM-288_fix_hal_map on the REST side. If it works I'll update the REST PR with IT

Whenever SSR is triggered, I see a 404 request to an /undefined path. This appears in my browser's DevTools "Network" tab.

It should be resolved with my last commit, could you check?

Thumbnail images appear to be accessed initially via the private URL.

I can't reproduce this issue. I encountered the problem during the implementation which led me to change the ServerHardRedirectService. Could you try again with latest changes?

I'm still working on addressing the other feedback

[pnbecker]

Hi @atarix83,

thank you for this important pr!

I tried to set the rest.ssrBaseUrl via environment variables and it failed. I added the following two lines to config.server.ts to configure it via environment variable. While I was able to get it running, the angular logs did not mentioned that it used the environment variable, which it normally does. Maybe you can take a closer look please?

  appConfig.rest.ssrBaseUrl = isNotEmpty(ENV('REST_SSRBASEURL', true)) ? ENV('REST_SSRBASEURL', true) : appConfig.rest.ssrBaseUrl;
  appConfig.rest.hasSsrBaseUrl = isNotEmpty(ENV('REST_SSRBASEURL', true)) ? isNotEmpty(ENV('REST_SSRBASEURL', true)) : appConfig.rest.hasSsrBaseUrl;

Regarding authentication: I was able to authenticate and it worked until I let my browser reload the page. Once I reloaded the page I was logged out. I have to test the branch you mentioned above, but did not had the time yet to do that.

So far, my test installation is an empty repository. I will add items and thumbnails and check the commit you mentioned above for the authentication. I will test more asap.

Best, Pascal

[github-actions[bot]]

Hi @atarix83, Conflicts have been detected against the base branch. Please resolve these conflicts as soon as you can. Thanks!

[atarix83]

@tdonohue I pushed changes to REST PR, could you please take a look? thanks

[github-actions[bot]]

Hi @atarix83, Conflicts have been detected against the base branch. Please resolve these conflicts as soon as you can. Thanks!

[atarix83]

@pnbecker

I've added the missing code in the config.server.ts (Please mind that only REST_SSRBASEURL is needed) I gave a test using this PR via environment variable and it worked for me. The ssrBaseUrl is valued despite it is not in the config.prod.yml.

Can i ask how have you set the environment variables during your test? I did using the following commands:

DSPACE_REST_SSRBASEURL='http://dspace.rest.org:8080/server'
export DSPACE_REST_SSRBASEURL

[pnbecker]

@atarix83 @tdonohue I updated the code and gave it another test. I still have the issue, that when I login to DSpace and reload the page in my browser, I'm getting logged out. Everything else works fine, but reloading the page should not log me out.

[pnbecker]

I have a setup here, when I remove the ssr base url from the configuration I get a 500 as angular is not allowed to access the official url of the rest api, while my browser can access it. When I login, the dsAuth cookie is created. It is valid for 24 hours. When I reload, the cookie is being removed.

@atarix83 would it be helpful to gain access to my test installation? Do you have a static IPv4 address?

[pnbecker]

Actually in Safari I do see the dsAuthInfo cookie until I refresh. In Chrome and Firefox I never see that cookie. In all three browser after refreshing the page, I'm logged out.

[tdonohue]

@pnbecker : @atarix83 just created dspace-8_x branches of this PR (and the backend one), so that you can try it out on 8.x instead of pre-9.0:

@atarix83 posted this to Slack:

here the branches for dspace-8_x angular https://github.com/4Science/dspace-angular/tree/task/dspace-8_x/DURACOM-288 REST https://github.com/4Science/DSpace/tree/task/dspace-8_x/DURACOM-288

When you have a chance, could you test this instead using the 8.x version? It might help us temporarily work around the issues you've hit on main, and ensure this code is working for the upcoming 8.1 release.

[grenierdev]

@tdonohue Sorry to jump in.

I'm trying to accomplish the same configuration and I found a way to test everything in a "production-like" with Docker. Docker offer some similarities with a "production-like" environment : containers can access each other internally and only what is exposed can be access by the host with localhost:4000.

It almost work. Seems like dspace-angular is initially using the SSR endpoint http://dpsace:8080/server but after I visit the main page, it use the public endpoint http://localhost:8080/server.

2025-02-05 12:20:17 {"message":"No _links section found at http://localhost:8080/server/api","timestamp":"2025-02-05T17:20:17.402Z","type":"err","process_id":7,"app_name":"dspace-ui"}

Steps to reproduce

1. Start containers

docker-compose -f docker-compose.yml -p d8 up -d

docker-compose.yml

#
# The contents of this file are subject to the license and copyright
# detailed in the LICENSE and NOTICE files at the root of the source
# tree and available online at
#
# http://www.dspace.org/license/
#

# Docker Compose for running the DSpace backend for testing/development
# This is based heavily on the docker-compose.yml that is available in the DSpace/DSpace
# (Backend) at:
# https://github.com/DSpace/DSpace/blob/main/docker-compose.yml
networks:
  dspacenet:
    ipam:
      config:
        # Define a custom subnet for our DSpace network, so that we can easily trust requests from host to container.
        # If you customize this value, be sure to customize the 'proxies.trusted.ipranges' env variable below.
        - subnet: 172.23.0.0/16
services:
  # DSpace (backend) webapp container
  dspace:
    container_name: dspace
    environment:
      # Below syntax may look odd, but it is how to override dspace.cfg settings via env variables.
      # See https://github.com/DSpace/DSpace/blob/main/dspace/config/config-definition.xml
      # __P__ => "." (e.g. dspace__P__dir => dspace.dir)
      # __D__ => "-" (e.g. google__D__metadata => google-metadata)
      # dspace.dir, dspace.server.url, dspace.ui.url and dspace.name
      dspace__P__dir: /dspace
      # Uncomment to set a non-default value for dspace.server.url or dspace.ui.url
      dspace__P__server__P__url: http://localhost:8080/server
      dspace__P__server__P__ssr__P__url: http://dspace:8080/server
      dspace__P__ui__P__url: http://localhost:4000
      dspace__P__name: 'ONICSE'
      # db.url: Ensure we are using the 'dspacedb' image for our database
      db__P__url: 'jdbc:postgresql://dspacedb:5432/dspace'
      # solr.server: Ensure we are using the 'dspacesolr' image for Solr
      solr__P__server: http://dspacesolr:8983/solr
      # proxies.trusted.ipranges: This setting is required for a REST API running in Docker to trust requests 
      # from the host machine. This IP range MUST correspond to the 'dspacenet' subnet defined above.
      proxies__P__trusted__P__ipranges: '172.23.0'
      LOGGING_CONFIG: /dspace/config/log4j2-container.xml
    image: "${DOCKER_REGISTRY:-docker.io}/${DOCKER_OWNER:-dspace}/dspace:${DSPACE_VER:-dspace-8_x}"
    depends_on:
    - dspacedb
    networks:
      - dspacenet
    ports:
    - published: 8080
      target: 8080
    stdin_open: true
    tty: true
    volumes:
    # Keep DSpace assetstore directory between reboots
    - assetstore:/dspace/assetstore
    # Ensure that the database is ready BEFORE starting tomcat
    # 1. While a TCP connection to dspacedb port 5432 is not available, continue to sleep
    # 2. Then, run database migration to init database tables
    # 3. Finally, start DSpace
    entrypoint:
    - /bin/bash
    - '-c'
    - |
      while (!</dev/tcp/dspacedb/5432) > /dev/null 2>&1; do sleep 1; done;
      /dspace/bin/dspace database migrate
      java -jar /dspace/webapps/server-boot.jar --dspace.dir=/dspace
  # DSpace database container    
  dspacedb:
    container_name: dspacedb
    # Uses a custom Postgres image with pgcrypto installed
    image: "${DOCKER_REGISTRY:-docker.io}/${DOCKER_OWNER:-dspace}/dspace-postgres-pgcrypto:${DSPACE_VER:-latest}"
    environment:
      PGDATA: /pgdata
      POSTGRES_PASSWORD: dspace
    networks:
      - dspacenet
    ports:
    - published: 5432
      target: 5432
    stdin_open: true
    tty: true
    volumes:
    # Keep Postgres data directory between reboots
    - pgdata:/pgdata
  # DSpace Solr container  
  dspacesolr:
    container_name: dspacesolr
    image: "${DOCKER_REGISTRY:-docker.io}/${DOCKER_OWNER:-dspace}/dspace-solr:${DSPACE_VER:-latest}"
    networks:
      - dspacenet
    ports:
    - published: 8983
      target: 8983
    stdin_open: true
    tty: true
    working_dir: /var/solr/data
    volumes:
    # Keep Solr data directory between reboots
    - solr_data:/var/solr/data
    # Initialize all DSpace Solr cores using the mounted local configsets (see above), then start Solr
    # * First, run precreate-core to create the core (if it doesn't yet exist). If exists already, this is a no-op
    # * Second, copy configsets to this core:
    #   Updates to Solr configs require the container to be rebuilt/restarted:
    #   `docker compose -p d7 -f docker/docker-compose.yml -f docker/docker-compose-rest.yml up -d --build dspacesolr`
    entrypoint:
    - /bin/bash
    - '-c'
    - |
      init-var-solr
      precreate-core authority /opt/solr/server/solr/configsets/authority
      cp -r /opt/solr/server/solr/configsets/authority/* authority
      precreate-core oai /opt/solr/server/solr/configsets/oai
      cp -r /opt/solr/server/solr/configsets/oai/* oai
      precreate-core search /opt/solr/server/solr/configsets/search
      cp -r /opt/solr/server/solr/configsets/search/* search
      precreate-core statistics /opt/solr/server/solr/configsets/statistics
      cp -r /opt/solr/server/solr/configsets/statistics/* statistics
      precreate-core qaevent /opt/solr/server/solr/configsets/qaevent
      cp -r /opt/solr/server/solr/configsets/qaevent/* qaevent
      precreate-core suggestion /opt/solr/server/solr/configsets/suggestion
      cp -r /opt/solr/server/solr/configsets/suggestion/* suggestion
      exec solr -f
  dspace-angular:
    container_name: dspace-angular
    environment:
      DSPACE_UI_SSL: 'false'
      DSPACE_UI_HOST: dspace-angular
      DSPACE_UI_PORT: 4000
      DSPACE_UI_NAMESPACE: /
      DSPACE_REST_SSL: 'false'
      DSPACE_REST_HOST: localhost
      DSPACE_REST_PORT: 8080
      DSPACE_REST_NAMESPACE: /server
      DSPACE_REST_SSRBASEURL: 'http://dspace:8080/server'
    image: "${DOCKER_REGISTRY:-docker.io}/${DOCKER_OWNER:-dspace}/dspace-angular:${DSPACE_VER:-dspace-8_x-dist}"
    # build:
    #   context: ./dspace-angular
    #   dockerfile: Dockerfile.dist
    networks:
      - dspacenet
    ports:
    - published: 4000
      target: 4000
    - published: 9876
      target: 9876
    stdin_open: true
    tty: true
volumes:
  assetstore:
  pgdata:
  solr_data:

2. Visit http://localhost:4000

What is happening

In the Chrome DevTool, I can confirm that the rest API uses the public endpoint http://localhost:8080. But after the initial load of the app, the dspace-angular container complains that No _links section found at http://localhost:8080/server/api. It seems like it doesn't use the SSR endpoint dspace:8080.

[tdonohue]

@grenierdev : If it fails immediately (after initial load), that sounds like a possible misconfiguration, or maybe you forgot to rebuild either the frontend or backend images using this PR or https://github.com/DSpace/DSpace/pull/9856 ? Or maybe Docker's network settings are somehow blocking the requests?

I'm not able to reproduce that immediate failure in all my testing of these two PRs while using https://localxpose.io/ . When using LocalXPose, I'm able to create a temporary public URL for the dspace.server.url while keeping dspace.server.ssr.url set to http://localhost:8080/server When I do that, these PRs work well, and I've not noticed any change in behavior for DSpace. (I'm able to login, submit, search, etc)

That said, I'll admit, I haven't tried setting these params via environment variables (though that would be odd if they only worked if set directly in configs and not via environment variables) or tried this sort of Docker approach.

I'll see if I can find time to try out the Docker approach. While I agree it hypothetically should work, I'm a bit worried that Docker networking (which can sometimes be tricky) could be getting in the way here.

[pnbecker]

I tested the backport to DSpace 8 of this PR in a production environment. There were two URLs to access the backend: one was accessible only from my browser, the other one from the frontend. The one accessible from my browser filtered the access based on the connecting ip address, the other one was routed via a private IP address space which my browser had no access to. The frontend was able to access the backend only via the private ip.

I was able to login, to submit a new Item, to load thumbnails and files with JavaScript switched off, and to load the sitemap. I was not able to search and browse without JavaScript, which is not a surprise after we merged the PR to limit SSR to certain paths and to exclude search and browse from ssr.

👍 I did not review the code, I just tested it.

[grenierdev]

@tdonohue I had the impression that the branch dspace-8_x was this PR. Sorry for my confusion.

I used the branches of both PR in Docker and everything worked. Even with the environment variables.

The frontend is accessible on http://localhost:4000, the API is on http://localhost:8080 and accessible within the containers (internal network of docker) with http://dspace:8080.

Great work!

[tdonohue]

Merging with +2 approvals (and an additional approval from @grenierdev as well, thanks!). Thanks again @atarix83 !

[tdonohue]

@atarix83 : As discussed in today's meeting, this will need to be ported manually to dspace-7_x. Please let me know when you have a backport PR completed.