sct
Dependency on Github causes status endpoint to fail
Description
After today's Github outage my Overseerr installation went into a crash loop because I use the /api/v1/status endpoint as a healthcheck probe in my Kubernetes deployment. This affected both a running, working stack and a stack attempting to start up.
IMO, being able to fetch the latest Overseerr release should not be a requirement for the application to be considered 'healthy'. There also doesn't seem to be a way to disable this functionality.
In the event that Github is down or there is a problem fetching releases, would it be okay to just fall back to updateAvailable: false?
From Kubernetes:
Last State: Terminated
Reason: Error
Exit Code: 1
Started: Thu, 15 Aug 2024 09:37:34 +1000
Finished: Thu, 15 Aug 2024 09:38:04 +1000
Version
1.33.2
Steps to Reproduce
Difficult to reproduce unless Github is down (returning 503) or you proxy requests through an SSL-enabled MITM proxy. (Alternatively point the URL for Github at a user-controlled endpoint in the code and have it return 503).
Screenshots
No response
Logs
yarn run v1.22.19
warning Skipping preferred cache folder "/.cache/yarn" because it is not writable.
warning Selected the next writable cache folder in the list, will be "/tmp/.yarn-cache-568".
$ NODE_ENV=production node dist/index.js
warning Cannot find a suitable global folder. Tried these: "/usr/local, /.yarn"
2024-08-14T23:37:35.668Z [info]: Commit Tag: $GIT_SHA
2024-08-14T23:37:35.918Z [info]: Starting Overseerr version 1.33.2
warn - You have enabled experimental features (scrollRestoration, largePageDataBytes) in next.config.js.
warn - Experimental features are not covered by semver, and may cause unexpected or broken application behavior. Use at your own risk.
2024-08-14T23:37:36.670Z [info][Notifications]: Registered notification agents
2024-08-14T23:37:36.691Z [info][Jobs]: Scheduled jobs loaded
2024-08-14T23:37:36.795Z [info][Server]: Server ready on port 80
2024-08-14T23:37:44.984Z [warn][GitHub API]: Failed to retrieve GitHub releases. This may be an issue on GitHub's end. Overseerr can't check if it's on the latest version. {"errorMessage":"Request failed with status code 503"}
2024-08-14T23:37:44.992Z [warn][GitHub API]: Failed to retrieve GitHub releases. This may be an issue on GitHub's end. Overseerr can't check if it's on the latest version. {"errorMessage":"Request failed with status code 503"}
2024-08-14T23:37:54.904Z [warn][GitHub API]: Failed to retrieve GitHub releases. This may be an issue on GitHub's end. Overseerr can't check if it's on the latest version. {"errorMessage":"Request failed with status code 503"}
2024-08-14T23:37:54.908Z [warn][GitHub API]: Failed to retrieve GitHub releases. This may be an issue on GitHub's end. Overseerr can't check if it's on the latest version. {"errorMessage":"Request failed with status code 503"}
Platform
desktop
Device
N/A
Operating System
Kubernetes (Talos)
Browser
N/A
Additional Context
Kubernetes (Talos)
Code of Conduct
- [X] I agree to follow Overseerr's Code of Conduct
@adampetrovic I've simulated this by blocking traffic to api.github.com on my router:
2025-03-04T11:16:18.599Z [warn][GitHub API]: Failed to retrieve GitHub releases. This may be an issue on GitHub's end. Overseerr can't check if it's on the latest version. {"errorMessage":"connect EHOSTUNREACH 140.82.121.6:443"}
However, api/v1/status returned 200, indicating healthy status.
Looking at the code, it seems there shouldn't be any issue:
router.get<unknown, StatusResponse>('/status', async (req, res) => {
const githubApi = new GithubAPI();
const currentVersion = getAppVersion();
const commitTag = getCommitTag();
let updateAvailable = false;
let commitsBehind = 0;
if (currentVersion.startsWith('develop-') && commitTag !== 'local') {
const commits = await githubApi.getOverseerrCommits();
if (commits.length) {
... updates local variables ...
}
} else if (commitTag !== 'local') {
const releases = await githubApi.getOverseerrReleases();
if (releases.length) {
... updates local variables ...
}
}
return res.status(200).json({
version: getAppVersion(),
commitTag: getCommitTag(),
updateAvailable,
commitsBehind,
restartRequired: restartFlag.isSet(),
});
});
and
public async getOverseerrReleases({
take = 20,
}: {
take?: number;
} = {}): Promise<GitHubRelease[]> {
try {
const data = await this.get<GitHubRelease[]>(
'/repos/sct/overseerr/releases',
{
params: {
per_page: take,
},
}
);
return data;
} catch (e) {
logger.warn(
"Failed to retrieve GitHub releases. This may be an issue on GitHub's end. Overseerr can't check if it's on the latest version.",
{ label: 'GitHub API', errorMessage: e.message }
);
return [];
}
}
Tested with develop hash db06f8.
Do you still have this issue?
I mentioned this in #4276 as well, Overseerr should really have a separate /health route available that doesn't require external API calls. Even if Github is available, the user will quickly get rate-limited by the Github API if this endpoint is used as a healthcheck.
