App doesn't report an error when it exists, but is not installed in any organisations

[testworksau]

I had created a GitHub Application in our private GitHub enterprise organisation, using the steps described at the bottom of this issue, and here.

I was quite perplexed as to why it wasn't applying any configurations. We were seeing repeated messages like the below on each configured loop interval, so it looked to us that it must have authenticated OK.

{"severity":"INFO","area":"bot","count":0,"time":"2022-09-21T13:56:53+10:00","message":"Enforcing policies on installations."}
{"severity":"INFO","area":"bot","size":0,"items":0,"time":"2022-09-21T13:56:53+10:00","message":"Total cache size."}
{"severity":"INFO","area":"bot","count":0,"results":{},"time":"2022-09-21T13:56:53+10:00","message":"EnforceAll complete."}

As a sanity test, we verified that we received an error when using an invalid PRIVATE_KEY:

{"severity":"ERROR","error":"could not parse private key: invalid key: Key must be a PEM encoded PKCS1 or PKCS8 key","time":"2022-09-21T14:04:42+10:00","message":"Unexpected error enforcing policies."}

and we also received an error when running with an invalid APP_ID:

{"severity":"ERROR","error":"GET https://api.github.com/app/installations?per_page=100: 401 Integration must generate a public key []","time":"2022-09-21T14:00:36+10:00","message":"Unexpected error enforcing policies."}

I realised that I needed to install the app into an organisation, in order for it to work. This step is missing in the docs linked at the top of this issue.

It would be useful to update the docs to add a step to install the app into an org (via the applications Settings page -> Install App tab), and also to log a message, e.g. "App authentication succeeded but no Installations were found", to direct people like me towards installing the app into the relevant orgs.

For completeness / because I was writing this up anyway, I thought I'd share the steps used to build and run this ourselves:

Installation steps

1. Create and configure a new GitHub app in the org, collecting the AppID and Private Key.
2. Install the app into the required organisation(s)
3. Add the AppID to an APP_ID environment variable on your system
4. Add the Private Key to a PRIVATE_KEY environment variable on your system (something like the below will work):

export PRIVATE_KEY=`cat /tmp/allstar-private-key.pem`

5. Set a KEY_SECRET environment variable to direct
6. Clone the Allstar repository git clone [email protected]:ossf/allstar.git
7. Run go build cmd/allstar/main.go
8. Run go run cmd/allstar/main.go

[jeffmendoza]

Looks like you figured it out. Yes, any updates to operator.md to make it more clear appreciated.