Convert protocol buffers into JSON objects

[wunderlejohn]

Is there any movement here on a basic example of convert protocol buffers into JSON objects? feel like that's a stumbling block many would run into using this client library. I'm currently struggling to get buffer data of type 'type.googleapis.com/google.cloud.audit.AuditLog' into anything that might be usable. Have tried installing 'proto3-json-serializer', 'protobufjs' and 'google-proto-files' and following this documentation without any luck.

Originally posted by @wunderlejohn in https://github.com/googleapis/nodejs-logging/issues/785#issuecomment-1559538367

[mlitvinav]

Hi @wunderlejohn, i am able to parse the log entries into json as follows. The following code is Bun with TypeScript and i am only looking for audit logs in my case. I did not manage to make it work with the included protos and had to use the google-proto-files package.

  "devDependencies": {
    "@types/bun": "latest"
  },
  "peerDependencies": {
    "typescript": "^5.0.0"
  },
  "dependencies": {
    "@google-cloud/logging": "^11.0.0",
    "google-proto-files": "^4.2.0"
  }

You will need to login into the right google project via gcloud auth application-default login in order to run this.

import { Logging } from '@google-cloud/logging'
import protofiles from 'google-proto-files'

const logging = new Logging()

async function listLogEntries() {
   const filter = `protoPayload.@type="type.googleapis.com/google.cloud.audit.AuditLog"`;
   const [entries] = await logging.getEntries({
      autoPaginate: false,
      filter,
      pageSize: 10,
      pageToken: undefined,
      orderBy: 'timestamp desc',
   });

   for (const entry of entries) {
      const payload = (entry as any).metadata.payload as string // typemismatch; value is present but invisible
      if (payload == 'protoPayload' && Buffer.isBuffer(entry.metadata[payload]?.value)) {
         const protopath = protofiles.getProtoPath('../google/cloud/audit/audit_log.proto')
         const root = protofiles.loadSync(protopath)
         const type = root.lookupType('google.cloud.audit.AuditLog')
         const value = type.decode(entry.metadata[payload]?.value as Uint8Array).toJSON()
         console.log(value) // the same structured log you see in Google Logging Explorer
      }
   }
}
await listLogEntries().catch(console.error)

[kylefoley]

@mlitvinav this is the only way I was able to get this to work, thank you for posting it. I'm trying to go deeper into the logs and I'm fighting it every step of the way. Having a generic JSON output would be amazing.