Not able to retreive log entries

[deama]

So I have the following function here:

const functions = require('@google-cloud/functions-framework');
const { Storage } = require('@google-cloud/storage');
const readline = require('readline');
const { Logging } = require('@google-cloud/logging');
const storage = new Storage();
const logging = new Logging();

async function searchLogs(date, timeStart, timeEnd, user) {
  const logFilter = `logName="timestamp>="2023-03-01T11:55:00Z" AND timestamp<="2023-03-03T11:55:51Z"`;
  const resourceNames = ['projects/production-logs'];
  const [entries] = await logging.getEntries({ resourceNames, filter: logFilter });

  console.log(`Searching logs with filter: ${logFilter}`);
  console.log(`Found ${entries.length} entries.`);

  for (const entry of entries) {
    const message = entry.data.jsonPayload.message;
    const ipAddress = message.match(/Ip Address:\s+(\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})/);
    if (ipAddress) {
      return ipAddress[1];
    }
  }

  console.log(`----`);
  return null;
}

//... rest of code

What I'm having an issue is I have my production logs stored in a logging bucket called "prod", however the above code only works on the project scope rather than on a per-bucket scope, so it basically can't find the logs, because it can't see the logging bucket.

Is there a way to direct it so that it sends the request to a specific logging bucket? I can't find much online, and I have found a function in the source code for the loggin API about retreiving a list of logging buckets in the project, which works, but how do I set it to use that bucket now?

Btw, I hardcoded the timestamp for now to keep it simple.

[deama]

I've tried setting the resourceNames to view:

const resourceNames = ['projects/production-logs/locations/global/buckets/prod/views/_AllLogs'];

And whilst that is what is said in the documentation: https://github.com/googleapis/nodejs-logging/blob/main/src/v2/logging_service_v2_client.ts [line 772]

• • projects/[PROJECT_ID]/locations/[LOCATION_ID]/buckets/[BUCKET_ID]/views/[VIEW_ID]

It compiles fine, and it gives me the entries, however if I now try and make a filter specific to the bucket in question, e.g a filter that only that bucket should contain, it gives me 0 entries, even though manually checking it displays actual entries. So looks like it still applies the getEntries on the project only rather than the logging bucket.

[cindy-peng]

Hi @deama, thank you for reporting the issue! Are you still currently running into this issue?

[deama]

Hi @deama, thank you for reporting the issue! Are you still currently running into this issue?

I'm not sure. I wasn't able to fix it so instead I switched to the python implementation instead, that worked fine.

I don't have that code or environment setup to retest that node.js implemenation anymore.

[cindy-peng]

Hi @deama, to filter logs for a specific bucket, you can specify resource.type and bucket_name restrictions in the log filter. For example, in your code you can specify:

const logFilter = 'resource.type="gcs_bucket" AND resource.labels.bucket_name="your-bucket-name"';

The filter string syntax for nodejs-logging repo is based on the Cloud Logging query language. Node.js applications using the nodejs-logging library can leverage the full power of the Cloud Logging query language to retrieve and filter log data.