ApplicationInsights-node.js Correlation between Frontend and Backend.

I have a React app running with @microsoft/applicationinsights-web that makes some HTTP calls to my NodeJS backend (B4F). The BE uses this package. Both are configured pointing to the same Application Insights instance on Azure.

I would like to correlate the FE calling the backend. I have added some headers to the request containing the traceId, sessionId and traceParent (from the FE). If this can be picked up automatically on the BE what do I call these headers in that case. Alternatively what code do I need to write to pass the data onto the BE AppInsights?

The BE also makes other HTTP calls to other systems I would like to correlate as well. I am using trackDependency for that but I am not seeing the 'call sequence' in the AppInsights detail view on the Azure Portal...? I suspect it is missing a session or parent id in order to tie them together. Any hint for that would be great.

May 23 '24 13:05 obiwanjacobi

Hey @JacksonWeber, how is the investigation going?

May 30 '24 05:05 obiwanjacobi

Hi @obiwanjacobi, apologies for the delay, finally got some time to break this down.

Looks like there's two questions here:

There shouldn't be any additional code change necessary (no need to manually handle context propagation!) assuming you've instrumented both the frontend and backend applications with application insights SDKs and are trying to track an HTTP/HTTPS call between the two. If you're having trouble setting this up I can provide a sample app that demonstrates the setup.
If you're looking to track HTTP/HTTPS calls to other services, you shouldn't need to utilize the trackDependency method as these calls should be auto-collected and auto-correlated. Since this isn't working for you, could you provide an example of a call to one of these calls to other systems and a copy of your package.json would be helpful. Thanks!

May 30 '24 19:05 JacksonWeber

We use Koa on the BE perhaps that is the issue? An example is very much appreciated.
We use fetch. This is an example of our post function. Similar for get, put etc.

private async post(resource: string, requestBody: any, headers: Record<string, string>): Promise<Response> {
        const allHeaders = headers
            ? { ...headers, Authorization: this.authorization }
            : { "Content-Type": "application/json", Authorization: this.authorization };

        const url = `${this.baseUrl}/rest/api/2/${resource}`;
        const start = Date.now();
        const response = await fetch(url, {
            method: "POST",
            headers: allHeaders,
            body: requestBody,
        });

        logRequest("POST", url, response.status, start);    // <= trackDependency

        return response;
    }

We use a mono-repo so the FE and BE is mixed.

package.json:

{
  "name": "<project name>",
  "version": "1.0.0",
  "private": true,
  "scripts": {
    "== our scripts ==" : "<removed>"
  },
  "dependencies": {
    "@azure/identity": "^3.4.1",
    "@azure/keyvault-secrets": "^4.7.0",
    "@fluentui/react": "^8.112.0",
    "@fluentui/react-calendar-compat": "^0.1.5",
    "@fluentui/react-components": "^9.46.4",
    "@fluentui/react-datepicker-compat": "^0.4.35",
    "@fluentui/react-icons": "^2.0.218",
    "@koa/cors": "^4.0.0",
    "@koa/router": "^12.0.1",
    "@lexical/react": "0.12.4",
    "@microsoft/applicationinsights-web": "^3.2.1",
    "@nx/webpack": "^16.10.0",
    "@rjsf/core": "^5.14.3",
    "@rjsf/fluentui-rc": "^5.14.3",
    "@rjsf/utils": "^5.14.3",
    "@rjsf/validator-ajv8": "^5.14.3",
    "@swc/helpers": "~0.5.2",
    "@tanstack/react-query": "^5.0.5",
    "@tanstack/react-query-devtools": "^5.4.3",
    "@tanstack/react-table": "^8.11.7",
    "@types/he": "^1.2.3",
    "ajv-errors": "^3.0.0",
    "applicationinsights": "^3.1.0",
    "async-wait-until": "^2.0.12",
    "axios": "^1.0.0",
    "azure-devops-extension-api": "4.229.0",
    "azure-devops-extension-sdk": "4.0.2",
    "azure-devops-ui": "^2.236.0",
    "chalk": "^4.1.2",
    "commander": "^10.0.1",
    "console-table-printer": "^2.11.1",
    "cors": "^2.8.5",
    "export-from-json": "^1.7.4",
    "express": "~4.18.1",
    "figlet": "^1.6.0",
    "he": "^1.2.0",
    "immer": "^10.0.3",
    "koa": "~2.14.1",
    "koa-404-handler": "^0.1.0",
    "koa-better-error-handler": "^11.0.4",
    "koa-body": "^6.0.1",
    "koa-morgan": "^1.0.1",
    "koa-static": "^5.0.0",
    "lexical": "0.12.4",
    "lodash.startcase": "^4.4.0",
    "loglevel": "^1.9.1",
    "markdown-to-jsx": "^7.3.2",
    "morgan": "^1.10.0",
    "node-cache": "^5.1.2",
    "node-fetch": "^2.7.0",
    "parse-json": "^7.0.0",
    "patch-package": "^8.0.0",
    "re-resizable": "^6.9.11",
    "react": "18.2.0",
    "react-dom": "18.2.0",
    "react-drawio": "^0.1.2",
    "react-grid-layout": "^1.4.4",
    "react-markdown": "^8.0.7",
    "react-modal": "^3.16.1",
    "react-router-dom": "^6.18.0",
    "shelljs": "^0.8.5",
    "source-map-support": "^0.5.13",
    "tslib": "^2.3.0",
    "typescript-logging": "^2.1.0",
    "typescript-logging-category-style": "^2.1.0",
    "undici": "^5.27.0",
    "uuid": "^9.0.1",
    "vss-web-extension-sdk": "^5.141.0",
    "yaml": "^2.2.2"
  },
  "devDependencies": {
    "@babel/core": "^7.14.5",
    "@babel/preset-react": "^7.14.5",
    "@nrwl/nx": "^7.8.7",
    "@nx/esbuild": "16.10.0",
    "@nx/eslint-plugin": "16.10.0",
    "@nx/jest": "16.10.0",
    "@nx/js": "16.10.0",
    "@nx/linter": "16.10.0",
    "@nx/node": "16.10.0",
    "@nx/playwright": "^16.10.0",
    "@nx/react": "16.10.0",
    "@nx/storybook": "^16.10.0",
    "@nx/vite": "16.10.0",
    "@nx/web": "16.10.0",
    "@nx/workspace": "16.10.0",
    "@playwright/test": "^1.36.0",
    "@storybook/addon-essentials": "7.4.6",
    "@storybook/addon-interactions": "^7.2.1",
    "@storybook/core-server": "7.4.6",
    "@storybook/jest": "~0.1.0",
    "@storybook/manager-api": "^7.4.6",
    "@storybook/react-vite": "7.4.6",
    "@storybook/test-runner": "^0.13.0",
    "@storybook/testing-library": "~0.2.0",
    "@storybook/theming": "^7.4.6",
    "@swc/cli": "^0.1.63",
    "@swc/core": "~1.3.85",
    "@swc/jest": "0.2.20",
    "@testing-library/jest-dom": "^6.1.4",
    "@testing-library/react": "14.0.0",
    "@types/cors": "^2.8.15",
    "@types/decompress": "^4.2.5",
    "@types/express": "~4.17.13",
    "@types/jest": "^29.4.0",
    "@types/koa": "~2.13.5",
    "@types/koa__router": "^12.0.3",
    "@types/lodash.startcase": "^4.4.9",
    "@types/morgan": "^1.9.7",
    "@types/node": "^18.14.2",
    "@types/node-fetch": "^2.6.6",
    "@types/react": "18.2.24",
    "@types/react-dom": "18.2.9",
    "@types/react-grid-layout": "^1.3.5",
    "@types/react-modal": "^3.16.2",
    "@types/shelljs": "^0.8.11",
    "@types/unzip-stream": "^0.3.2",
    "@types/uuid": "^9.0.7",
    "@typescript-eslint/eslint-plugin": "^5.60.1",
    "@typescript-eslint/parser": "^5.60.1",
    "@vitejs/plugin-basic-ssl": "^1.0.1",
    "@vitejs/plugin-react": "~4.0.0",
    "@vitejs/plugin-react-swc": "~3.3.2",
    "@vitest/coverage-c8": "~0.32.0",
    "@vitest/ui": "~0.32.0",
    "ajv": "^8.12.0",
    "babel-jest": "^29.4.1",
    "decompress": "^4.2.1",
    "esbuild": "~0.19.2",
    "eslint": "~8.46.0",
    "eslint-config-prettier": "8.1.0",
    "eslint-plugin-import": "2.27.5",
    "eslint-plugin-jsx-a11y": "6.7.1",
    "eslint-plugin-playwright": "^0.15.3",
    "eslint-plugin-react": "7.32.2",
    "eslint-plugin-react-hooks": "4.6.0",
    "jest": "^29.4.1",
    "jest-environment-jsdom": "^29.4.1",
    "jest-environment-node": "^29.4.1",
    "jsdom": "~22.1.0",
    "kill-port": "^2.0.1",
    "nodemon": "^3.1.0",
    "nx": "16.10.0",
    "otpauth": "^9.1.5",
    "postcss-url": "^10.1.3",
    "prettier": "^2.6.2",
    "react-docgen": "^7.0.3",
    "rollup-plugin-copy": "^3.5.0",
    "sass": "^1.69.0",
    "scheduler": "^0.23.0",
    "tfx-cli": "^0.14.0",
    "tfx-umbrella": "file:./packages/tfx-umbrella",
    "the-new-css-reset": "^1.11.0",
    "ts-jest": "^29.1.0",
    "ts-json-schema-generator": "^1.5.0",
    "ts-node": "10.9.1",
    "typedoc": "^0.25.10",
    "typedoc-plugin-missing-exports": "^2.2.0",
    "typescript": "~5.1.3",
    "unzip-stream": "^0.3.1",
    "verdaccio": "^5.0.4",
    "vite": "~4.5.3",
    "vite-plugin-dts": "~2.3.0",
    "vitest": "~0.32.0"
  },
  "optionalDependencies": {
    "@nx/nx-darwin-arm64": "16.10.0",
    "@nx/nx-darwin-x64": "16.10.0",
    "@nx/nx-linux-x64-gnu": "16.10.0",
    "@nx/nx-linux-x64-musl": "16.10.0",
    "@nx/nx-win32-x64-msvc": "16.10.0"
  },
  "engines": {
    "npm": ">=9.6.7",
    "node": ">=18.17.0"
  }
}

May 31 '24 05:05 obiwanjacobi

As for the simple example app, I've attached a zipped project. web-node-correlation-test.zip

Your issue appears to be that you're using the fetch, which we don't support auto-instrumentation for yet. Support for fetch will be coming upon adoption of the fetch instrumentation from upstream OpenTelemetry in the Azure Monitor OpenTelemetry project (which 3.X of this project is a wrapper around).

If you need support for this instrumentation today to allow the use of fetch, you can pair Azure Monitor OpenTelemetry with the community instrumentation for the fetch API here.

May 31 '24 21:05 JacksonWeber

Closing with this comment. Please reopen if you have further questions.

Jun 04 '24 00:06 JacksonWeber

I have tried your example but I don't see the frontend and backend calls correlated in AppInsights..? All I see is the backend with it's 'dependency' to bing that is being called. Am I doing something wrong or are my expectations unrealistic?

Jun 06 '24 06:06 obiwanjacobi

@obiwanjacobi Updated the example above to be simpler and more straightforward. Let me take a look regarding the correlation.

Jun 06 '24 23:06 JacksonWeber

@obiwanjacobi Fixed the issue with the original sample app. You should now see correlated telemetry like this example in the Azure Portal:

Jun 07 '24 20:06 JacksonWeber

I'm sorry but it does not work at my end. I am not sure if I am doing something wrong...

I run npm install on both frontend and node-server folders
Copy in my AppInsights connection string (both frontend & node-server)
Open frontend url in Firefox (no errors in console or network (not counting favicon fail)) and click button

I only see the bing dependency being logged...

Jun 10 '24 08:06 obiwanjacobi

@obiwanjacobi Apologies, looks like there were issues with CORS in the example. I've updated the example to set the correct CORS policy for you in the node-server and updated the frontend HTML to enableCorsCorrelation: true.

Steps are:

run npm install in both the frontend and backend packages
Update both the index.html in frontend and index.js in the node-server folder with your connection string
Navigate into the node-server folder and run node index.js
Navigate into the frontend folder and run node index.js
Navigate to http://localhost:8080 (frontend server address) once both are running
Hit the "Start" button element to send an XMLHTTP request to the node server (which makes the call to the bing.com dependency)

Final result should look like:

Jun 10 '24 22:06 JacksonWeber

@obiwanjacobi Let me know if this resolved your issue, thanks!

Jul 11 '24 21:07 JacksonWeber

No it did not - but I gave up.

Aug 19 '24 14:08 obiwanjacobi

hey @JacksonWeber @obiwanjacobi I want to thank you for the chat on this thread. It helped on my quest!

I found this thread trying to understand how to get the stamp ('Operational ID') that AppInsights put on the requests, the example code you shared helped me to understand the flow between front and back. But I still could not get the information I need.

If I may, I want to share my use case.

I have a front end in React and back end APIs in Node. Those APIs are exposed through Azure APIM. So, something like this: Front (React) -> APIM -> Back (Node).

I would like to get a response header with the operational ID o the request I made on the front end. Using this APIM Policy I could come up with a custom response header with a GUID searchable on logs.

The question is: is it possible to pass to the front end a response header with the operation ID, that AppInsights stamped a request?

Let me know if that is out of scope and I can raise another issue.

Cheers. Denis

Sep 04 '24 18:09 docouto

@obiwanjacobi Let me know if this resolved your issue, thanks!

What call should I use to log into app insight (related to the current call/span) on the frontend and the server? It seems console.log() doesn't cut it.

Oct 01 '24 07:10 obiwanjacobi

@docouto I'm a little confused. Are you trying to propagate context (in this case just via a header) from the Node.js backend -> your React frontend?

Oct 01 '24 22:10 JacksonWeber

@obiwanjacobi I'm not sure I understand your question. What kind of information are you trying to log, and where are you trying to view those logs?

Oct 01 '24 22:10 JacksonWeber

@obiwanjacobi I'm not sure I understand your question. What kind of information are you trying to log, and where are you trying to view those logs?

I want to log custom entries into the call hierarchy I see in App-Insights (where I view these logs).

For instance on the frontend (from the sample) - using console as a log api:

function submit() {

    console.log('this is submit'); // <= custom log entry

    var xhr = new XMLHttpRequest();
    xhr.open('get', 'http://localhost:3000');
    xhr.send();
}

I want to do similar things on the backend too.

Because calls to console.log() do not seem to be intercepted (does not appear in app-insights), what function should I use that my log entry will appear (in this case) just before the call to the backend in app-insights?

The @microsoft/applicationinsights-web we were using had the trackTrace() and trackEvent() functions.

Oct 03 '24 10:10 obiwanjacobi

Hey @JacksonWeber,

Can you explain how I can obtain the sdk instance? I have tried using an onInit function to grab the sdk instance, but even returning false (or void) screws up the default configuration and I do not get the app insights output I am looking for. Not using an onInit callback does give me the output I need.

I have also tried to return the sdk-instance from the - minified - head-script function (variable i seems to be what is passed to the onInit callback) but that also didn't work. Good chance I screwed that up because that code is seriously hard to grasp.

I would prefer to have an onInit callback and still use the default initialization...

Oct 10 '24 13:10 obiwanjacobi

@obiwanjacobi I'm not sure I understand your question. What kind of information are you trying to log, and where are you trying to view those logs?

I want to log custom entries into the call hierarchy I see in App-Insights (where I view these logs).

For instance on the frontend (from the sample) - using console as a log api:

function submit() {
console.log('this is submit'); // <= custom log entry

var xhr = new XMLHttpRequest();
xhr.open('get', 'http://localhost:3000');
xhr.send();
} I want to do similar things on the backend too.

Because calls to console.log() do not seem to be intercepted (does not appear in app-insights), what function should I use that my log entry will appear (in this case) just before the call to the backend in app-insights?

The @microsoft/applicationinsights-web we were using had the trackTrace() and trackEvent() functions.

Please read https://github.com/microsoft/ApplicationInsights-node.js?tab=readme-ov-file#configuration - as it mentions you need to explicitly enable collection of console.log calls.

Oct 10 '24 19:10 JacksonWeber

Hey @JacksonWeber,

Can you explain how I can obtain the sdk instance? I have tried using an onInit function to grab the sdk instance, but even returning false (or void) screws up the default configuration and I do not get the app insights output I am looking for. Not using an onInit callback does give me the output I need.

I have also tried to return the sdk-instance from the - minified - head-script function (variable i seems to be what is passed to the onInit callback) but that also didn't work. Good chance I screwed that up because that code is seriously hard to grasp.

I would prefer to have an onInit callback and still use the default initialization...

I'm assuming this question is about the web SDK, in which case it'd be better to raise it on that repository. Happy to discuss further if I'm mistaken.

Oct 10 '24 19:10 JacksonWeber

Please read https://github.com/microsoft/ApplicationInsights-node.js?tab=readme-ov-file#configuration - as it mentions you need to explicitly enable collection of console.log calls.

I am confused @JacksonWeber

I thought the ApplicationInsights-node package is for the (node) backend/server. I am talking about the frontend - where we used to use the ApplicationInsights-web package.

Another question: that minified head-script, is that compatible with the ApplicationInsights-web package, or is that specific? (I would rather use an official npm package than some minified script magic)

I'm assuming this question is about the web SDK, in which case it'd be better to raise it on that repository.

What repo is that? (like I said, I am confused about all these packages and when/where to use them)

Oct 11 '24 07:10 obiwanjacobi

@obiwanjacobi You're correct that this package (ApplicationInsights-node) is not compatible with web JS, which is why I mentioned it would make more sense to raise frontend JS related issues against the web JS Application Insights SDK.

The basic rule for usage between the usage of these two SDKs is just that if you're writing Node.js backend/server code, expect to be able to instrument using this SDK, whereas if you're running JS in the browser, use the JS SDK linked above.

Oct 11 '24 22:10 JacksonWeber

@JacksonWeber jumping on this thread too as I'm having trouble understanding how correlation across the stack is to be expected. Albiet I haven't played with the samples.

Would love some documentation on or refered to by https://learn.microsoft.com/en-us/azure/azure-monitor/app/nodejs

Questions I have related to this thread is:

Do we set headers in out FE requests with traceparent? As per https://www.w3.org/TR/trace-context/#traceparent-header?
How does applicationinsights take the header value and insert it into the object provided by getCorrelationContext?
- Are we supposed to set the correlation context upon endpoint request recieved? We use nestjs, would this mean that we have to add a global middleware to call getCorrelationContext and overwrite the object provided?
- Or are there some magic that happens?

Dec 19 '24 17:12 ThomasShih