opencensus-go-exporter-stackdriver icon indicating copy to clipboard operation
opencensus-go-exporter-stackdriver copied to clipboard

No Resource information tied with traces?

Open someone1 opened this issue 7 years ago • 9 comments

I am moving to this library from cloud.google.com/go/trace due to the obsolete notice, but noticed a few things that has me scratching my head.

I setup the package as follows (I'm using AppEngine Flexible):

import (
	"net/http"
	"os"

	"contrib.go.opencensus.io/exporter/stackdriver"
	"contrib.go.opencensus.io/exporter/stackdriver/propagation"
	"go.opencensus.io/plugin/ocgrpc"
	"go.opencensus.io/plugin/ochttp"
	"go.opencensus.io/stats/view"
	"go.opencensus.io/trace"
	mrpb "google.golang.org/genproto/googleapis/api/monitoredres"
)

func InitTraceClient() error {
        res := &mrpb.MonitoredResource{}
	res.Type = "gae_app"
	res.Labels = make(map[string]string)
	res.Labels["project_id"] = os.Getenv("GOOGLE_CLOUD_PROJECT")
	res.Labels["module_id"] = os.Getenv("GAE_SERVICE")
	res.Labels["version_id"] = os.Getenv("GAE_VERSION")

	exporter, err := stackdriver.NewExporter(stackdriver.Options{
		ProjectID: os.Getenv("GOOGLE_CLOUD_PROJECT"),
		Resource:  res,
	})
	if err != nil {
		return err
	}

	view.RegisterExporter(exporter)
	trace.RegisterExporter(exporter)

	if err = view.Register(ochttp.DefaultClientViews...); err != nil {
		return err
	}

	if err = view.Register(ocgrpc.DefaultClientViews...); err != nil {
		return err
	}
        return nil
}

For my datastore client I do:

func getDatastoreClient(ctx context.Context) (*datastore.Client, error) {
	var options []option.ClientOption
	options = append(options, option.WithGRPCDialOption(grpc.WithStatsHandler(&ocgrpc.ClientHandler{})))
	return datastore.NewClient(ctx, "", options...)
}

And I add middleware to start tracing on every request:

func TraceHandler(h http.Handler) http.Handler {
	traceHandler := &ochttp.Handler{
		Handler:          h,
		Propagation:      &stackdriver.HTTPFormat{},
		IsPublicEndpoint: false, // I've tried true here as well...
		StartOptions: trace.StartOptions{
			Sampler: trace.AlwaysSample(),
		},
	}
	fn := func(w http.ResponseWriter, r *http.Request) {
		traceHandler.Handler.ServeHTTP(w, r)
	}
	return http.HandlerFunc(fn)
}

However, I get no HTTP traces in StackDriver, just datastore ones and other gRPC traces I didn't even intend to monitor (Logging). There are no labels associated with the spans to help identify the service/version/http request the spans originated from.

image

Am I missing something or is this package still not ready to trace requests like cloud.google.com/go/trace is able to?

Thanks for this feature in general, it's a really helpful tool and I really like the idea of OpenCensus!

someone1 avatar Apr 24 '18 14:04 someone1

And of course I figure out it was an issue on my end as soon as I post the issue... Middleware was wrong by using the passed in Handler vs the ochttp one...

Should be:

        fn := func(w http.ResponseWriter, r *http.Request) {
		traceHandler.ServeHTTP(w, r)
	}

However, I'm still not sure how to link things like Service/Version to the traces, are there special labels we can use? Is there a master list of all special labels for the Stackdriver platform and/or to AppEngine specifically somewhere?

someone1 avatar Apr 24 '18 14:04 someone1

I guess PR #2 would solve this issue for you, wouldn't it?

gottwald avatar May 16 '18 07:05 gottwald

Looks like a useful feature but it's not relevant to the issue I've described here. I am able to figure out some labels that the StackDriver platform uses but it still doesn't seem to work correctly (I pulled these from older trace libraries I found for NodeJS/Python/etc.):

attrs = append(attrs, trace.StringAttribute("g.co/gae/app/version", os.Getenv("GAE_VERSION")))
attrs = append(attrs, trace.StringAttribute("gae/instance_id", os.Getenv("GAE_INSTANCE")))
attrs = append(attrs, trace.StringAttribute("g.co/gae/app/module", os.Getenv("GAE_SERVICE")))
attrs = append(attrs, trace.StringAttribute("g.co/gae/app/module_version", os.Getenv("GAE_VERSION")))

These seem to label themselves correctly in the dashboard but when I filter on say Version from the dropdown, no traces show up, even though the Version seems to be tagged appropriately?

I can Filter on Service: image

But not on Version: image

someone1 avatar May 16 '18 14:05 someone1

gotcha. Sadly I don't know the fields either. Maybe this is something that's faster solved by asking Google support.

gottwald avatar May 17 '18 07:05 gottwald

/cc @ramonza

rakyll avatar May 29 '18 21:05 rakyll

Wanted to have that for us as well so I opened a Google Support ticket yesterday and asked (Case 15951800). Once I have a positive response I'll share it here.

gottwald avatar May 30 '18 09:05 gottwald

@someone1 They converted my support ticket to a "official" public feature request. You can star this issue to get updates on the implementation progress: https://issuetracker.google.com/issues/109733852 As always there's no ETA given for this FR.

gottwald avatar Jun 06 '18 06:06 gottwald

I think this issue has been solved since, because: We now have resource detection as per https://github.com/census-ecosystem/opencensus-go-exporter-stackdriver/pull/84

Kindly paging @songy23 @rghetia @bogdandrutu

odeke-em avatar Jun 23 '19 20:06 odeke-em

This issue still exists, is there a reason https://github.com/census-ecosystem/opencensus-go-exporter-stackdriver/blob/master/trace_proto.go#L189 is prefixing all attributes with g.co/r/ ?

looking at the traces that do work in GAE standard it seems they are using g.co/gae/app/module go.co/gae/app/version for these

nickm4062 avatar Aug 22 '19 00:08 nickm4062