Evidently latest: How to create a custom metric with value specified at creation time

[jsimao71]

WHAT: I'm trying to create a custom metric following the example in the documentation MaxMetric . https://docs.evidentlyai.com/metrics/customize_metric

We want the current value of the metric to be specified as a constructor field. That is, rather than being a value derived from the dataset, the value is specified when the metric is created.

When we add the metric to the project the server UI show each point as a separate point with a different label name. Locally it works fine, and show the plot as a line. But not in the server.

How to fix the code below, so the server show a single line like the MaxMetric example. Or alternatively, how to use an existing Metric to do the same. Is there any link to example you can share showing how to pass values to the metric to render on server? Is there any repository/folder with community contributed custom Metrics?!

Thanks!

class CustomMetric(SingleValueMetric):
    column: str
    value: float

    def _default_tests(self) -> List[BoundTest]:
        return [eq(0).bind_single(self.get_fingerprint())]

    def _default_tests_with_reference(self) -> List[BoundTest]:
        return [eq(Reference(relative=0.1)).bind_single(self.get_fingerprint())]


class CustomMetricImplementation(SingleValueCalculation[CustomMetric]):
    values = []
    
    def calculate(self, context: Context, current_data: Dataset, reference_data: Optional[Dataset]) -> SingleValue:
        value = float(self.metric.value)
        self.values.append(value)
        result = self.result(value=value)
        figure = line(self.values)
        result.widget = [plotly_figure(title=self.display_name(), figure=figure)] 
        return result

    def display_name(self) -> str:
        return f"{self.metric.column}"

[Liraim]

Hi, @jsimao71!

Thanks for your question! From what I can see, your sample code looks mostly correct, with just a small detail to clarify:

• The custom metric takes column and value as parameters.
• In the implementation, the metric simply returns value — which means it will always produce that fixed result.
• The metric also customizes the widget to show a line graph with a single point. That part might not work as expected.

It seems the issue may be more about how the widget is displayed in the report rather than the metric itself. Could you tell me a bit more about what you’d like the metric to show in the report? That will help in suggesting the right approach.