Add a custom Metric or Test

To add a custom metric or test, you need to directly implement the calculation method and (optional) visualization.

You can use the metrics and tests that exist in the codebase as an example.

Note: this is advanced functionality that expects you to be able to work with the codebase independently. To implement the visualization, you should be familiar with Plotly.

This example Jupyter notebook shows how to implement a simple metric and test and include them in a custom Report and Test Suite:

https://github.com/evidentlyai/evidently/blob/main/examples/how_to_questions/how_to_make_custom_metric_and_test.ipynbgithub.com

We plan to simplify this functionality in the future.

If you want to suggest adding specific metrics and tests that are currently not covered, you can also open a GitHub issue with the feature request.