IssueTracker

This will create a feature request which anybody can view and comment on.

Please describe your requested enhancement. Good feature requests will solve common problems or enable new use cases.

What you would like to accomplish:
Client mentions that currently GCP monitoring metrics only support data drift/training-serving skew during prediction.
They would like to have the following monitoring metrics via the GCP UI:

Feature drift:
-Variable stability index (VSI)
-Descriptive statistics of each feature


Model score drift:
-Population stability index (PSI)
-Descriptive statistics of model score


Model performance for binary classification:
-AUC-ROC
-AUC-PR
-K-S Statistic
-LogLoss
-Recall
-Precision
-F1-score
-Accuracy
-False Positive Rate
-False Negative Rate
-Confusion matrix
-ROC curve
-Gains chart
-Lift chart
-Decile analysis table

Model performance for regression:
-MAE
-MSE
-RMSE
-MAPE
-NMAE (normalized MAE)
-MSLE (mean square logarithm error)
-RMSLE (root mean square logarithm error).


How this might work:
In the UI customer would like to have a monitoring service which take input a dataset with features, model scores and SME’s labels, and output the aforementioned metrics.

If applicable, reasons why alternative solutions are not sufficient: