components Model Monitoring documentation

Model Monitoring

Components in this category

---

• action_analyzer_correlation_test

  Perform correlation test on different groups to generate actions.

• action_analyzer_identify_problem_traffic

  Separate bad queries into different groups.

• action_analyzer_metrics_calculation

  Calculate futher metrics for generating actions.

• action_analyzer_output_actions

  Merge and output actions.

• data_drift_compute_metrics

  Compute data drift metrics given a baseline and a deployment's model data input.

• data_drift_signal_monitor

  Computes the data drift between a baseline and production data assets.

• data_quality_compute_metrics

  Compute data quality metrics leveraged by the data quality monitor.

• data_quality_data_statistics

  Compute data statistics leveraged by the data quality monitor.

• data_quality_metrics_joiner

  Join baseline and target data quality metrics into a single output.

• data_quality_signal_monitor

  Computes the data quality of a target dataset with reference to a baseline.

• feature_attribution_drift_compute_metrics

  Feature attribution drift using model monitoring.

• feature_attribution_drift_signal_monitor

  Computes the feature attribution between a baseline and production data assets.

• feature_importance_metrics

  Feature importance for model monitoring.

• genai_mdc_preprocessor

  Filters the raw span log based on the window provided, and aggregates it to trace level.

• genai_token_statistics_compute_metrics

  Compute token statistics metrics.

• genai_token_statistics_signal_monitor

  Computes the token and cost metrics over LLM outputs.

• generation_safety_quality_signal_monitor

  Computes the content generation safety metrics over LLM outputs.

• gsq_annotation_compute_histogram

  Compute annotation histogram given a deployment's model data input.

• gsq_annotation_compute_metrics

  Compute annotation metrics given a deployment's model data input.

• gsq_input_schema_adaptor

  Adapt data to fit into GSQ component.

• model_data_collector_preprocessor

  Filters the data based on the window provided.

• model_monitor_action_analyzer

  Generate and output actions to the default datastore.

• model_monitor_action_detector

  Generate and output actions

• model_monitor_azmon_metric_publisher

  Azure Monitor Publisher for the computed model monitor metrics.

• model_monitor_compute_histogram

  Compute a histogram given an input data and associated histogram buckets.

• model_monitor_compute_histogram_buckets

  Compute histogram buckets given up to two datasets.

• model_monitor_create_manifest

  Creates the model monitor metric manifest.

• model_monitor_data_joiner

  Joins two data assets on the given columns for model monitor.

• model_monitor_evaluate_metrics_threshold

  Evaluate signal metrics against the threshold provided in the monitoring signal.

• model_monitor_feature_selector

  Selects features to compute signal metrics on.

• model_monitor_metric_outputter

  Output the computed model monitor metrics.

• model_monitor_output_metrics

  Output the computed model monitor metrics to the default datastore.

• model_performance_compute_metrics

  Compute model performance metrics leveraged by the model performance monitor.

• model_performance_signal_monitor

  Computes the model performance

• prediction_drift_signal_monitor

  Computes the prediction drift between a baseline and a target data assets.

• token_statistics_compute_metrics

  Compute token statistics metrics.