Skip to content

DeepchecksModelChecker

The DeepchecksModelChecker class is a subclass of the BaseModelChecker class. This class provides methods to check a model against data, save the Deepchecks model suite report in HTML format, and compare the performance of two models on data.

Configuration

Required Configuration

The Deepchecks data checker requires the following configuration:

  • local_dir: Location of a local directory to output files generated by this component.

Optional Configuration

The Deepchecks data checker uses the following optional configuration:

  • model_target: The column name of the model target, or label.
  • model_index: The column name of the model index, if it exists. The index uniquely identifies a row.
  • model_time_index: The column name of the model time_index, if it exists. The time index records when the observation occured.
  • model_cat_features: A list of columns that correspond to categorical features for the model.

Default Configuration

The Deepchecks data checker uses the following optional configuration:

  • deepchecks_model_report_name: The file name of the generated model report. Defaults to DEEPCHECKS_MODEL_REPORT.HTML.
  • deepchecks_model_drift_report_name: The file name of the generated drift report. Defaults to DEEPCHECKS_MODEL_DRIFT_REPORT.HTML.

Methods

The following are the methods that are available in the DeepchecksModelChecker class:

check_model

This method checks the given model against the provided data dict and returns the model suite report, file path of the saved HTML file, and the status of the checks.

def check_model(data_dict, model, *args, **kwargs)

Parameters

  • data_dict (dictionary): The dictionary containing data for the model.
  • model (object): The object of the model class.

Returns

  • model_report (object): The model suite report.
  • file_path (str): The location of the saved HTML file.
  • checks_status (str): The status (PASS, ERROR or WARN) of model checks.

Example 

from lolpop.component import DeepchecksModelChecker

... 

data_dict = data_splitt.split(data)
model = hyperparameter_tuner.run_experiment(data_dict, model_version) 

config = {
    #insert component config here 
}
checker = DeepchecksModelChecker(conf=config)
model_report, file_path, checks_status = checker.check_model(data_dict, model)

print(checks_status)

calculate_model_drift

This method compares the performance of two models on the given data and returns the train/test drift report object and file path of the saved HTML file.

def calculate_model_drift(data, current_model, deployed_model, *args, **kwargs)

Parameters

  • data (dictionary): The dictionary containing data for the models.
  • current_model (object): The current state model object.
  • deployed_model (object): The already deployed model object.

Returns

  • model_report (object): The train/test drift report object.
  • file_path (str): The location of the saved HTML file.

Example 

from lolpop.component import DeepchecksModelChecker

... #set up data an model 

data_dict = data_splitt.split(data)
model = hyperparameter_tuner.run_experiment(data_dict, model_version) 

config = {
    #insert component config here 
}
checker = DeepchecksModelChecker(conf=config)
model_report, file_path = checker.calculate_model_drift(data, current_model, deployed_model)

print(model_report.get_all_checks())