Challenger Models

A model risk assessment policy to perform effective challenge using challenger models. Developing a “challenger model” is another approach to measuring potential model risk.

With this technique, a new model is developed, sometimes on the basis of an alternative model theory or different data set, for comparison to the “champion model” (i.e., the primary model being used).

Why is Challenger Models Analysis important?

The Challenger Models Analysis is important because it proves that the champion model is in fact the best model to be used in prodctuion.

How does FAIRLY perform Challenger Models Analysis?

When a user adds the Model Risk - Challenger Models policy to their project, they will be asked to select all or some of the seven challenger models to generate comparative performance metircs (customizable) for effective challenge:

Logistic Regression Model: Generate logistic regression sklearn model to compare against input test model.

KNN Model: Generate KNN sklearn model to compare against input test model.

Gaussian Naive Bayes: Generate Gaussian Naive Bayes sklearn model to compare against input test model.

MLP: Generate MLP sklearn model to compare against input test model.

Multinomial Naive Bayes: Generate multinomial naive bayes sklearn models to compare against input test model.

Random Forest: Generate random forest sklearn model to compare against input test model.

SVM: Generate SVM sklearn models to compare against input test model.

XGBoost: Generate XGBoost sklearn models to compare against input test model.

How to activate the Challenger Model Analysis?

Please contact support@fairly.ai for this advance feature.