Convenience method for testing a model's fit. The strategy is simple: one builds a YARF model from the residuals of the model you wish to fit and performs an omnibus test of all covariates against those residuals. This effecticely answers the question: "is there out-of-sample predictive information left over after this model was fit?" The p-val is determined by a permutation-like test.

Convenience method for testing a model's fit. The strategy is simple: one builds a YARF model from the residuals of the model you wish to fit and performs an omnibus test of all covariates against those residuals. This effecticely answers the question: "is there out-of-sample predictive information left over after this model was fit?" The p-val is determined by a permutation-like test.

Usage

model_fit_test(
  X,
  y,
  model_fit = NULL,
  num_permutation_samples = 100,
  plot = TRUE,
  ...
)

Arguments

X

The training data frame

y

The vector of training responses

model_fit

An already-fit model that you wish to test if there is any predictive power left over. This model's class must respond to the "predict" S3 method. The default is NULL which indicates the standard linear model (missingness not allowed there).

num_permutation_samples

The resolution of the test. See cov_importance_test. The default is 100.

plot

Plot the result as a histogram. Default is TRUE.

...

Additional parameters to be passed to YARF.

Value

The same as cov_importance_test.

Author

Adam Kapelner