Tests the effect of H0: one, or two, ... or all covariates are not predictive (out of sample) via a permutation test. We permute the covariates column(s) and build YARF models num_permutation_samples times. The p-val is determined by a permutation-like test.
Source: R/YARF_hypothesis_tests.R
cov_importance_test.RdTests the effect of H0: one, or two, ... or all covariates are not predictive (out of sample) via a permutation test.
We permute the covariates column(s) and build YARF models num_permutation_samples times.
The p-val is determined by a permutation-like test.
Usage
cov_importance_test(
yarf_mod,
covariates = NULL,
num_permutation_samples = 100,
plot = TRUE
)Arguments
- yarf_mod
The fit model to test against permuted training data.
- covariates
The covariates to permute as a vector of indices or names. If multiple covariates they are permuted as a single block to not break collinearity. If this parameter is specified as
NULL, we test all the covariates by permuting y instead. Default isNULL.- num_permutation_samples
How many different YARF models should be built - one for each different permutation of the variables tests. The is the resolution of the test. The default is
100.- plot
Plot a histogram of the permutation samples with the original sample displayed? Default is
TRUE.
Value
A list with the following components: permutation_samples_of_error whose value
is a vector with entries being the error values for each permutation sample,
observed_error_estimate is the error value of the original model built with
unpermuted data and pval is the estimated significance level of the permutation test.