Skip to contents

Select an optimal m-out-of-n bootstrap resample size via the minimum volatility method

Source: R/select_optimal_m_prop.R
select_optimal_m_prop.Rd

PTE_bootstrap_inference's m_pow_of_n argument sets each bootstrap replicate's resample size to \(m = \lfloor n^{\gamma} \rfloor\), which is required to be less than \(n\) (i.e. \(\gamma < 1\)) for the bootstrap to be consistent for our non-smooth policy-value estimands (see PTE_bootstrap_inference). There is, however, no single population-optimal \(\gamma\) – asymptotic theory only requires \(m \to \infty\) and \(m / n \to 0\). Politis, Romano & Wolf (Subsampling, 1999, Ch. 9) and Bickel & Sakov (2008) address this by proposing the "minimum volatility" (MV) method: compute the bootstrap answer (here, the width of the rate-corrected percentile confidence interval for one of the three PTE estimands) over a grid of candidate \(\gamma\) (equivalently \(m\)) values, then, for each grid point, measure how much that answer varies over a small centered neighborhood of adjacent grid points ("volatility"). The chosen \(\gamma\) is the one whose neighborhood is most stable, i.e. the region of the grid where the bootstrap's answer has stopped changing in response to further tuning of \(m\) – this is the practical, data-driven proxy for "\(m\) large enough for the CLT-type approximation underlying the bootstrap to have kicked in, but still small enough relative to \(n\) for consistency."

Usage

select_optimal_m_prop(
  X,
  y,
  regression_type = "continuous",
  incidence_metric = "odds_ratio",
  personalized_model_build_function = NULL,
  censored = NULL,
  predict_function = NULL,
  difference_function = NULL,
  cleanup_mod_function = NULL,
  y_higher_is_better = TRUE,
  pct_leave_out = 0.1,
  m_pow_of_n_grid = seq(0.5, 1, by = 0.05),
  estimand = "average",
  B = 250,
  alpha = 0.05,
  volatility_window = 3,
  num_cores = NULL
)

Arguments

X

A \(n \times p\) dataframe of covariates where one column is labeled "treatment" and it is a binary vector of treatment allocations in the study. See PTE_bootstrap_inference.

y

An \(n\)-length numeric vector which is the response. See PTE_bootstrap_inference.

regression_type

See PTE_bootstrap_inference. Default "continuous".

incidence_metric

See PTE_bootstrap_inference. Default "odds_ratio".

personalized_model_build_function

See PTE_bootstrap_inference. Default NULL (use the built-in default for regression_type).

censored

See PTE_bootstrap_inference. Required if regression_type is "survival".

predict_function

See PTE_bootstrap_inference. Default NULL (use the built-in default).

difference_function

See PTE_bootstrap_inference. Default NULL (use the built-in default for regression_type).

cleanup_mod_function

See PTE_bootstrap_inference. Default NULL (no cleanup).

y_higher_is_better

See PTE_bootstrap_inference. Default TRUE.

pct_leave_out

See PTE_bootstrap_inference. Default 0.10.

m_pow_of_n_grid

The grid of candidate \(\gamma\) exponents to evaluate (each entry is a candidate value for PTE_bootstrap_inference's m_pow_of_n argument). Must have at least volatility_window entries so every interior grid point has a complete neighborhood. The default, seq(0.5, 1, by = 0.05), brackets the Politis-Romano-Wolf-recommended \(\gamma = 0.75\) on both sides, up to (but not below) \(\gamma = 0.5\) (\(m = \sqrt{n}\), a common lower-rate benchmark) and \(\gamma = 1\) (the classical, here potentially inconsistent, \(n\)-out-of-\(n\) bootstrap, included only as a reference point – see volatility_window).

estimand

Which of the three PTE estimands' confidence interval width to stabilize: "adversarial", "average" (the default) or "best"; see PTE_bootstrap_inference.

B

The number of bootstrap samples to take at each grid point. Since the total cost is length(m_pow_of_n_grid) * B bootstrap replicates, this defaults to a much smaller 250 than PTE_bootstrap_inference's own default of 3000 – this function is for calibrating m_pow_of_n, not for the final reported inference.

alpha

Confidence interval size (1 - alpha) used to measure CI width at each grid point. Defaults to 0.05.

volatility_window

The number of adjacent grid points (must be odd and \(\geq 3\)) averaged into the volatility measure at each interior grid point. Only grid points with a complete, centered window are eligible to be selected (this excludes the (volatility_window - 1) / 2 grid points at each end of m_pow_of_n_grid from selection, since their neighborhoods would otherwise be one-sided and their apparent volatility artificially low). Default 3.

num_cores

See PTE_bootstrap_inference. Passed through unchanged to each grid point's call (grid points are evaluated sequentially, not in additional parallel, to avoid nested parallelism on top of PTE_bootstrap_inference's own).

Value

A list with class "PTE_optimal_m_selection" containing:

m_pow_of_n_optimal

The selected \(\gamma\), ready to pass as m_pow_of_n to PTE_bootstrap_inference.

m_optimal

The corresponding resample size, round(n^m_pow_of_n_optimal).

grid_table

A data.frame with one row per grid point: m_pow_of_n, m, the point estimate and ci_width for estimand, and its volatility (NA for the non-eligible boundary points described above).

estimand, B, alpha, volatility_window

Echoed back for reference.

Details

This function is a calibration step: it runs PTE_bootstrap_inference once per grid point (so its total cost is roughly length(m_pow_of_n_grid) times that of a single call) and does not itself return final inference – rerun PTE_bootstrap_inference at the selected m_pow_of_n_optimal (with a larger B) to get the confidence intervals and p-values you report.

References

Politis, D.N., Romano, J.P. and Wolf, M. (1999) Subsampling. Springer Series in Statistics.

Bickel, P.J. and Sakov, A. (2008) On the choice of m in the m out of n bootstrap and confidence bounds for extrema. Statistica Sinica, 18(3), 967-985.

Author

Adam Kapelner

Examples

if (FALSE) { # \dontrun{
  library(PTE)
  data(continuous_example)
  X = continuous_example$X
  y = continuous_example$y

  m_selection = select_optimal_m_prop(
    X, y,
    regression_type = "continuous",
    m_pow_of_n_grid = seq(0.5, 1, by = 0.05),
    B = 250,
    num_cores = 1
  )
  m_selection$grid_table
  m_selection$m_pow_of_n_optimal

  #now rerun inference at the selected m_pow_of_n with a production-sized B
  pte_results = PTE_bootstrap_inference(
    X, y,
    regression_type = "continuous",
    m_pow_of_n = m_selection$m_pow_of_n_optimal,
    B = 3000
  )
  pte_results
} # }