Kapelner and Krieger's (2021) Outcome-Weighted Sequential Matching-on-the-Fly Design

A DesignSeqOneByOneKK14 extension that replaces KK14's unweighted Mahalanobis matching distance with a response-weighted squared distance: at each assignment, a per-covariate weight vector is re-estimated from the responses observed so far (weighting each covariate by the estimated strength of its association with the response, e.g. an absolute standardized regression coefficient), and the new subject is matched to its nearest reservoir subject under that weighted distance rather than the raw (unweighted) Mahalanobis distance KK14 uses. Weighting the match by outcome association means matching effort is spent preferentially on prognostic covariates (those that actually explain outcome variance) rather than equally on every covariate, which is intended to improve estimator efficiency beyond what outcome-agnostic matching achieves. morrison = TRUE additionally switches to Morrison and Owen's (2025) alternative calibration of the matching-acceptance threshold (differing in the fixed- vs. variable-\(n\) settings) and removes KK14's burn-in wait before matching begins.

Weight estimation. compute_weights() dispatches on response_type to a corresponding kk21_*_weights_cpp() backend that fits a per-covariate simple regression of the response on that covariate (given all responses observed so far) and returns the absolute t-statistic (coefficient over its standard error) as that covariate's weight: OLS for "continuous", logistic for "incidence", negative-binomial (or, if count_use_speedup = TRUE, OLS on log(y + 1)) for "count", beta regression (or OLS on the logit scale if proportion_use_speedup = TRUE) for "proportion", Weibull/lognormal/ log-logistic AFT (or OLS on log(y) if survival_use_speedup_for_no_censoring = TRUE and there is no censoring yet) for "survival", and proportional-odds (or OLS on the numeric-coerced level if ordinal_use_speedup = TRUE) for "ordinal". Weights are normalized to sum to 1 and cached per-iteration in private$iteration_weights (retrievable via get_iteration_weights()); the *_use_speedup flags trade weight accuracy for speed by substituting a fast continuous-regression proxy for the response-type-appropriate GLM/AFT/ordinal fit on every single assignment call.

How much weight accuracy do the speedups cost? A weight-level simulation (October 2026; eight standard-normal covariates with decaying true coefficients, \(n\) from 15 to 250, count, proportion and ordinal responses) found the loss to be negligible. Across all three response types the fast OLS-on-transform weights were as close to the GLM's own large-sample target (its normalized weights at \(n = 20000\)) as the GLM's finite-sample weights themselves were: the total-variation distance between the normalized fast and GLM weight vectors was 0.02 to 0.16, versus 0.1 to 0.43 for each method's own sampling noise, and the two large-sample weight vectors differed by under 2% of total weight mass. The two weight vectors selected the same nearest reservoir match 75 to 98% of the time, and the disagreements were near-ties between two reservoir candidates. Running the full design end to end (\(n = 200\), 150 replications per setting, with DesignSeqOneByOneKK14 as the unweighted reference) confirmed this downstream: the share of the true linear predictor's variance remaining inside matched pairs was indistinguishable between the fast and GLM weights for count (0.192 vs 0.191), sparse count with about 70% zeros (0.187 vs 0.188), proportion (0.147 vs 0.148) and four-level ordinal (0.181 vs 0.183) responses, with Monte Carlo standard errors near 0.004 and the same number of matched pairs (about 90 of 200) under both. That within-pair imbalance is the design-side term in the matched-pairs estimator's variance, so it is the channel through which the weights affect power; power itself was not simulated, because a gap this small (under 2% of a term that is itself only part of the total variance) is far below what a direct power simulation of feasible size could resolve. The approximation error is therefore swamped by the estimation noise both methods share, which is why the *_use_speedup flags default to TRUE.

Weighted matching test. The weighted squared distance from the new subject to every reservoir subject is computed (compute_weighted_sqd_distances_cpp()), and the match is accepted only if the minimum weighted distance falls below the private$compute_lambda() quantile of a bootstrapped reference distribution of weighted pairwise distances among subjects enrolled so far (compute_bootstrapped_weighted_sqd_distances_cpp(), num_boot resamples) — a nonparametric, simulation-based acceptance threshold, in contrast to KK14's closed-form F-distribution threshold. As in KK14, an accepted match receives the opposite treatment of its match; a rejected (or empty-reservoir) draw is randomized and added to the reservoir.

Fallback to KK14. Before enough responses have accumulated to fit the weight-estimation regressions reliably (fewer than 2 * (ncol(X) + 2) non-missing responses — two observations per regression parameter, at minimum ncol(X) + 2), assign_wt() falls back to the inherited (unweighted) KK14 assignment rule entirely, via super$assign_wt().

References

Kapelner, A., and Krieger, A. M. (2014). "Matching on-the-fly: Sequential allocation with higher power and efficiency." Biometrics, 70(2), 378-388, doi:10.1111/biom.12148 , for the base matching-on-the-fly algorithm this class extends with outcome-weighted distances (Kapelner and Krieger, 2021); see also Morrison, T., and Owen, A. B. (2025) for the alternative morrison = TRUE threshold calibration referenced by the morrison argument.

Super classes

Design -> DesignSeqOneByOne -> DesignSeqOneByOneKK14 -> DesignSeqOneByOneKK21

Methods

Public methods

• DesignSeqOneByOneKK21$new()

• DesignSeqOneByOneKK21$get_iteration_weights()

• DesignSeqOneByOneKK21$get_covariate_weights()

• DesignSeqOneByOneKK21$assign_wt()

• DesignSeqOneByOneKK21$clone()

+ inherited public methods from DesignSeqOneByOneKK14

• DesignSeqOneByOneKK14$add_all_subject_matched_pair_ids()
• DesignSeqOneByOneKK14$assert_blocking_design()
• DesignSeqOneByOneKK14$assert_equal_block_sizes()
• DesignSeqOneByOneKK14$assert_matching_design()
• DesignSeqOneByOneKK14$get_block_ids()
• DesignSeqOneByOneKK14$get_cmh_se_w_mat()
• DesignSeqOneByOneKK14$get_matching_cluster_ids()
• DesignSeqOneByOneKK14$inject_cmh_se_w_mat()
• DesignSeqOneByOneKK14$is_a_kk_matching_capable()
• DesignSeqOneByOneKK14$is_blocking_design()
• DesignSeqOneByOneKK14$is_complete_blocking_design()
• DesignSeqOneByOneKK14$is_matching_design()
• DesignSeqOneByOneKK14$set_m()
• DesignSeqOneByOneKK14$summarize_blocks()

+ inherited public methods from DesignSeqOneByOne

• DesignSeqOneByOne$add_one_subject()
• DesignSeqOneByOne$add_one_subject_to_experiment_and_assign()
• DesignSeqOneByOne$print_current_subject_assignment()

+ inherited public methods from Design

• Design$add_all_subject_responses()
• Design$add_one_subject_response()
• Design$any_censoring()
• Design$applicable_inference_class_names()
• Design$assert_all_responses_recorded()
• Design$assert_all_subjects_arrived()
• Design$assert_even_allocation()
• Design$assert_fixed_sample()
• Design$capabilities()
• Design$check_experiment_completed()
• Design$draw_ws_according_to_design()
• Design$duplicate()
• Design$get_X()
• Design$get_X_imp()
• Design$get_X_raw()
• Design$get_design_formula()
• Design$get_edi_version_created()
• Design$get_effective_dead()
• Design$get_effective_time()
• Design$get_missingness_method()
• Design$get_n()
• Design$get_ordinal_levels()
• Design$get_original_ordinal_levels()
• Design$get_prob_T()
• Design$get_response_type()
• Design$get_response_type_original()
• Design$get_t()
• Design$get_w()
• Design$get_y()
• Design$get_y_L()
• Design$get_y_R()
• Design$get_y_original()
• Design$has_general_censoring()
• Design$incompatible_inference_classes_due_to_design_structure()
• Design$is_a_bernoulli_capable()
• Design$is_a_cluster_capable()
• Design$is_fixed_sample_size()
• Design$overwrite_all_subject_assignments()
• Design$prepare_for_resampling_replay()
• Design$randomization_family()
• Design$supports()
• Design$supports_randomization_draw()
• Design$supports_resampling()
• Design$supports_resampling_replay()
• Design$transform_y()
• Design$unavailable_inference_classes_due_to_missing_packages()
• Design$warm_all_subject_data_cache()

---

DesignSeqOneByOneKK21$new()

Initialize a matching-on-the-fly sequential experimental design which matches based on Kapelner and Krieger (2021) with option to use matching parameters of Morrison and Owen (2025)

Usage

DesignSeqOneByOneKK21$new(
  response_type,
  prob_T = 0.5,
  include_is_missing_as_a_new_feature = TRUE,
  n = NULL,
  verbose = FALSE,
  lambda = NULL,
  t_0_pct = NULL,
  morrison = FALSE,
  p = NULL,
  num_boot = NULL,
  count_use_speedup = TRUE,
  proportion_use_speedup = TRUE,
  survival_use_speedup_for_no_censoring = TRUE,
  ordinal_use_speedup = TRUE,
  missingness_method = "impute",
  design_formula = ~.,
  seed = NULL,
  ...
)

Arguments

response_type

The data type of response values which must be one of the following: "continuous", "incidence", "proportion", "count", "survival". This package will enforce that all added responses via the add_one_subject_response method will be of the appropriate type.

prob_T

The probability of the treatment assignment. This defaults to 0.5.

include_is_missing_as_a_new_feature

If missing data is present in a variable, should we include another dummy variable for its missingness in addition to imputing its value? If the feature is type factor, instead of creating a new column, we allow missingness to be its own level. The default is TRUE.

n

The sample size (if fixed). Default is NULL for not fixed.

verbose

A flag indicating whether messages should be displayed to the user. Default is FALSE.

lambda

The quantile cutoff of the subject distance distribution for determining matches. If unspecified and morrison = FALSE, default is 10%.

t_0_pct

The percentage of total sample size n where matching begins. If unspecified and morrison = FALSE, default is 35%.

morrison

Default is FALSE which implies matching via the KK14 algorithm using lambda and t_0_pct matching. If TRUE, we use Morrison and Owen (2025)'s formula for lambda which differs in the fixed n versus variable n settings and matching begins immediately with no wait for a certain reservoir size like in KK14.

p

The number of covariate features. Must be specified when morrison = TRUE otherwise do not specify this argument.

num_boot

the number of bootstrap samples taken to approximate the subject-distance distribution. Default is 500.

count_use_speedup

Should we speed up the estimation of the weights in the response = count case via a continuous regression on log(y + 1). instead of a negative binomial regression each time? This is at the expense of the weights being less accurate. Default is TRUE. See the How much weight accuracy do the speedups cost? paragraph of the Details section for simulation evidence that the accuracy lost is negligible.

proportion_use_speedup

Should we speed up the estimation of the weights in the response = proportion case via a continuous regression on log(y / (1 - y)) instead of a beta regression each time? This is at the expense of the weights being less accurate. Default is TRUE. See the How much weight accuracy do the speedups cost? paragraph of the Details section for simulation evidence that the accuracy lost is negligible.

survival_use_speedup_for_no_censoring

Should we speed up the estimation of the weights in the response = survival case via a continuous regression on log(y) instead of a Weibull AFT regression each time, but only when there is no censoring in the data collected so far? This is at the expense of the weights being less accurate when censoring is present. Default is TRUE.

ordinal_use_speedup

Should we speed up the estimation of the weights in the response = ordinal case via a continuous regression on the ordinal levels coerced to numeric. instead of a proportional odds model each time? This is at the expense of the weights being less accurate. Default is TRUE. See the How much weight accuracy do the speedups cost? paragraph of the Details section for simulation evidence that the accuracy lost is negligible.

missingness_method

How to handle missing values in covariates.

design_formula

A formula object.

seed

Integer seed for reproducibility.

...

Extra arguments passed to the DesignSeqOneByOneKK14 superclass.

Returns

A new `DesignSeqOneByOneKK21` object

Examples

seq_des = DesignSeqOneByOneKK21$new(n = 6, response_type = "continuous")
seq_des$add_one_subject_to_experiment_and_assign(data.frame(x = rnorm(1)))

---

DesignSeqOneByOneKK21$get_iteration_weights()

Retrieve the full history of normalized covariate weight vectors computed by compute_weights() across every assignment call so far (see class documentation), keyed by subject index t, for inspecting how the outcome-informed weighting evolved as data accrued.

Usage

DesignSeqOneByOneKK21$get_iteration_weights()

Returns

A list of numeric weight vectors (one per assignment call at which weights were computed), each summing to 1 and named by covariate.

---

DesignSeqOneByOneKK21$get_covariate_weights()

Retrieve the normalized covariate weight vector from the most recent assignment call (see class documentation for how weights are estimated).

Usage

DesignSeqOneByOneKK21$get_covariate_weights()

Returns

A numeric vector of weights, one per covariate, summing to 1 and named by covariate; NULL if weights have not yet been computed (e.g. still in the KK14 fallback regime).

---

DesignSeqOneByOneKK21$assign_wt()

Draw the next subject's treatment assignment via the KK21 outcome-weighted matching-on-the-fly rule (see class documentation): falls back to unweighted KK14 matching if too few responses have accumulated to estimate covariate weights, otherwise re-estimates weights, matches to the nearest reservoir subject under the weighted distance if it clears the bootstrapped acceptance threshold (assigning the opposite treatment), or randomizes into the reservoir otherwise.

Usage

DesignSeqOneByOneKK21$assign_wt()

Returns

The treatment assignment (0 or 1) for the next subject.

---

DesignSeqOneByOneKK21$clone()

The objects of this class are cloneable with this method.

Usage

DesignSeqOneByOneKK21$clone(deep = FALSE)

Arguments

deep

Whether to make a deep clone.

Examples

seq_des = DesignSeqOneByOneKK21$new(n = 6, response_type = 'continuous')
seq_des$add_one_subject_to_experiment_and_assign(data.frame(x1 = rnorm(1)))
#> [1] 0

## ------------------------------------------------
## Method `DesignSeqOneByOneKK21$new()`
## ------------------------------------------------

# \donttest{
seq_des = DesignSeqOneByOneKK21$new(n = 6, response_type = "continuous")
seq_des$add_one_subject_to_experiment_and_assign(data.frame(x = rnorm(1)))
#> [1] 1
# }