Fast Hurdle Negative-Binomial Regression, with Variance (C++ Backend)

Fits a two-part hurdle negative-binomial model for count data with excess zeros: (1) a hurdle part — logistic regression of the binary indicator \(I(Y_i > 0)\) on X_hurdle — models whether the hurdle is crossed at all, and (2) a count part — a zero-truncated negative-binomial regression fit only on the subset of subjects with \(Y_i > 0\), using X — models the count given the hurdle is crossed. Unlike a zero-inflated model (which mixes a point mass at zero with an untruncated count distribution that can itself also produce zeros), the hurdle model's two parts are a clean partition: every zero comes from the hurdle part, and every positive count's distribution is exactly the negative-binomial conditional on being positive (left-truncated at 1).

Usage

fast_hurdle_negbin_with_var_cpp(
  X_r,
  y_r,
  X_hurdle_r,
  j = 2L,
  warm_start_params = NULL,
  smart_cold_start = TRUE,
  maxit = 1000L,
  tol = 1e-08,
  fixed_idx = NULL,
  fixed_values = NULL,
  optimization_alg = "lbfgs",
  warm_start_fisher_info = NULL,
  warm_start_hurdle_fisher_info = NULL
)

Arguments

X_r: Numeric matrix of predictors for the count component (the zero-truncated negative-binomial part), \(n \times p\).
y_r: Numeric vector of length \(n\): observed non-negative integer counts (zeros are handled by the hurdle part; only the positive subset is passed to the truncated count part).
X_hurdle_r: Numeric matrix of predictors for the hurdle (zero-vs- positive) logistic component, \(n \times p_{\mathrm{hurdle}}\); may differ from X_r (a different covariate set for "does an event occur at all" vs. "how many, given at least one").
j: 1-based index (into the count model's \(p\) coefficients) of the coefficient to report ssq_b_j for.
warm_start_params: Optional starting values for the count model's full c(beta, log(theta)) parameter vector. If provided, smart_cold_start is ignored.
smart_cold_start: Logical. If TRUE, use an initial OLS-based guess when starting from scratch (a "cold start") with no prior knowledge. This is ignored if a warm start is provided.
maxit: Maximum number of count-model optimizer iterations.
tol: Convergence tolerance (count model).
fixed_idx: Optional integer indices (into the count model's c(beta, log(theta)) layout) of parameters to hold fixed rather than estimate.
fixed_values: Optional values to fix the parameters named by fixed_idx at.
optimization_alg: Optimization algorithm for the count model; the hurdle logistic part always uses the same algorithm internally.
warm_start_fisher_info: Optional initial Fisher Information matrix for the count model's first optimizer iteration.
warm_start_hurdle_fisher_info: Optional initial Fisher Information matrix for the hurdle logistic model's first optimizer iteration.

Value

A list with components b (count-model coefficients \(\hat\beta\)), theta_hat (the zero-truncated NB dispersion), converged (count model), hurdle_b (hurdle logistic coefficients), hurdle_converged, ssq_b_j/ssq_b_2 (count-model coefficient variances), hurdle_ssq_b_j/ hurdle_ssq_b_2 (hurdle-model coefficient variances), observed_information/fisher_information/information (three aliases for the count model's observed information, over c(beta, log(theta))), information_type = "observed", hessian (the negative of that information), hurdle_fisher_information (the hurdle model's own information matrix), and failure_message (empty on success, otherwise an explanatory string for a degenerate count-part fit).

Details

Hurdle part. Fit via fast_logistic_regression_cpp's internal engine on y_pos_ind = as.numeric(y > 0) regressed on X_hurdle; if y_pos_ind has no variation (all-zero or all-positive y), the hurdle part is skipped (hurdle_b is all NA, hurdle_converged = FALSE) rather than erroring.

Count part. Fit on the positive-count subset (\(n_+ = \sum_i I(y_i > 0)\) rows) via maximum likelihood on the zero-truncated negative-binomial density with mean-parameterized dispersion \(\theta\): parameter vector c(beta, log(theta)) (length p + 1), with \(\theta\) optimized on the log scale for positivity and reported back as theta_hat = exp(params[p+1]). If \(n_+ \le p\) (too few positive observations to identify the count-model coefficients), the count part returns b as all NA and converged = FALSE with an explanatory failure_message, while the hurdle part (which does not depend on \(n_+\)) is still fit and returned normally.

Variance. ssq_b_j/ssq_b_2 are the variances of the j-th and 2nd count-model coefficients (from the count part's observed information, restricted to free/non-fixed_idx parameters); hurdle_ssq_b_j/hurdle_ssq_b_2 are the analogous variances for the hurdle-model coefficients (index j into that model's own coefficient vector, from the hurdle logistic regression's own information matrix — no fixed_idx applies to the hurdle part). Both use a targeted diagonal-entry inversion rather than a full matrix inverse, and are NA if the relevant coefficient was fixed, out of range, or its model failed to converge/produce a finite information matrix.

Fast Hurdle Negative-Binomial Regression, with Variance (C++ Backend)

Usage

Arguments

Value

Details

See also