Check Simulation Results Against Theoretical Values

Compares empirical simulation results from rOncoEndpoints with their corresponding theoretical values. This function calculates standard performance metrics (Bias, Relative Bias, SE, MSE, RMSE) for validating simulation implementations and assessing the accuracy of the proposed random number generation method.

Usage

CheckSimResults(
  dataset,
  p = NULL,
  hazard_OS = NULL,
  hazard_PFS = NULL,
  rho_tte_resp = NULL,
  copula = NULL
)

Arguments

dataset

A data frame. The output from rOncoEndpoints containing simulated endpoint data with columns simID, Group, and endpoint variables (OS, PFS, and/or Response).

p

Named numeric vector or NULL. The true probabilities of the binary response endpoint for each group. Names must match group names in dataset. Required if Response endpoint is present. Example: c(Treatment = 0.4, Control = 0.3)

hazard_OS

Named numeric vector or NULL. The hazard rates for OS for each group. Names must match group names in dataset. Required if OS endpoint is present. Example: c(Treatment = 0.05, Control = 0.07)

hazard_PFS

Named numeric vector or NULL. The hazard rates for PFS for each group. Names must match group names in dataset. Required if PFS endpoint is present. Example: c(Treatment = 0.08, Control = 0.10)

rho_tte_resp

Named numeric vector or NULL. The specified correlations between TTE and Response for each group. Names must match group names in dataset. Required if both TTE and Response endpoints are present. Example: c(Treatment = 0.3, Control = 0.2)

copula

Character or NULL. The copula family used for modeling dependence. Options are "Clayton" or "Frank". Required if both TTE and Response endpoints are present.

Value

A tibble with the following columns:

Group

Treatment group name

Endpoint

Name of the endpoint or statistic

Empirical

Mean of estimates across simulations

Theoretical

Expected theoretical value

Bias

Bias = Empirical - Theoretical (signed difference showing direction of systematic error)

Relative_Bias

Relative bias as percentage: 100 × Bias / Theoretical. Positive values indicate overestimation, negative values indicate underestimation

SE

Empirical standard error (SD of estimates across simulations)

MSE

Mean squared error = Bias² + SE²

RMSE

Root mean squared error = √MSE (overall accuracy in original scale)

Assessment

Quick interpretation: "Excellent" (|Relative_Bias| < 5%), "Acceptable" (5% ≤ |Relative_Bias| < 10%), "Review" (|Relative_Bias| ≥ 10%)

Details

This function calculates both empirical and theoretical values for:

Time-to-event endpoints (OS/PFS):

• Mean: Empirical vs 1/hazard

• Median: Empirical vs log(2)/hazard

Binary endpoint (Response):

• Proportion: Empirical vs p

Correlations:

• Corr(OS, Response): Empirical vs rho_tte_resp (specified)

• Corr(PFS, Response): Empirical vs value calculated by CorResponsePFS

• Corr(OS, PFS): Empirical vs hazard_OS / hazard_PFS (Fleischer model)

Performance metrics:

• Bias: Measures systematic error. Should be close to 0 for unbiased methods. Sign indicates direction: positive = overestimation, negative = underestimation

• Relative Bias (%): Bias relative to true value. Recommended interpretation: <5% excellent, <10% acceptable

• SE (Standard Error): Measures variability of estimates. Decreases with √nsim. Smaller is better

• MSE: Combines bias and variance into single metric. Smaller is better

• RMSE: MSE in original scale. Directly comparable to SE

• Assessment: Automatic interpretation based on relative bias

The function is particularly useful for:

• Validating that rOncoEndpoints correctly implements the models

• Demonstrating unbiasedness of the proposed method for publication

• Checking if sufficient simulations have been run (SE should be small)

• Understanding the relationship between parameters and resulting correlations

• Quality control in simulation studies

Note

Interpretation guidelines:

• Bias close to 0: Method is unbiased (desirable for publication)

• Relative Bias < 5%: Excellent performance

• Relative Bias < 10%: Acceptable performance

• Small SE: Stable estimates (increase nsim to reduce SE)

• RMSE ≈ SE when Bias ≈ 0: Indicates unbiased estimator

• Correlations typically show higher variability (larger SE/RMSE) than means/medians, especially with smaller sample sizes

For publication:

• Report Bias and Relative Bias to demonstrate unbiasedness

• Report SE to show precision

• Report MSE or RMSE for overall accuracy

• Typical table format: Group | Endpoint | Theoretical | Empirical | Bias | Relative Bias (%) | SE | RMSE | Assessment

References

Fleischer, F., Gaschler-Markefski, B., & Bluhmki, E. (2009). A statistical model for the dependence between progression-free survival and overall survival. Statistics in Medicine, 28(21), 2669-2686.

See also

rOncoEndpoints for generating correlated oncology endpoints, CorResponsePFS for calculating PFS-Response correlation, CorBoundResponseTTE for correlation bounds, CopulaParamResponseTTE for copula parameters

Examples

# Example 1: OS and Response with Clayton copula
set.seed(123)
sim_data1 <- rOncoEndpoints(
  nsim = 1000,
  group = c("Treatment", "Control"),
  n = c(100, 100),
  p = c(0.4, 0.3),
  hazard_OS = c(0.05, 0.07),
  rho_tte_resp = c(0.3, 0.2),
  copula = "Clayton"
)

check1 <- CheckSimResults(
  dataset = sim_data1,
  p = c(Treatment = 0.4, Control = 0.3),
  hazard_OS = c(Treatment = 0.05, Control = 0.07),
  rho_tte_resp = c(Treatment = 0.3, Control = 0.2),
  copula = "Clayton"
)
print(check1, n = Inf)
#> # A tibble: 8 × 10
#>   Group     Endpoint Empirical Theoretical     Bias Relative_Bias     SE     MSE
#>   <chr>     <chr>        <dbl>       <dbl>    <dbl>         <dbl>  <dbl>   <dbl>
#> 1 Treatment OS_Mean     20.0         20    -4.56e-2     -0.228    1.94   3.76   
#> 2 Treatment OS_Medi…    13.9         13.9   1.73e-2      0.125    1.95   3.81   
#> 3 Treatment Response     0.400        0.4   2.00e-5      0.00500  0.0488 0.00238
#> 4 Treatment Cor_OS_…     0.300        0.3   2.54e-6      0.000847 0.0936 0.00875
#> 5 Control   OS_Mean     14.4         14.3   6.90e-2      0.483    1.42   2.01   
#> 6 Control   OS_Medi…     9.96         9.90  6.00e-2      0.606    1.42   2.03   
#> 7 Control   Response     0.300        0.3   1.70e-4      0.0567   0.0457 0.00209
#> 8 Control   Cor_OS_…     0.205        0.2   4.61e-3      2.30     0.0993 0.00988
#> # ℹ 2 more variables: RMSE <dbl>, Assessment <chr>

# Interpretation:
# - Bias close to 0: method is unbiased
# - Relative_Bias < 5%: excellent performance
# - Small SE: precise estimates
# - RMSE ≈ SE when Bias ≈ 0: confirms unbiasedness

# Example 2: All three endpoints (OS, PFS, Response) with Frank copula
set.seed(456)
sim_data2 <- rOncoEndpoints(
  nsim = 1000,
  group = c("Experimental", "Standard"),
  n = c(150, 150),
  p = c(0.5, 0.35),
  hazard_OS = c(0.04, 0.06),
  hazard_PFS = c(0.08, 0.10),
  rho_tte_resp = c(0.4, 0.25),
  copula = "Frank"
)

check2 <- CheckSimResults(
  dataset = sim_data2,
  p = c(Experimental = 0.5, Standard = 0.35),
  hazard_OS = c(Experimental = 0.04, Standard = 0.06),
  hazard_PFS = c(Experimental = 0.08, Standard = 0.10),
  rho_tte_resp = c(Experimental = 0.4, Standard = 0.25),
  copula = "Frank"
)
print(check2, n = Inf)
#> # A tibble: 16 × 10
#>    Group     Endpoint Empirical Theoretical    Bias Relative_Bias     SE     MSE
#>    <chr>     <chr>        <dbl>       <dbl>   <dbl>         <dbl>  <dbl>   <dbl>
#>  1 Experime… OS_Mean     25.2        25     1.53e-1       0.613   2.07   4.31   
#>  2 Experime… OS_Medi…    17.4        17.3   6.68e-2       0.385   2.07   4.29   
#>  3 Experime… PFS_Mean    12.5        12.5   2.57e-2       0.205   0.995  0.990  
#>  4 Experime… PFS_Med…     8.72        8.66  5.32e-2       0.614   1.03   1.07   
#>  5 Experime… Response     0.502       0.5   1.63e-3       0.327   0.0404 0.00163
#>  6 Experime… Cor_OS_…     0.502       0.5   2.08e-3       0.415   0.0847 0.00719
#>  7 Experime… Cor_OS_…     0.403       0.4   3.48e-3       0.869   0.0643 0.00414
#>  8 Experime… Cor_PFS…     0.274       0.271 2.95e-3       1.09    0.0718 0.00516
#>  9 Standard  OS_Mean     16.7        16.7   4.84e-4       0.00291 1.42   2.01   
#> 10 Standard  OS_Medi…    11.6        11.6   7.90e-2       0.684   1.40   1.97   
#> 11 Standard  PFS_Mean    10.0        10     2.97e-3       0.0297  0.833  0.695  
#> 12 Standard  PFS_Med…     6.96        6.93  2.39e-2       0.345   0.801  0.643  
#> 13 Standard  Response     0.351       0.35  1.07e-3       0.307   0.0386 0.00149
#> 14 Standard  Cor_OS_…     0.603       0.6   3.10e-3       0.516   0.0810 0.00657
#> 15 Standard  Cor_OS_…     0.258       0.25  8.23e-3       3.29    0.0777 0.00611
#> 16 Standard  Cor_PFS…     0.190       0.184 6.12e-3       3.33    0.0833 0.00698
#> # ℹ 2 more variables: RMSE <dbl>, Assessment <chr>

# Note: PFS-Response correlation theoretical values are calculated
# using CorResponsePFS function, demonstrating the consistency
# of the three-endpoint model