--- title: "flipscores" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{flipscores} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r setup, include = FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>" ) ``` ## The "flipscores" package The package provides functions for hypothesis tests in GLMs based on sign-flipping score contributions. The tests are robust against overdispersion, heteroscedasticity and, in some cases, ignored nuisance variables. In this vignette, we provide some use cases and a simulation study to show possible usage of the package. ```{r, echo = FALSE, message = FALSE, warning = FALSE} # devtools::install_github("livioivil/flipscores") # you need devtools! library("flipscores") ``` The `flipscores` function is the main function of the package. It supports the standard usage of `R` modeling functions, such as `glm`, from which it inherits the majority of the arguments. ```{r} nn <- 30 set.seed(123) dt <- data.frame(X = rnorm(nn), Z = factor(rep(LETTERS[1:3], length.out = nn))) dt$Y <- rpois(n = nn, lambda = exp(dt$X)) mod <- flipscores(Y ~ Z + X, data = dt, family = "poisson", x = TRUE) summary(mod) ``` Functions like `summary` or `anova` can be applied to `flipscores` objects, obtaining an output in the usual format. The p-values are obtained by randomly sign-flipping score contributions and comparing the observed score with the flipped scores. ```{r} # Anova test anova(mod) # or mod0 <- flipscores(Y~Z,data=dt,family="poisson",x=TRUE) anova(mod0,mod) # and mod0 <- flipscores(Y~X,data=dt,family="poisson",x=TRUE) anova(mod0,mod) ``` Equivalently, one might apply the `flipscores` function to a precomputed `glm` model. ```{r} model <- glm(Y~Z+X,data=dt,family="poisson") mod2 <- flipscores(model) summary(mod2) ``` Notice that the p-values are not identical to the ones of the previous model. In fact, computing all possible combinations of flipped score contributions would be computationally unfeasible. Only a subset of this possibilities is computed, regulated by the parameter `n_flips`, with default at `5000`. Reproducibility can be achieved by setting the argument `seed` or precomputing flips with the function `make_flips`. ```{r} flps <- make_flips(n_obs = nn, n_flips = 1000) summary(flipscores(model, flips = flps)) summary(flipscores(model, flips = flps)) ``` One-sided testing alternatives are also covered ```{r} summary(flipscores(model, alternative = "less")) summary(flipscores(model, alternative = "greater")) ``` If only a subset of parameters is of interest, the `to_be_tested` parameter can be set to avoid computing the p-values for all the others ```{r} summary(flipscores(model, to_be_tested = "X", flips = flps)) summary(flipscores(model, to_be_tested = c(1, 4), flips = flps)) ``` `to_be_tested` accepts vectors with both the name of the variables and indexing numbers; these indicators refer to the name or the cardinality of the variable in the summary output, so the Intercept will always correspond to number 1, while particular attention should be paid if `factor/character` variables are involved. ```{r} summary(flipscores(model, to_be_tested = "ZB", flips = flps)) summary(flipscores(model, to_be_tested = 2, flips = flps)) ``` `flipscores` supports all the available families in `glm`, and also the Negative Binomial family. This, in particular, should be indicated as `"negbinom"` between quotes. ```{r warning = FALSE} summary(flipscores(Y ~ Z + X, data = dt, family = "negbinom")) ``` The `confint` method is also applicable to provide confidence intervals based on the inversion of the sign-flip score tests. These intervals inherit the properties of robustness to variance misspecification of the test. ```{r} confint(mod, n_flips = 1000) ``` As before, various options are allowed, including the possibility to generate symmetric confidence intervals. ```{r} confint(mod, parm = 4, flips = flps) # parm == to_be_tested! confint(mod, parm = 4, level = 0.99, flips = flps) confint(mod, parm = 4, alternative = "greater", flips = flps) confint(mod, parm = 4, type = "symmetric", flips = flps) ``` ## Other examples ```{r} set.seed(1) dat <- data.frame( treatment = factor(rep(c("Control", "Drug"), each = 30)), sex = factor(rep(c("F", "M"), times = 30)), age_group = factor(rep(c("Young", "Old", "Middle"), length.out = 60)), severity = runif(60, 0, 1) ) eta <- with(dat, -1 + 1.2 * (treatment == "Drug") * (sex == "M") + 0.6 * (age_group == "Old") - 0.3 * severity ) dat$y <- rbinom(nrow(dat), 1, plogis(eta)) flips=make_flips(60,5000) fit1 <- glm( y ~ treatment * sex + age_group + severity, data = dat, family = binomial ) # Formula interface: build contrasts and apply flip-score tests. cntr1=flipscores_contrasts( fit1, pairwise ~ treatment, flips = flips, seed = 1 ) fit2 <- flipscores( y ~ treatment + sex * age_group + severity, data = dat, family = binomial, flips = flips ) summary(fit2) # Formula interface: build contrasts and apply flip-score tests. cntr2=flipscores_contrasts( fit2, pairwise ~ treatment, flips = flips, seed = 1 ) cmbn=combine_contrasts(fit2) summary(cmbn) #perchè non coimbina tra livelli di group?? # jfs è la vecchia classe join_flipscores flipscores(list(cntr1,cntr2,fit2)) cres=flipscores:::c.jfs(cntr1,cntr2,fit2,cmbn) #deve funzionare anche per flipscores, fs_contrasts, fs_cobine,jfs #forse meglio chiamarla join_fs() o fualcosa del genere? #rivedere il nome degli objects: mode1 (evitare duplicati, cambia nome in caso) e credo anche .assign della summary_table #aggiungere pcor nel summary dei contrasti? #Le classi fs_contrast (e clip, flip2sss etc) devono avere i seguenti methods: #model.matrix per gli obs_names #update indispensabile per refit con stessa matrice flip #as.jfs convertire ciascuno così poi il c() è più facile. c'è già un default che dovrebbe coprire quasi tutti i casi: basta che nell'oggetto ci sia una $call, una $summary_table e un $Tspace, il resto viene cacciato in $objects) #fs_contrast (e clip, flip2sss etc devono avere gli argomenti n_flips (forse questo non serve in realtà) e flips in modo che poi l'update(flips=FLIP) funzioni # fs_combine è in pratica un offetto jfs (che non serva nemmeno farne un metodo a parte?) #IMPORTANTE: c() o come lo chiamiamo non controlla la coerenza dei flip (altrimenti fs_combine non può essere agganciato). il controllo lo fa flipscores() - usando quindi model.matrix e update. combine_contrasts(fit2) #non combine per livelli di age_group perchè non è un oggetto joint_flipscores combine_contrasts(cres) #invece funziona combine_tests(cres) combine_tests(cres,by="coefficient",comb_funct = "Fisher") #ora construire la selezione [[]] usa .assign length(cres$objects) ############### set.seed(1) n <- 50 df <- data.frame( y1 = rnorm(n), y2 = rnorm(n), yb = rbinom(n, 1, 0.5), y1p = rpois(n, lambda = 3), y2p = rpois(n, lambda = 5), x1 = rnorm(n), x2 = rnorm(n) ) #' # Case 1: single formula res=flipscores(y1 ~ x1 + x2, family = gaussian(), data = df) summary(res) #' # Case 2: list of glm objects res_lm=flipscores_lm(y1 ~ x1 + x2, data = df, tested_coeffs = "x2") summary(res_lm) models <- list( glm(y1 ~ x1 + x2, data = df, family = gaussian()), cntr1, cntr2, fit2, res_lm ) res=flipscores(models, family = gaussian(), data = df) summary(res) flipscores_lm(y1 ~ x1 + x2, data = df) ```