The Frequentist, Assisted by Bayes (FAB) framework aims to construct confidence regions that leverage information about parameter values in the form of a prior distribution. FAB confidence regions (FAB-CRs) have smaller volume for values of the parameter that are likely under the prior, while maintaining exact frequentist coverage. This work introduces several methodological and theoretical contributions to the FAB framework. For Gaussian likelihoods, we show that the posterior mean of the parameter of interest is always contained in the FAB-CR. As such, the posterior mean constitutes a natural notion of FAB estimator to be reported alongside the FAB-CR. More generally, we show that for a likelihood in the natural exponential family, a transformation of the posterior mean of the natural parameter is always contained in the FAB-CR. For Gaussian likelihoods, we show that power law tails conditions on the marginal likelihood induce robust FAB-CRs, that are uniformly bounded and revert to the standard frequentist confidence intervals for extreme observations. We translate this result into practice by proposing a class of shrinkage priors for the FAB framework that satisfy this condition without sacrificing analytical tractability. The resulting FAB estimators are equal to prominent Bayesian shrinkage estimators, including the horseshoe estimator, thereby establishing insightful connections between robust FAB-CRs and Bayesian shrinkage methods.