By Jon Wakefield

ISBN-10: 1441909249

ISBN-13: 9781441909244

This ebook presents a balanced, smooth precis of Bayesian and frequentist tools for regression analysis.

Table of Contents

Cover

Bayesian and Frequentist Regression Methods

ISBN 9781441909244 ISBN 9781441909251

Preface

Contents

Chapter 1 advent and Motivating Examples

1.1 Introduction

1.2 version Formulation

1.3 Motivating Examples

1.3.1 Prostate Cancer

1.3.2 final result After Head Injury

1.3.3 Lung melanoma and Radon

1.3.4 Pharmacokinetic Data

1.3.5 Dental Growth

1.3.6 Spinal Bone Mineral Density

1.4 Nature of Randomness

1.5 Bayesian and Frequentist Inference

1.6 the administrative Summary

1.7 Bibliographic Notes

Part I

bankruptcy 2 Frequentist Inference

2.1 Introduction

2.2 Frequentist Criteria

2.3 Estimating Functions

2.4 Likelihood

o 2.4.1 greatest probability Estimation

o 2.4.2 editions on Likelihood

o 2.4.3 version Misspecification

2.5 Quasi-likelihood 2.5.1 greatest Quasi-likelihood Estimation

o 2.5.2 A extra advanced Mean-Variance Model

2.6 Sandwich Estimation

2.7 Bootstrap Methods

o 2.7.1 The Bootstrap for a Univariate Parameter

o 2.7.2 The Bootstrap for Regression

o 2.7.3 Sandwich Estimation and the Bootstrap

2.8 number of Estimating Function

2.9 speculation Testing

o 2.9.1 Motivation

o 2.9.2 Preliminaries

o 2.9.3 rating Tests

o 2.9.4 Wald Tests

o 2.9.5 chance Ratio Tests

o 2.9.6 Quasi-likelihood

o 2.9.7 comparability of try Statistics

2.10 Concluding Remarks

2.11 Bibliographic Notes

2.12 Exercises

bankruptcy three Bayesian Inference

3.1 Introduction

3.2 The Posterior Distribution and Its Summarization

3.3 Asymptotic houses of Bayesian Estimators

3.4 earlier Choice

o 3.4.1 Baseline Priors

o 3.4.2 major Priors

o 3.4.3 Priors on significant Scales

o 3.4.4 Frequentist Considerations

3.5 version Misspecification

3.6 Bayesian version Averaging

3.7 Implementation

o 3.7.1 Conjugacy

o 3.7.2 Laplace Approximation

o 3.7.3 Quadrature

o 3.7.4 built-in Nested Laplace Approximations

o 3.7.5 value Sampling Monte Carlo

o 3.7.6 Direct Sampling utilizing Conjugacy

o 3.7.7 Direct Sampling utilizing the Rejection Algorithm

3.8 Markov Chain Monte Carlo 3.8.1 Markov Chains for Exploring Posterior Distributions

o 3.8.2 The Metropolis-Hastings Algorithm

o 3.8.3 The city Algorithm

o 3.8.4 The Gibbs Sampler

o 3.8.5 Combining Markov Kernels: Hybrid Schemes

o 3.8.6 Implementation Details

o 3.8.7 Implementation Summary

3.9 Exchangeability

3.10 speculation trying out with Bayes Factors

3.11 Bayesian Inference according to a Sampling Distribution

3.12 Concluding Remarks

3.13 Bibliographic Notes

3.14 Exercises

bankruptcy four speculation checking out and Variable Selection

4.1 Introduction

4.2 Frequentist speculation Testing

o 4.2.1 Fisherian Approach

o 4.2.2 Neyman-Pearson Approach

o 4.2.3 Critique of the Fisherian Approach

o 4.2.4 Critique of the Neyman-Pearson Approach

4.3 Bayesian speculation trying out with Bayes components 4.3.1 assessment of Approaches

o 4.3.2 Critique of the Bayes issue Approach

o 4.3.3 A Bayesian View of Frequentist speculation Testing

4.4 The Jeffreys-Lindley Paradox

4.5 checking out a number of Hypotheses: common Considerations

4.6 checking out a number of Hypotheses: mounted variety of Tests

o 4.6.1 Frequentist Analysis

o 4.6.2 Bayesian Analysis

4.7 checking out a number of Hypotheses: Variable Selection

4.8 techniques to Variable choice and Modeling

o 4.8.1 Stepwise Methods

o 4.8.2 All attainable Subsets

o 4.8.3 Bayesian version Averaging

o 4.8.4 Shrinkage Methods

4.9 version development Uncertainty

4.10 a realistic Compromise to Variable Selection

4.11 Concluding Comments

4.12 Bibliographic Notes

4.13 Exercises

Part II

bankruptcy five Linear Models

5.1 Introduction

5.2 Motivating instance: Prostate Cancer

5.3 version Specifiation

5.4 A Justificatio for Linear Modeling

5.5 Parameter Interpretation

o 5.5.1 Causation as opposed to Association

o 5.5.2 a number of Parameters

o 5.5.3 facts Transformations

5.6 Frequentist Inference 5.6.1 Likelihood

o 5.6.2 Least Squares Estimation

o 5.6.3 The Gauss-Markov Theorem

o 5.6.4 Sandwich Estimation

5.7 Bayesian Inference

5.8 research of Variance

o 5.8.1 One-Way ANOVA

o 5.8.2 Crossed Designs

o 5.8.3 Nested Designs

o 5.8.4 Random and combined results Models

5.9 Bias-Variance Trade-Off

5.10 Robustness to Assumptions

o 5.10.1 Distribution of Errors

o 5.10.2 Nonconstant Variance

o 5.10.3 Correlated Errors

5.11 evaluate of Assumptions

o 5.11.1 assessment of Assumptions

o 5.11.2 Residuals and In uence

o 5.11.3 utilizing the Residuals

5.12 instance: Prostate Cancer

5.13 Concluding Remarks

5.14 Bibliographic Notes

5.15 Exercises

bankruptcy 6 normal Regression Models

6.1 Introduction

6.2 Motivating instance: Pharmacokinetics of Theophylline

6.3 Generalized Linear Models

6.4 Parameter Interpretation

6.5 chance Inference for GLMs 6.5.1 Estimation

o 6.5.2 Computation

o 6.5.3 speculation Testing

6.6 Quasi-likelihood Inference for GLMs

6.7 Sandwich Estimation for GLMs

6.8 Bayesian Inference for GLMs

o 6.8.1 previous Specification

o 6.8.2 Computation

o 6.8.3 speculation Testing

o 6.8.4 Overdispersed GLMs

6.9 evaluation of Assumptions for GLMs

6.10 Nonlinear Regression Models

6.11 Identifiabilit

6.12 chance Inference for Nonlinear types 6.12.1 Estimation

o 6.12.2 speculation Testing

6.13 Least Squares Inference

6.14 Sandwich Estimation for Nonlinear Models

6.15 The Geometry of Least Squares

6.16 Bayesian Inference for Nonlinear Models

o 6.16.1 earlier Specification

o 6.16.2 Computation

o 6.16.3 speculation Testing

6.17 overview of Assumptions for Nonlinear Models

6.18 Concluding Remarks

6.19 Bibliographic Notes

6.20 Exercises

bankruptcy 7 Binary facts Models

7.1 Introduction

7.2 Motivating Examples 7.2.1 final result After Head Injury

o 7.2.2 plane Fasteners

o 7.2.3 Bronchopulmonary Dysplasia

7.3 The Binomial Distribution 7.3.1 Genesis

o 7.3.2 infrequent Events

7.4 Generalized Linear types for Binary info 7.4.1 Formulation

o 7.4.2 hyperlink Functions

7.5 Overdispersion

7.6 Logistic Regression types 7.6.1 Parameter Interpretation

o 7.6.2 chance Inference for Logistic Regression Models

o 7.6.3 Quasi-likelihood Inference for Logistic Regression Models

o 7.6.4 Bayesian Inference for Logistic Regression Models

7.7 Conditional chance Inference

7.8 evaluate of Assumptions

7.9 Bias, Variance, and Collapsibility

7.10 Case-Control Studies

o 7.10.1 The Epidemiological Context

o 7.10.2 Estimation for a Case-Control Study

o 7.10.3 Estimation for a Matched Case-Control Study

7.11 Concluding Remarks

7.12 Bibliographic Notes

7.13 Exercises

Part III

bankruptcy eight Linear Models

8.1 Introduction

8.2 Motivating instance: Dental progress Curves

8.3 The Effciency of Longitudinal Designs

8.4 Linear combined types 8.4.1 the final Framework

o 8.4.2 Covariance versions for Clustered Data

o 8.4.3 Parameter Interpretation for Linear combined Models

8.5 probability Inference for Linear combined Models

o 8.5.1 Inference for fastened Effects

o 8.5.2 Inference for Variance elements through greatest Likelihood

o 8.5.3 Inference for Variance parts through constrained greatest Likelihood

o 8.5.4 Inference for Random Effects

8.6 Bayesian Inference for Linear combined types 8.6.1 A Three-Stage Hierarchical Model

o 8.6.2 Hyperpriors

o 8.6.3 Implementation

o 8.6.4 Extensions

8.7 Generalized Estimating Equations 8.7.1 Motivation

o 8.7.2 The GEE Algorithm

o 8.7.3 Estimation of Variance Parameters

8.8 evaluation of Assumptions 8.8.1 evaluate of Assumptions

o 8.8.2 techniques to Assessment

8.9 Cohort and Longitudinal Effects

8.10 Concluding Remarks

8.11 Bibliographic Notes

8.12 Exercises

bankruptcy nine basic Regression Models

9.1 Introduction

9.2 Motivating Examples

o 9.2.1 birth control Data

o 9.2.2 Seizure Data

o 9.2.3 Pharmacokinetics of Theophylline

9.3 Generalized Linear combined Models

9.4 chance Inference for Generalized Linear combined Models

9.5 Conditional chance Inference for Generalized Linear combined Models

9.6 Bayesian Inference for Generalized Linear combined types 9.6.1 version Formulation

o 9.6.2 Hyperpriors

9.7 Generalized Linear combined versions with Spatial Dependence 9.7.1 A Markov Random box Prior

o 9.7.2 Hyperpriors

9.8 Conjugate Random results Models

9.9 Generalized Estimating Equations for Generalized Linear Models

9.10 GEE2: attached Estimating Equations

9.11 Interpretation of Marginal and Conditional Regression Coeffiients

9.12 creation to Modeling based Binary Data

9.13 combined versions for Binary info 9.13.1 Generalized Linear combined versions for Binary Data

o 9.13.2 chance Inference for the Binary combined Model

o 9.13.3 Bayesian Inference for the Binary combined Model

o 9.13.4 Conditional probability Inference for Binary combined Models

9.14 Marginal versions for based Binary Data

o 9.14.1 Generalized Estimating Equations

o 9.14.2 Loglinear Models

o 9.14.3 additional Multivariate Binary Models

9.15 Nonlinear combined Models

9.16 Parameterization of the Nonlinear Model

9.17 probability Inference for the Nonlinear combined Model

9.18 Bayesian Inference for the Nonlinear combined Model

o 9.18.1 Hyperpriors

o 9.18.2 Inference for services of Interest

9.19 Generalized Estimating Equations

9.20 evaluate of Assumptions for common Regression Models

9.21 Concluding Remarks

9.22 Bibliographic Notes

9.23 Exercises

Part IV

bankruptcy 10 Preliminaries for Nonparametric Regression

10.1 Introduction

10.2 Motivating Examples

o 10.2.1 gentle Detection and Ranging

o 10.2.2 Ethanol Data

10.3 The optimum Prediction

o 10.3.1 non-stop Responses

o 10.3.2 Discrete Responses with ok Categories

o 10.3.3 basic Responses

o 10.3.4 In Practice

10.4 Measures of Predictive Accuracy

o 10.4.1 non-stop Responses

o 10.4.2 Discrete Responses with okay Categories

o 10.4.3 common Responses

10.5 a primary examine Shrinkage Methods

o 10.5.1 Ridge Regression

o 10.5.2 The Lasso

10.6 Smoothing Parameter Selection

o 10.6.1 Mallows CP

o 10.6.2 K-Fold Cross-Validation

o 10.6.3 Generalized Cross-Validation

o 10.6.4 AIC for normal Models

o 10.6.5 Cross-Validation for Generalized Linear Models

10.7 Concluding Comments

10.8 Bibliographic Notes

10.9 Exercises

bankruptcy eleven Spline and Kernel Methods

11.1 Introduction

11.2 Spline tools 11.2.1 Piecewise Polynomials and Splines

o 11.2.2 average Cubic Splines

o 11.2.3 Cubic Smoothing Splines

o 11.2.4 B-Splines

o 11.2.5 Penalized Regression Splines

o 11.2.6 a short Spline Summary

o 11.2.7 Inference for Linear Smoothers

o 11.2.8 Linear combined version Spline illustration: probability Inference

o 11.2.9 Linear combined version Spline illustration: Bayesian Inference

11.3 Kernel Methods

o 11.3.1 Kernels

o 11.3.2 Kernel Density Estimation

o 11.3.3 The Nadaraya-Watson Kernel Estimator

o 11.3.4 neighborhood Polynomial Regression

11.4 Variance Estimation

11.5 Spline and Kernel equipment for Generalized Linear Models

o 11.5.1 Generalized Linear types with Penalized Regression Splines

o 11.5.2 A Generalized Linear combined version Spline Representation

o 11.5.3 Generalized Linear versions with neighborhood Polynomials

11.6 Concluding Comments

11.7 Bibliographic Notes

11.8 Exercises

bankruptcy 12 Nonparametric Regression with a number of Predictors

12.1 Introduction

12.2 Generalized Additive types 12.2.1 version Formulation

o 12.2.2 Computation through Backfittin

12.3 Spline tools with a number of Predictors

o 12.3.1 normal skinny Plate Splines

o 12.3.2 skinny Plate Regression Splines

o 12.3.3 Tensor Product Splines

12.4 Kernel tools with a number of Predictors

12.5 Smoothing Parameter Estimation 12.5.1 traditional Approaches

o 12.5.2 combined version Formulation

12.6 Varying-Coefficien Models

12.7 Regression bushes 12.7.1 Hierarchical Partitioning

o 12.7.2 a number of Adaptive Regression Splines

12.8 Classificatio

o 12.8.1 Logistic types with ok Classes

o 12.8.2 Linear and Quadratic Discriminant Analysis

o 12.8.3 Kernel Density Estimation and Classificatio

o 12.8.4 Classificatio Trees

o 12.8.5 Bagging

o 12.8.6 Random Forests

12.9 Concluding Comments

12.10 Bibliographic Notes

12.11 Exercises

Part V

Appendix A Differentiation of Matrix Expressions

Appendix B Matrix Results

Appendix C a few Linear Algebra

Appendix D chance Distributions and producing Functions

Appendix E services of ordinary Random Variables

Appendix F a few effects from Classical Statistics

Appendix G simple huge pattern Theory

References

Index

**Read or Download Bayesian and Frequentist Regression Methods PDF**

**Similar probability books**

This vintage textual content offers a rigorous advent to uncomplicated chance thought and statistical inference, with a different stability of concept and technique. fascinating, proper functions use actual info from real stories, exhibiting how the recommendations and strategies can be utilized to resolve difficulties within the box.

**Ecole d'Ete de Probabilites de Saint-Flour XIII by D. J. Aldous PDF**

Examines using symbols in the course of the international and the way they're used to speak with out phrases.

**Tomasz R. Bielecki's Credit risk: modeling, valuation and hedging PDF**

The most goal of credits hazard: Modeling, Valuation and Hedging is to provide a accomplished survey of the prior advancements within the region of credits possibility study, in addition to to place forth the newest developments during this box. a big point of this article is that it makes an attempt to bridge the distance among the mathematical idea of credits threat and the monetary perform, which serves because the motivation for the mathematical modeling studied within the e-book.

- An Introduction to Stochastic Processes
- Seminaire de Probabilites XII
- Statistical mechanics, kinetic theory, and stochastic processes
- Introduction to Probability
- The Empire of Chance: How Probability Changed Science and Everyday Life (Ideas in Context)

**Additional resources for Bayesian and Frequentist Regression Methods**

**Sample text**

In the following, for ease of presentation, we assume that Yi , i = 1, . . 7) for all θ. The estimating function Gn (θ) is a random variable because it is a function of Y . The corresponding estimating equation that defines the estimator θn has the form 1 Gn (θn ) = n n G(θn , Yi ) = 0. 8) i=1 For inference the asymptotic properties of the estimating function are derived (which is why we index the estimating function by n), and these are transferred to the resultant estimator. 8) will often be unavailable in closed form and so deriving its distribution from that of the estimating function 2 In a regression setting we have independently distributed observations only, because the distribution of the outcome changes as a function of covariates.

For example, boy 10 has consistently higher measurements than the majority of boys. There are two distinct approaches to modeling longitudinal data. In the marginal approach, the average response is modeled as a function of covariates (including time), and standard errors are empirically adjusted for dependence. In the conditional approach, the response of each individual is modeled as a function of individualspecific parameters that are assumed to arise from a distribution, so that the overall variability is partitioned into within- and between-child components.

5) is nonlinear in the parameters. Such models will be considered in Chap. 6, including their use in situations in which additional information on the parameters is incorporated via the specification of a prior distribution. 2 are from a single subject. In the original study, data were available for 12 subjects, and ideally we would like to analyze the totality of data; hierarchical models provide one framework for such an analysis. Hierarchical nonlinear models are considered in Chap. 9. 3 gives dental measurements of the distance in millimeters from the center of the pituitary gland to the pteryo-maxillary fissure in 11 girls and 16 boys recorded at the ages of 8, 10, 12, and 14 years.

### Bayesian and Frequentist Regression Methods by Jon Wakefield

by Daniel

4.2