Ecological Archives A016-001-A3

N. Thompson Hobbs and Ray Hilborn. 2006. Alternatives to statistical hypothesis testing in ecology: a guide to self teaching. Ecological Applications 16:5–19.

Appendix C. A discussion on how to use likelihood and information theoretics to evaluate support in data for competing models.

Here we illustrate how to use likelihood and information theoretics to evaluate support in data for competing models. The supplement to this paper illustrates the computation in detail and can be downloaded into a spreadsheet. Our example uses the data of Vucetich et al. (2002) who examined the strength of evidence in data competing models of functional response of wolves preying on moose in Isle Royale National Park.  In particular, they compared the ability of two types of models to predict per capita kill rates of moose by wolves, models that based their predictions purely on prey density and models that included prey and predator density as predictors.  A detailed treatment of the motivation for their work is beyond the scope of this box; however, the paper provides a fine illustration of the clear thinking and scholarship that should stand behind the development of a candidate set of models.  In addition, Vucetich et al. (2002) illustrate the use of the new statistics to address a fundamentally important problem in ecology that is not amenable to traditional, manipulative experiments — understanding mechanisms of functional response of large carnivores.

 

 Models

Although Vucetich et al. considered 14 models, we will limit our treatment to three of them to provide a concise example.  The authors grouped the models they considered into three classes: prey dependent, predator dependent, and ratio dependent.  Prey dependence was portrayed using the disc equation of Holling (1959),

(C.1)

 

where N is the population size of moose, P is the population size of wolves,  is the instantaneous, per capita kill rate, a is the capture efficiency, and h is the handling time.  This model represents the hypothesis that capture rates depend solely on the abundance of prey and increase asymptotically with prey density because handling time becomes a larger fraction of the total time spent foraging as encounter rate with prey increases.  Vucetich et al. (2002) review reasons why kill rates may also respond to predator density, including predator avoidance by prey, group hunting, interference among predators, and the limitation of the predator population by resources other than prey.  The combined effects of predator and prey on kill rates can be represented using a predator dependent model (Beddington 1975, Deangelis et al. 1975),

(C.2)

 

which represents the hypothesis that foraging time by predators includes three mutually exclusive activities: searching, handling, and avoiding other predators.  Thus, increasing numbers of predators causes foraging time to decline as the time spent avoiding other predators increases. Finally, the kill rate may depend on the ratio of predators to prey (Arditi and Ginzburg 1989):

(C.3)

 

which represent the hypothesis that function response is a function of the per capita prey abundance rather than absolute abundance.

 

Data

To evaluate these models, Vucetich et al. (2002) used 94 observations of wolf kill rates, , estimates of moose abundance (N) and estimates of wolf abundance (P).  The data set included 1–4 observations per year during 1971–2001.

 

Likelihood estimation of model parameters

The first step in evaluating the support in data for these three models is to choose a likelihood function, a probability density function that specifies the probability of an observation conditional on the models parameters.  Because we can think of function response as an additive process — the total number of prey consumed is the sum of the individual prey — a logical starting point for data like these is to use the normal probability density function (Appendix B).  Thus, a likelihood function for evaluating our models given the data is

(C.4)

 

As before, the is a set of value model parameters, yiis the ith observation in the data set, is the ith prediction of the model, n is the number of data points, and is

(C.5)

 

It is important to understand that the i’s index the observations of kill rates, moose densities, and wolf densities.  Equation C.4 gives the likelihood of a single observation.  The likelihood of a model given the full data set is the product of the individual likelihoods,

(C.6)

 

or alternatively, we can find the log likelihood as the sum of the logs of the individual likelihoods,

(C.7)

 

We seek estimates for the parameters in the model, , such that the likelihood or the log likelihood are maximum; that is, the maximum likelihood estimates (MLE) of the model’s parameters, . These are the parameter values that minimize the differences between model predictions and the observations.  Historically, analytical methods offered the only method for finding the MLE, but modern computers allow us to find them using a numerical search of the parameter space.  Although Table C1 shows only four pairs of observations and predictions, it is easy to imagine how a total likelihood could be composed from all 94 observations in the dataset (see archive reference for example).    

By conducting the computations illustrated in Table C1 for all models, we now have a set of r = 1…3 candidate models, each with MLE estimates for their parameters, .  The maximized likelihood for each model is .

 

Evaluating the models

We use the to evaluate the relative support in the data for each model by estimating the Kullback-Leiblier information discrepancy, the information that is lost when model r is used to estimate the "true model" of functional response.  This estimate is computed for each model using Akaike Information Criterion adjusted for small samples, AICc,

(C.8)

 

where K is the number of parameters in the model + 1, the + 1 allowing for the estimate of  (Table C2).  Equipped with the AICc, we can compare the strength of evidence in the data for each model (Table C2) using Akaike weights, wr.  The wr values show that the ratio dependent model has almost twice as much support in the data (wr  = 0.65 vs. 0.35) as its closest competitor, the predator dependent model (wr = 0.35).  The prey dependent model has virtually no support relative to the other models (wr = 0).  Thus, this analysis provides strong evidence that wolf functional response depends on wolf population size, as well as the population size of moose.  Purely prey dependent models are clearly inferior to those that include predators as well as prey.  even if the ratio dependent model emerged as the best Kullback-Leibler model given the data and the models considered, there was also substantial support for the predator dependent model.

 

Multi-model inference

 Because there was uncertainly about which of the predator-based models was the best, we need to use multi-model inference for estimating parameter values or for making predictions of kill rates.  For example, all models contained the parameter a portraying the attack rate.  The MLE value of a for the ratio dependent model is 0.042; for the predator dependent model the value is 0.0339.  The model averaged estimate of a incorporates uncertainty in model selection using the wr to calculate a weighted average, that is aw = 0.042 × 0.65 + 0.034 × 0.35 = 0.039.  In this case, the wr for the third model is 0 to several decimal places and so, it is not used in the calculation of aw. However, if it had a non-zero weight, we would need to include it, or we would calculate the wr renormalized to include only those models with substantial support in the data, for example, those that had wr > 0.05.  We can also average predictions of the models; for example, in 1983 there were 820 moose and 23 wolves.  The ratio dependent model predicts a kill rate of 0.80 moose/(wolf × month), while the kill rate predicted by the predatory dependent model is 0.55.  The model averaged prediction is 0.65 × 0.80 + 0.35 × 0.55 = 0.71.

 

Confidence intervals

Confidence intervals on model parameters can be calculated using likelihood profiles, which show how the likelihood of the model changes as the value of a parameter of interest is varied systematically on each side of the MLE (Fig. C1).  Likelihood profiles are calculated by iterating over a relevant range of fixed values of the target parameter, estimating the MLEs of other parameters, and tabulating the resulting model likelihood for each fixed value of the target (Fig. C1).  To calculate a confidence interval on the estimate of a model parameter we use the deviance (), defined as two times the difference between the log likelihood obtained using the MLE value of the parameter () and the log likelihood obtained when using an alternative value of the parameter ():

(C.9)

 

Because the deviance is distributed as chi-square with one degree of freedom, we can calculate a 1 - a confidence interval on the maximum likelihood estimate of the parameter by finding the range of values of  for which  (Fig. C1).  Thus, a 95% confidence interval on the parameter a is 0.0350–0.051 (Fig. C1).  These confidence intervals do not incorporate model selection uncertainty. Bootstrapping is the preferred approach to incorporating this uncertainty in confidence intervals on model parameters (Burnham and Anderson 2002:166).

 

Software for likelihood estimation

The calculations shown above are illustrated in a Microsoft Excel workbook accessible in Ecological Archives.  Simple computations of likelihoods and AIC values do not require elaborate software. All that is needed is a way to code ecological models, access to simple statistical functions for discrete and continuous distributions, and a non-linear optimizer. We have found Microsoft Excel provides a very flexible tool for doing the type of computations illustrated above, a tool that is particularly useful for teaching because all of the steps are transparent.  Many other software packages, including standard statistical software [e.g., SAS (SAS 2000), Splus (Splus 2000), and R (Venables et al. 2003)], as well as software specialized for fitting models (A.D. Model Builder, http://otter-rsch.com/admodel.htm), allow calculation of maximum likelihoods and associated model selection statistics.


 

TABLE C1. Illustrations of computations used to find maximum likelihood estimates of parameters (a, h) of the prey dependent model (Eq. C.1).  Four observations of kill rates paired with observations of moose density are shown here; the full data set includes 94 of these observations

 

Index

Number of moose

Kill rate

Model prediction

Observation – prediction

Likelihood

Log (likelihood)

Years

i

1971

1

1156

1.37

1.00

0.37

0.650

-0.431

1972

2

1191

.81

1.01

-0.20

0.871

-0.138

1972

3

1191

1.23

1.01

0.22

0.853

-0.159

1973

4

1153

1.07

1.00

0.07

0.967

-0.033

 

TABLE C2.   Selection statistics for three models of wolf functional response.

   

AICc

Dr

L(Mr|Y)

wr

Model, Mr

K

Ratio Dependent

3

82.65

0.00

1.000

0.65

Predator Dependent

4

83.88

1.23

0.542

0.35

Prey Dependent

4

103.45

20.80

0.000

0.00


 

 
   FIG. C1. Likelihood profile for the estimate of the attack rate in the ratio dependent model.  Model likelihoods are calculated by iterating over the range of values of a and recalculating the model likelihood based on the MLE of the other model parameter, h, at each value of a.  Confidence intervals (vertical lines) on the maximum likelihood estimate of the parameter are estimated as the parameter values producing model likelihoods that differ from the maximum likelihood by (horizontal lines). Unlike the illustration here, likelihood profiles, and hence, confidence intervals based on them, will often be asymmetrical.



[Back to A016-001]