The parametric bootstrap test is powerful, in the statistical sense.
However, it may become too liberal if the model you use to generate the
sequences is too simple, relative to the "true model". So, we can attempt
to minimize this potential error by using a complex and presumably realistic
substitution model. The effect of inappropriate model assumptions has been
shown in simulations (Huelsenbeck, 1996?), and in only one empirical study
that I know of (Sullivan and Swofford, 1997). In my experience the null
distribution is very strongly effected by the model used, and so the power
of the test changes markedly with the substitution model used. So, I think
you raise a good point that requires further study.
Durham, NC 27708-0325
E-mail: tbuckley at duke.edu
From: owner-mol-evol at hgmp.mrc.ac.uk [mailto:owner-mol-evol at hgmp.mrc.ac.uk]
On Behalf Of Vijay Aswani
Sent: Thursday, May 04, 2000 2:59 PM
To: mol-evol at net.bio.net
Subject: Parametric bootstrapping
First of all, I would like to thank all those who responded to my earlier
posting asking for information on parametric bootstrapping. They were Chris
Conroy, Andrew Rambaut, John Huelsenbeck, Thomas Buckley and Jack Sullivan.
A number of those who responded pointed me to the web site of Nick Goldman
who had a manuscript and some very helpful procedural tips on the process.
I am writing to report that I have carried out the process and ... have some
First of all, the procedure seems to call for simulating datasets (I used a
100 replications) using the null hypothesis tree and estimated likelihood
parameters of that tree. The null hypothesis tree, as I understand it, is
not the best ML tree but the hypothetical topology I wish to test (eg.
monophyly of a particular group).
Once I have the 100 datasets, I then calculate the ML score of the best tree
(by heuristic search) and the ML score of the hypothesis tree. I subtract:
Lnull - LML and plot the distribution of these 100 values.
I ran the test twice with 2 different hypotheses. I therefore simulated 100
datasets in each case. I noticed that in both of these results, the
difference in ML values between the ML score of the best tree and the null
hypothesis tree was very small. (As it turns out, in both cases, the
hypotheses were rejected because of a larger difference between the real
best ML score and null hypothesis ML score).
This brings me to my question: isn't using the hypothesis tree's topology
and ML parameters to build the 100 datasets and then computing the best tree
in each dataset a bit circular. Wouldn't the best tree in each case be the
same tree whose topology and ML parameters were used to create the data sets
in the first place? Perhaps the reason why L null and L ML differ so little
is that the dataset was created from the parameters of the null tree.
If this is true, then the range of L ml - L null would be very small (since
they would be almost the same) and almost every hypothesis tested would be
I would appreciate any thoughts on this ...
Vijay Aswani, Ph.D.
Smithsonian Tropical Research Institute,
APO AA 34002-0948
Email: aswaniv at naos.si.edu or vaswani at sinfo.net