headers
Kenny Crump | 19 Sep 17:57
Favicon

MCMC question

This is our first experience with MCSIM.  We are attempting an MCMC run involving a fairly complicated PBPK model.  Last night we made runs on four different machines of 25,000 simulations, all identical except for the random number seed.  Output distributions for some parameters were dramatically different (based on the last 5,000 sims) , with some having very small variances on some runs compared to other runs.  Also, sometimes in the same run the distribution of the same parameter for different experiments are vastly different.  E.g., in one case the blood/fat partition coefficient got stuck on one value in one experiment but not in other experiments in the same run. 

 

It seems to me that this behavior could be caused by a bug in the dynamic method presently used to set the proposal distribution in the Metropolis algorithm.  I note that the papers on the Bois perc model do not mention this dynamic approach, instead refer to setting the variance in the proposal distribution based on preliminary runs.  Does the dynamic approach possibly have a bug?  Can you suggest other reasons for the observed behavior?

 

Thanks!

 ******************************************************************

Kenny S. Crump

ENVIRON

602 E. Georgia Ave.

Ruston, LA 71270

318-251-6985, 318-255-2277

FAX: 318-255-2040

kcrump <at> environcorp.com

http://www.angelfire.com/la2/kennycrumpspage/
 

 

____________________________________________________________________________
This message may contain confidential or privileged information.  If you have received the message in
error, please notify the sender by reply email @environcorp.com and delete the message without copying
or disclosing.  Thank you very much.
____________________________________________________________________________
Permalink | Reply | 0 comments |
headers
Frédéric BOIS | 23 Sep 15:26
Picon
Favicon

Objet : Help-mcsim digest, Vol 1 #127 - 1 msg

Hi!

Your diagnostic seems right: you are experiencing convergence problems. 
The distributed version of MCSim adjusts the kernel of each sampled parameter 
after 30 iterations and then less and less frequently, on the basis of the acceptance 
rate (target: about 30% accepted jumps). So far we have not found problems with 
this in well-behaved applications. The initial value of the kernel SD for a given parameter 
is obtained by sampling 4 values from the prior and taking there range (you get an SD 
commensurate with the prior). We know that there problem when the prior is very vague
(eg, for vague inverse gamma for variances). In that case the starting kernel SD can be 
many orders of magnitude larger than the target distribution and is very difficult to tune.
I am not sure if you have used such vague priors. 
In our experience convergence problems like yours may come from strong correlations 
between parameter estimates (do you see some in the simulation output ?) which may 
require model reparameterization, or from a structural model that does not "fit" well 
given the data likelihood used (a more difficult problem).
Just in case there would be a problem with the version of MCSim you are using, I am 
attaching the latest version (v5_beta). The code is actually ok (I think), but the manual
revision is not finished (optimal design and the optional  XWindows interface are not 
documented). The installation is automated and is simpler and more standard than
previously. You may want to try it.

Frederic Bois

-----Original Message-----
From: Kenny Crump 
Sent: Thursday, September 19, 2002 10:57 AM
To: help-mcsim <at> gnu.org 
Cc: Harvey Clewell; Eric Hack
Subject: MCMC question

This is our first experience with MCSIM.  We are attempting an MCMC run
involving a fairly complicated PBPK model.  Last night we made runs on
four different machines of 25,000 simulations, all identical except for
the random number seed.  Output distributions for some parameters were
dramatically different (based on the last 5,000 sims) , with some having
very small variances on some runs compared to other runs.  Also,
sometimes in the same run the distribution of the same parameter for
different experiments are vastly different.  E.g., in one case the
blood/fat partition coefficient got stuck on one value in one experiment
but not in other experiments in the same run.  

It seems to me that this behavior could be caused by a bug in the
dynamic method presently used to set the proposal distribution in the
Metropolis algorithm.  I note that the papers on the Bois perc model do
not mention this dynamic approach, instead refer to setting the variance
in the proposal distribution based on preliminary runs.  Does the
dynamic approach possibly have a bug?  Can you suggest other reasons for
the observed behavior?

Thanks!
 ******************************************************************
Kenny S. Crump
ENVIRON 
602 E. Georgia Ave.
Ruston, LA 71270
318-251-6985, 318-255-2277
FAX: 318-255-2040
kcrump <at> environcorp.com 
http://www.angelfire.com/la2/kennycrumpspage/
Attachment (mcsim-v5beta.tar.gz): application/octet-stream, 427 KiB
Permalink | Reply | 0 comments |

Gmane