Friday, November 29, 2013

MARSS 3.6 up on CRAN. Significant speed increases for large models

MARSS 3.6 has been uploaded to CRAN.  I fixed some inefficiencies that were causing DFA models with many time-series (n>100) and R="diagonal and unequal" to be very, very slow.  My tests show 10x faster fits for n=100 and R="diagonal and unequal" for DFA models.

http://cran.r-project.org/web/packages/MARSS/index.html

Tuesday, October 8, 2013

New paper out by Jim and Eric on using Delta-GLMMs to analyze fisheries survey data

Thorson, J.T. and E.J. Ward. 2013. Accounting for space-time interactions in index
standardization models. Fisheries Research,147:426:433

Scientific survey data are used to estimate abundance trends for fish populations worldwide, and are frequently analyzed using delta-generalized linear mixed models (delta-GLMMs). Delta-GLMMs incorporate information about both the probability of catch being non-zero (catch probability) and the expected value for non-zero catches (catch rates). Delta-GLMMs generally incorporate year as a main effect, and frequently account for spatial strata and/or covariates. Many existing delta-GLMMs do not account for random or systematic differences in catch probability or rates in particular combinations of spatial strata and year (i.e., space–time interactions), and do not recognize potential correlation in random space–time interactions between catch probability and catch rates. We therefore develop a Bayesian delta-GLMM that estimates correlations between catch probability and rates, and compare it with either (a) ignoring year–strata interactions, (b) modeling year–strata interactions as fixed effects, or (c) estimating year–strata interactions in catch probability or rates as independent random effects. These four models are fitted to bottom trawl survey data for 28 species off the U.S. West Coast. The posterior median of the correlation is positive for the majority (18) of species, including all five for which the posterior distribution has little overlap with zero. However, estimating this correlation has little impact on resulting abundance indices or credible intervals. We therefore conclude that the correlated random model will have a little impact on index standardization of the West Coast bottom trawl dataset. However, we propose that the correlated model can quickly identify correlations between occupancy probability and density, and provide our code to allow researchers to quickly identify whether such a correlation is likely to be significantly different from zero for their chosen data set.

Friday, July 5, 2013

Building R packages with RStudio and embedding R in your documents and reports

Building R packages with RStudio plus Embedding R in documents short-course on-line:

http://www.iugo-cafe.org/chinook/view_node.php?id=2962

Topics:
  • how (and why) to make an R package using RStudio
  • installing packages from github, git or a url to your tar.gz file
  • Using Sweave and RStudio to do 'reproducible research/programming'.
  • Using OpenOffice + R to do the same, if you don't like LaTeX
  • Creating web-apps that run your R code (a few links to demos)

Tuesday, March 12, 2013

Week 10: Applied Time-Series Analysis for Fisheries and Environmental Data

Class material: webpage

Week 10: Dynamic linear models
This week, we give a brief introduction to an important class of MARSS models: dynamic linear models. These are multivariate linear regression models where the regression parameters (slope and intercepts) are treated as a AR process and thus are allowed to time evolve. We also review some of the diagnostics for MARSS models fits.

Lab 10 In the lab, you'll go through an simple example of a univariate dynamic linear model with time-varying slope and intercept.
Lecture 10 You can find the pdf of lecture on the class webpage along with the link to watch a recording of the lecture.

Tuesday, March 5, 2013

Week 9: Applied Time-Series Analysis for Fisheries and Environmental Data

Class material: webpage

Week 9: Bayesian hierarchical multivariate state-space models
This week, we discuss fitting non-linear MARSS models and MARSS models with non-Gaussian errors using Bayesian methods. This is a very brief introduction and many shows some examples of how one sets up a MARSS model in JAGS and shows you what the posteriors of some models look like.
* Posteriors for MARSS models
* Intro to JAGS (the Gibbs sampler we will be using)
* many examples of fitting non-linear and non-Gaussian MARSS models

Lab 9 The main lab is to go through the JAGS examples shown in the lecture.
Lecture 9 You can find the pdf of lecture on the class webpage along with the link to watch a recording of the lecture.

Tuesday, February 26, 2013

Week 8: Applied Time-Series Analysis for Fisheries and Environmental Data

Class material: webpage

Week 8: Estimating interactions (the B matrix)
This week, we discuss issues related to the estimation of the B matrix in the context of using it to represent species interactions in a community dynamics models.
* univariate discrete time Gompertz model
* multivariate discrete time Gompertz model
* including covariates
* spurious density dependence resulting from ignoring observation error
* uncertainty in B elements resulting from estimating observation variance
* different methods for estimating confidence intervals: bootstrapping, hessian approximation, profile likelihood
* diagnostics

Lab 8
The main lab is to go through case study 7 in the MARSS User Guide and the corresponding code.
Lecture 8
You can find the pdf of lecture on the class webpage.


Week 7: Applied Time-Series Analysis for Fisheries and Environmental Data


Class material: webpage

Week 7: Dynamic factor analysis

This week, we use MARSS to do dynamic factor analysis (DFA), which allows us to look for a set of common underlying trends among a relatively large set of time series (Harvey, 1989, sec. 8.5). This is conceptually different than what we have been doing in the previous weeks. Here we are trying to explain temporal variation in a set of n observed time series using linear combinations of a set of m hidden random walks, where m << n. You can think of this as PCA for time-series data. Zuur et al. (2003) show a number of examples of DFA applied to catch data and densities of zoobenthos.

Lab 7
The main lab is to go through the dynamic factor analysis chapter in the MARSS User Guide and the corresponding code.

Lecture 7
You can find the ppt of lecture on the class webpage. Technical difficulties prevented recording of the lecture.

Wednesday, February 13, 2013

Week 6: Applied Time-Series Analysis for Fisheries and Environmental Data


Class material: webpage

Week 6: Introduction to including covariates in multivariate time-series model
This week we introduce the inclusion of covariates using the framework of a multivariate autoregressive model written in state-space form.  You will understand the lecture better if you read the chapter on covariates in the MARSS User Guide first.  Much of the lecture is about how to include covariates in different mathematically equivalent ways.  You'll want to translate the R code in the lecture into the mathematical formulas (matrix form) to see how covariates are entering the mathematical model.


Lab topic:
The main lab is to go through the covariate chapter and examples in the MARSS User Guide.  Then we have some salmon data to play with to try different ways of including cycles (in this case driven by cohort strength) into an analysis.

Lecture 6
Click the big arrow to start.  You can also find the ppt of lecture t on the class webpage.



Wednesday, February 6, 2013

Week 5: Applied Time-Series Analysis for Fisheries and Environmental Data

Class material: webpage

Week 5: Introduction to multivariate autoregressive state-space models
Lecture topics:
  • Review of dealing with obs error with ARIMA (from last week)
  • Multivariate state space models
  • How these are expressed mathematically
  • Analysis of multi-site data using this framework
  • Parameter estimation: Kalman filter, Newton methods and EM algorithm
Lab topic:
The main lab is to go through case study 2 in the MARSS User Guide. I have a Fish 507 specific version of the case study code on the course website with questions to answer as you go through. Case study 3 and 8 are optional but going through them will help solidify your understanding of multivariate state-space models. Do go through the ARMA code as it discusses some important points about the effects of data transformation (in this case differences) on the time-series model that is appropriate for the data.

Lecture 5
Click the big arrow to start the show. You can also find just a pdf of lecture 5 on the class webpage.

Thursday, January 31, 2013

Week 4: Applied Time-Series Analysis for Fisheries and Environmental Data

Class material: webpage

Week 4: Introduction to univariate autoregressive state-space models
Topics:
  • State-space models
  • Process versus observation error
  • Model Selection


Lecture 4
Click the big arrow to start the show. You can also find just the ppt of lecture 3 on the class webpage.

Tuesday, January 22, 2013

Week 3: Applied Time-Series Analysis for Fisheries and Environmental Data

Class material: webpage

Week 3: Estimation, model selection, and forecasting for time series models
Topics:
  • Summarizing ARIMA models
  • Estimation
  • Model Selection
  • Prediction & forecasting
  • Evaluating forecasts
  • Functions: arima(), lm(), Arima()
Lecture 3
This is our second attempt at recording a lecture. Still much to be learned but we are getting better.  Click the big arrow to start the show. You can also find just the ppt of lecture 3 on the class webpage.

Tuesday, January 15, 2013

Week 2: Applied Time-Series Analysis for Fisheries and Environmental Data

Class material: webpage

Week 2: Correlation, stationarity & stationary time-series models
The lecture introduces the ACF, PACF, and basic properties of AR, MA and ARMA models. The computer code section shows R code to analyze simulated time-series data so that participants get a feel for ACF and PACF and get a feel for AR and MA processes. The participants then move to analyzing some real time-series data using the 30+ year time-series of Lake Washington plankton.

Lecture 2
This is our first attempt at recording a lecture. Ahem, there is clearly much to be learned to improve the process...Click the big arrow to start the show. You can also find just the ppt of lecture 2 on the class webpage.