Comparison between AD Model Builder and other statistical modeling packages

Description of the fisheries model test

The performance test described here were performed by independent (that is independent of Otter Research Ltd.) researchers Jon Schnute and Norm Olsen at the Pacific Biological Station in Nanaimo, British Columbia.

To compare the performance of AD Model Builder with other statistical modeling packages a "typical" fisheries management model was chosen. This is a catch-at-age model which is described in the PDF file Schnute et al (1998). It is similar to the CATAGE example described in the AD Model Builder documentation. For the runs done here the model had 100 parameters. The model was coded in AD Model Builder, Gauss, Matlab, and S-plus. All versions were given the same initial starting values for the parameters and the various optimization schemes were run until convergence. The S-plus version crashed due to lack of memory (this is apparently due to either a memory leak in S-plus or inefficient garbage collection). For the Gauss runs the Gauss optimization toolkit solver, Optmum, was employed. For MATLAB runs the optimization toolkit solver, Fminu, was employed.

These results were presented in a paper presented by Jon Schnute at the 15th Lowell Wakefield Fisheries Symposiumi, Anchorage, October 8-11, 1997--> Code for these models was written by Norm Olsen.

Gauss and MATLAB perform many more function evaluations than AD Model Builder because they estimate the derivatives by finite differences. In contrast AD Model Builder can compute exact values for the derivatives at the same time as it evaluates the function and this extra computation requires only about 4 times as much time as it takes to calculate the function itself. Even with this extra overhead for derivative calculation, AD Model Builder was still slightly faster per function evaluation than Gauss. This speed reflects the efficiency of compiled C++ code.