Weekly Report -- 04/05/2012
Attempted to implement an ARIMA self-updating forecaster in python, using the formula that R claims to be using for its own modelling. This worked reasonably well, but was nowhere near as accurate as the forecasts that R had been producing. Spent a fair bit of time looking into what math I should be using for ARIMA forecasting without much success.
Eventually settled on just calling R itself for a while until I'm satisfied that ARIMA is suitable for what we are trying to do. This is a lot slower than I'd like as the model has to be re-applied everytime we see a new measurement. This produces the forecasts that I was expecting and now the main problem is to work out how we should be setting the threshold for determining whether an event has occurred or not. Initially, I tried to use the variance of the residuals in the initial model as a starting point but there was no obvious relationship between that and the forecast errors for genuine events.
Also found that we're going to have to limit the number of wavelet transforms we use for smoothing our original data, as each additional transform will increase the gap between the event occurring and the event being detected.
In between all that, marked 513 libtrace assignments. Was mildly amused by the range of mistakes and errors that the students made -- maybe libtrace programming isn't as easy as we thought!