Weekly Report for week ending 20 July 2012
Started pulling things together over the past few weeks. The modules are now all integrated into the main application. Some of the forking and threading code needs some TLC but things are at least functional at this point. The main problem is scalability. I'm starting a thread for each data source which falls over when you follow a few hundred data sources. I need to be more clever about identifying resources and ensuring I'm not reading from them more than once for multiple streams. I may also require some changes/additions to the AMP api to allow me to request multiple data items in a single request.
Next Brendon wants me to stream the data (from multiple sources) in order. I'm just restructuring code so I can get at the time codes together, however I've been sick on and off this week which hasn't helped progress. After that I'll try and get the database implementation going.