I saw Murray Jorgensen about the Veitch paper. He suggested that the Algorithm could not be sped up, and that the selection of alpha values may be unduly strict. He also said that it would be useful to compare the distributions of LB valencies collected under the regimes of the first paper and the the second paper on the topic of stopping probabilities i.e. the original Scamper vs. the Veitch 99% MDA analysis modes. I summarised this comparison for using the existing data, with a long time interval, however I will do it again once the new data is collected.

A further analysis of port bits was carried out using Tony's method. A further modification was made because significance was being attributed in the situation where all signs for 0.5-meanbit were the same. If all signs were the same the value for these accumulated differences divided by the number of successors was zeroed. There may be a gentler approach to this.

As the new scamper driver for collecting UDP and TCP sport data under original scamper MDA conditions is running, I have avoided running the after-run database analysis on yoyo, as this uses as much CPU as it can, and also floods the database with requests. I did not want to upset the scamper run. As yoyo data is not really protected against a disk crash I would prefer to move a copy of the database elsewhere, as well. This analysis will produce UDP TCP comparison data with confirmation of low values through carrying out high confidence repeats when necessary during MDA data collection.

A program was written to simulate MDA analysis and collect confidence information relating to k and n, the number of successors and the number of packets to send given k found. It does not run as slow as the Veitch approach and gives slightly different numbers, but may be more well defined and believable in terms of the path length. It uses the simple case of when all the nodes have the same number of successors. The n values for high confidence are a bit wobbly, as I expected they might be, requiring that the number of cycles be increased in further runs involving these high values.

More editing was carried out on the document on the type of result that should go into the paper.

My progress report is due in a month.

Updated my detectors to output events using the new protocol and hooked it
into the eventing tool. It now runs online and stores events in the
database rather than text files. Put together an example webpage to try to
expose most of the information from the database and see what it might be
interesting to do - presenting just the useful information in a tidy way
is going to be an interesting project. It's quite cool to be able to see
all events going on to and/or from every site, but a little overwhelming
in terms of the amount of data presented.

Sat down with Jamie and Shane and went over some of the sysadmin things
that we will be taking over. Got some ldap account management scripts to
update which should help bring me up to speed with how it all works.

I spent this week working on my 520. I have implemented an API inside my web app that will accept data from my collector I have written as a plugin for collectd.

Next job is to start drawing some pretty graphs.

Started on metadata support this week. I'm really starting to get a feel for how the protocol has been influenced by Nicira's work on Open vSwitch. Although it doesn't yet support much of the 1.1 and 1.2 specifications, they use the constants from 1.2 to define matches. I've submitted a patch for review, parsing metadata messages. With the feedback come back I'm looking at how I can use NXAST (Nicira's action type extensions) to test and verify that the patch works. When the 1.1 instruction support goes through, I'll be able to get OpenFlow to control the metadata rather than just NXAST.

Developed a web-based system for performing manual classification of anomalous traffic measurements. This should greatly speed up the process of validating my anomaly detection code and will make it possible to start to crowd-source the validation in the long term.

Using my new system, I classified a new time series which had quite a few false positives. Based on this, I've concluded that my most recent threshold changes were a step in the right direction but using a fixed threshold (unsurprisingly) only works well in some instances. In response, I've started experimenting with trying to calculate a new threshold for sample mean movement that is based on the properties of the time series itself.

Managed to get OpenSSL certificates playing together with the Java system in the ActiveMQ STOMP server, which took far more effort than it should have and documented how I did it (there may be a more sensible way, I'll look into that later). Made some headway into some basic STOMP frame parsing - string manipulation in C is so simple and bullet proof secure! Didn't make quite as much headway as I would have liked but I'm getting there.

This week was spent solely on creating an I2C library for communication with the temperature sensor and accelerometer. The library is finished and the all functions of the temperature sensor have been tested. This coming week will be spent studying for my exam then it will be onto making the accelerometer work.

A brief report on what numbers should be included in the paper was written. Further work on what is actually a novel contribution is required.

Two analyses of the relationship between next hop address and flow ID were carried out. These two parameters were plotted against each other, in order to scan for patterns.

An initial analysis of port bit and segregation into successor groups of packets was carried out. Further validation of these results is required. The analysis determining if the first link group of packets of a load balancer set the same bit to the same value. If it did comparison with the other links was carried out and if a link group which set the bit the other way was found, then the count for that bit was incremented. This is a slightly over-simplified approach as only one link group is compared to the others. As it is, it does run quite fast.

Finished all my assignments for A semester this week ready to start work on my 520 again next week.

As expected, I spent this week on my last 513 and 553 assignments. Synced in with Ben regarding my plans to implement Metadata support, so going to focus on digging into the code this week, and do a bit of research on what other features are still needed for OF1.1+ support.