This week we have been learning about operating systems and how to create daemons to run on them. We have read lot of articles and found how to create a robust daemon to run on linux. We have created this as a superclass that each of the daemons we create will subclass this to give the daemon operation. All each subclass will have to do is overload the run method to work. They will also need a command method to be started, restarted and stopped by the main module. We decided on having many daemons running on the system rather than a big script in a loop for many reasons. One of the biggest was the ease at which modules could be turned on and off. It also allows for some modules to not start or break and the system not care about what the other modules are doing.
With the new daemon class we have tested it with the metric exporter class. This means that our metric exporter class just looks at logs that all other modules write their metrics to and send them to a graphite instance running. It sends them as udp packets across the network for fast and lightweight (network wise) metrics. This method seems to be working really well because we don’t care massively if we miss some metrics because the metric resolution we need is not high.
Now that we have graphite and grafana installed on our main server, we have our metric exporter daemon, and we have a test script to generate metrics, we pretty much have our base flow down. We can see at every step where the test metric is generated, all the way until it is graphed by grafana. This is giving us great inspiration for our project to have this base flow so early on. We have a long way to go but seeing testing pass like this makes the project really seem possible. A lot of our experimental scripts are too ad-hoc and held together with glue code to be used for testing the metrics at this time so what is flying across the network is hardly of any use but the framework is their to be built upon and we hope after the exams to get a good burst at creating plugins for the script to export real metrics.