Once in a while I end up facing problems due to the most disgusting global variable at all: time. Time can not be mocked easily. You'll have features based on time. You'll have reports based on time. You'll have to make the right decisions at the most appropriate time, etc... I could - and probably will - start a series of posts about time and its presence in software development. This post is dedicated to distributed build systems. I have a setup of a master Jenkins server, and several virtual and physical boxes around the globe as slaves. Oh. One more thing about time. It's relative. If you're locked in a windowless room with only one device capable of measuring time, can you be sure that what you see is the time as we know it? Guess what! Our computers have only got a single clock. This leaves me with a conclusion, that a box can have its own relative time, which is not a fine goal for cloud based (=stored across multiple instances) applications. It leads to strange errors, like when a Jenkins master would declare a fresh report from a slave outdated.

We've evolved so much towards shiny applications, that we have forgotten about the basic ideas which led to the evolution of the world-wide web. I remember when I first encountered the Internet, I usually pinged one host which I hoped to be on-line 24/7. This was: time.kfki.hu. A time server! NTP is a protocol designed to synchronize the clocks of computers over a network. The first NTP implementation started around 1980 with an accuracy of only several hundred milliseconds!

So wouldn't it be nice if all boxes would sync from the same provider? But what happens if your crucial information is corrupted by the mistake of that third party service? You wouldn't like that, would you? The great part follows: You can have your Jenkins master server act as a time provider for all connected devices. This could mean user PC's, test boxes and even the inflatable pirate's controller which is hooked on the notification system.

The implementation

The Server

I'll show a Ubuntu Natty configuration, but it should be pretty much the same on every distribution. The first thing you'd want to do is to remove the ntpdate application from all instances. It's not harmful, but can cause minor turbulences.

apt-get remove ntpdate

Then, you need to install the NTP daemon to every instance

apt-get install ntp

On the time server instance, open up and edit /etc/ntp.conf. The existing contents of the file can be left as it is, but you need to add some lines. Under the list of servers, add a reference to the instance itself. This will serve as a back-up if your network should fail.

server 127.127.1.0
fudge 127.127.1.0 stratum 10

You need to enable the synchronization service. Suppose you have a network with 192.168.2.xxx IP addresses, then you'll have to add:

restrict 192.168.2.0 mask 255.255.255.0 nomodify notrap

I prefer to insert it before the line: restrict 127.0.0.1. You need to insert such line for every network you plan to server.

Once you are done, save the file and restart the service:

/etc/init.d/ntp restart

 

The Client

So you have your network a time provider, you need to get your boxes to sync with that. If you carefully followed along, you already have the package ntp installed. Open up /etc/ntp.conf and edit as follows.

Remove every single server line, and add only those which are relevant for your box. If your time server is on 192.168.1.123, than leave a single

server 192.168.1.123

line in the list.

Right before the line: restrict 127.0.0.1 enter two more restrictions. The first will prevent access to the ntp services of this machine, the second will back-up the localhost serving of the time.

restrict default notrust nomodify nopeer
restrict <external ip of the current box>

Of course, you have to replace "<external ip of the current box>" with the external ip of the current box.

Once you are done, save the file and restart the service:

/etc/init.d/ntp restart

Debugging

The ntp software is anything, but verbose. If you want to see what's going on, than stop the service with

/etc/init.d/ntp stop

and run

ntpt -d

Twitter is an awesome platform. A lot of information comes and goes. On the sleepy Friday of the 18th of November, 2011 I read about a global event called "Global Day of Coderetreat". As the software craftsmanship community in Budapest, Hungary is pretty fragmented, I started whining publicly that I'd love to have a Coderetreat here too. That's when Yves Hanoulle gave me the perfect kick (in the butt) in one of his tweets. So thanks again Yves for igniting our little group!

The day has come, and so did the 8am when we started setting up the scene at MyCorporation co-working space. I secretly planned on starting at 9, and it seemed to be a good idea. At 8:30, there were only 4 of people present. At 9:00am, we started the day with 4 pairs watching Corey and his greeting. After a short explanation of the day and the problem we're going to work on, session 1 started. Most of the attendees were familiar with pair programming, TDD and clean code, but never had the chance to try them "in production". The pairs were formed by people knowing each other, this gave them something to hang on to while facing new challenges. The first session was aimed to show the way pair-programming is done in practice, so I issued a role change every 15 minutes. The 45 minutes passed in a blink of an eye. Deleting code at the end caused some muttering, but we were counting on that. It seemed that the in-session role changing slowed the pairs down, so we voted to stop that for the next rounds. Two more people joined us during the first retrospective, and we maintained this number of 5 pairs until the very end of the day.

Session 2 started with the same pairs, as they now got to know the problem domain, and learned a lot from the previous session. Some didn't really stick to TDD at first, but in the second run, they tried. As I was walking around in the room, I could feel the innovation coming from those awesome minds trying to solve the Game of Life and the communication barriers. It was very interesting to hear that even co-workers who have been working together for years now, have never known each other as well as they did after 2 sessions of pair-programming. The participants were mainly from the world of PHP and Java, but thanks to some brave guys, we even had Scala, C# and Ruby popping up. The third session begun with new pairs. People paired up according to what they've heard from each other in the previous retrospectives. Some wanted to try out new languages that the other half knew, some wanted to try working with someone else. We introduced TDD ping-pong. I'm not sure if the language learning goal was a good choice, because we ended up with some pairs where only one participant knew the given syntax. Nevertheless, the problem was solved by the one not knowing the language told the other what he'd want to write. During the coding session, we've joined a global Google hangout, where fellow coderetreaters from Germany, Sweden and Belgium were doing their thing. Right after our third session, we saw that the guys from Ghent are standing in front of the camera and waiting for someone to talk to them. So we did, and had a great time talking to J. B. Rainsberger.
Our catering for the day was sponsored by Mimox, and they did invest in some really tasty food. MyCorporation organized it to be at the restaurant next door.

For the fourth session, we introduced the primitive obsession constraint. I made a mistake in facilitating and did not discover in time that a pair got sucked into an unnecessary generalization problem which held them back from achieving the primary goal. Lesson learned; as a facilitator you need to see their code, not just look at it. Another thing I've learned is that introducing these constraints without the participants understanding why they are good at all, isn't really a viable practice. I will create slides for each for the next time and speak about them a bit prior the first session.
For the final 2 sessions I suggested that everyone should return to their primary language. What they didn't know is that I had an evil plan prepared. (Thanks Jim for the idea).
We voted for the 3 lined methods constraint, and started coding. I encouraged the pairs to commit themselves to any other constraint too if they like, and most of them happily took the challenge. By the end of the 45 minutes, we had one team which completed the task. The evil plan was to ask them not to delete their code, but for the next session swap their product with another pair and they'll be working on the inherited code in the final session. Shame and laughter filled the air all at once, but they did it anyway. I hope that the message was clear that you should always leave code the way that you can even project it on the wall. A very appropriate global constraint was voted for this session: no swearing! The pair which inherited the already finished code received some guidelines from the original creators and started working on making the tests more readable, while the others tried to continue and complete each-others work.

Due to the delayed start, we had to leave the venue as soon as we finished session #6, so the final retrospective and the closing circle was held in a restaurant nearby. I was really happy that everyone stayed up until the end, and everyone wanted to have more coderetreats. Most people said that they will try this exercise at work, and they never thought pair programming was such fun and productive. My remark was that I really enjoyed facilitating, but I'd love to code too. For my greatest surprise, there were several volunteers for the role. We agreed upon rotating the role amongst these individuals, which is - I think - a great contribution to the coderetreat community.

What I learned from the day?

• There's hope for a change of culture
• There are craftsmen in our city
• People need to experience agile development in practice to understand its values
• Hands-on TDD and pair programming trainings should be present in our professional community
• Organizing is fun and I still love it

What will happen?

• We created a Facebook page for the newly forming community
• We will conduct a coderetreat once a month
• We will share knowledge between the participants
• We will try and "infect" others with this experience
• We will do this for fun and knowledge

Thank you all who had made this global day possible! Thank you all who got up early on Saturday and coded all day long. You have proven yourselves dedicated to your profession, there's no other way to put it! :)