University of Nottingham Agile Software Development Talk

A work colleague and I recently attended the University of Nottingham Jubiliee Campus, to present a talk to second year Computer Science students that were taking part in the group project module. The talk was entitled 'What's it like to work in an agile software development team?'.

The talk focused on how the Esendex development team tackle software development, and we presented some ideas for the planning and development aspects of their own group projects. I did the same module when I was an undergraduate at Nottingham and the module stands out for me as the closest thing you experience at university to working as part of a real software team in industry.

It was tough selecting exactly what to put in the talk, 'agile development' is a broad subject and means different things to different people. We could have easily talked about the Agile Manifesto for the 50 minutes we had, but in the end I think we did well cherry picking the parts that would interest the students and be relevant to their group projects. We also thought it important to run through a demo of some of the concepts we talked about and this part was especially received well.

We carried out a 'mini retro' at the end of the talk, asking the students to leave some feedback on their way out. We split the board in two; 'What went well?' and 'What could have gone better?', providing the students with some post-its and pens. We were very pleased with the feedback, as you can see the feedback was 90% positive :)

You can't see the comments on that pic unfortunately, but here's some I've cherry picked...

Content truly excellent! All highly relevant & interesting, right level of detail.

First live demo from a guest lecturer :)

Great to see a live demo, putting theory into context.

Actually related to the group project.

Students might have struggled following the detail of some examples. Slow down a little. Font size & colour, for demo.

Here's the slides if you're interested, looking forward to getting invited back again next year hopefully.

One Git Clone to Rule Them All

git clone --recursive REPOSITORY_URL -o REPOSITORY_ALIAS

• --recursive - Gets the submodules too
• --o - Sets an alias for remote repo rather than defaulting to origin (useful when working in teams)

Calculating MySQL Database Size

A neat script for retrieving database size in MB. I found it useful for checking on Amazon RDS instances.

SELECT table_schema "Database", SUM( data_length + index_length) / 1024 / 1024 "Size (MB)" 
FROM information_schema.TABLES 
GROUP BY table_schema ;

Should I Dispose of my LINQ DataContext?

Most developers construct a DataContext within a using statement to make sure that resources are cleaned up when leaving the block.

using( var dataContext = new LinqDataContext() )
{
   // Perform some data operations
   dataContext.Insert(obj)

} // Dispose of the DataContext and release resources

However it's not needed, the .Net garbage collector does that for you and the need to dispose of the DataContext is made less of critical requirement knowing that LINQ DataContexts do not keep expensive database connections open (something that surprised me) like some other ADO.Net objects.

To summarise, it isn't the end of the world and you aren't a bad developer if you don't dispose of your DataContext objects. I do however think it's good practice to work with IDisposable classes in this way; plus it releases those resources slightly quicker than just leaving it to the garbage collector.

Powershell Sleep

Zzzzzz....

Start-Sleep -s 5 # 5 seconds
Start-Sleep -m 5000 # 5000 milliseconds

How Powershell Behaves with Errors in the Script

PowerShell has two kinds of errors; terminating errors and non-terminating errors

Terminating errors are the errors that can stop command execution cold. Non-terminating errors provide an additional challenge, as you need to be notified of failed operations and continue with pipeline operations. Powershell by default tries to continue when non-terminating errors occur. Something I don't want to happen when I call Powershell with my CI server to release software. If there are errors I want the script to stop and tell me what is wrong - not the script to continue and report that everything went OK.

$ErrorActionPreference is a variable that describes how PowerShell will treat non-terminating errors. $ErrorActionPreference has four self-explanatory options:

• Continue (the default)
• SilentlyContinue
• Inquire
• Stop

To exit your script with an error code (if an error occurred), you'll need to exit using the $LastExitCode variable. This variable returns a number that represents the exitcode of the last script or application, needed for your CI server to know if the script ran successfully. Here's a template for what I use with the Jenkins CI server...

$ErrorActionPreference = "Stop"

# Do something...

exit $LastExitCode

Anything else?

Well, yeah... $error is an array of all the errors that have occurred in the current session up to the number specified in $MaximumErrorCount. Also, $? is a boolean value that is $true if the previous operation succeeded and $false if it did not.

CQRS Commands (aka the write side)

Commands are created by a client, put onto a command bus for the server to process in it's domain. Commands are simple messages/requests to mutate state. It's not about editing data, but about performing tasks. Commands are always in imperative tense (present), asking the system to do something. As they are in the imperative tense, they are essentially questions and as the language suggests, the server has the right to refuse.

Command handlers handle these commands and are simple facades/coordinators over the domain below and methods should be thin; like MVC controller actions. Behaviours should be pushed down to the domain below and not exist in the handlers themselves. As each command changes state, it also typically results in events being pushed onto an event bus for recording and processing by event handlers.

Commands and events are very similar in design, only differing by their intentions. Events are described as verbs in the past tense. As the language suggests, it has already occurred, so no point in throwing exceptions. However, you can come up with a counter event. Event handlers typically insert data into denormalised database tables in the query database (the read side) but can perform a range of other tasks, such as send an email.

7 Ways to optimise Jenkins/Hudson

We've been having some 'difficulties' with Jenkins the past couple of weeks. This whitepaper from the guy that wrote most of the Jenkins core code looks like it could be interesting... 7 Ways to optimise Jenkins/Hudson.

CQRS Queries (aka the read side)

Commands mutate state and trigger events which describe the changes which have occurred inside the write store. It is the responsibility of the read store to subscribe to these change events and update their read store appropriately (using event handlers).

All application queries from end users are made to the read store, which can be anything from a relational database to XML files. This data store is simply a thin read layer that's heavily optimised for reads. Each application's data view (think view models in MVC), typically maps to denormalised data tables rather than using joins (which slow down queries and prohibit scaling without headaches).

That's it, in a nutshell! The read side is a lot simpler than the write side of CQRS and something developers are more accustomed to. Working with denormalised data rather than third normal form makes queries quick and very easy to maintain; something for the junior devs :)

Consistency is over rated!!!

In a CQRS system, making a write does not immediately change what is read.

This is why CQRS systems are described as offering eventual consistency. CQRS evangelists question the importance of real-time consistency. They argue that by the time end users see data it is already stale. FOr example, requesting a data object via a web app results in a series of delays before the data can be displayed to the end user and hence is to a degree stale i.e. retrieving data from the data store, populating a DTO, returning data in a web response, page rendering in a browser.

I think I agree and if this is an issue for you, I suggest you question the importance of real-time consistency for your business. I'd argue that there is a time period/delta that for which in-consistencies can be tollerated and hidden from the end users. However, I do think that efforts should be made to keep this to a minimum and perhaps this delta should be actively monitored, especially for systems where it matters (e.g. a message system with high SLAs).

Command Query Responsibility Segregation (CQRS)

We've begun working on a new project at Esendex, one that involves making use of (amongst other things) the fashionable CQRS pattern. I've been asked to head up the project and over the past week I've read a number of articles and looked at some example projects. So far, I like what I'm seeing; an architecture for developing scalable, extensible and maintainable applications. It looks promising and if proven successful could be an approach we adopt at Esendex to pre-empt any future scalability problems that could result from our continued growth.

What is CQRS?

CQRS is an evolution of command-query separation by Bertrand Meyer. It works using Domain Driven Design where the behaviour sits in the domain objects; utilising DDD means the language used is important.

As the name suggests, the CQRS pattern separates out the objects for commands (for changing application state) and objects for queries (that expose the application state). This is quite a contrast to the more typical CRUD patterns that most developers (including me!) are comfortable with. Systems typically written as one, are split into a read side and a write side.

High level CQRS diagram ('borrowed' from another blog)

Confused?!

Expect follow up posts soon, in the mean time, here's some useful links:

• Greg Young (aka CQRS God) lecture video (6 hours+ long) and accompanying notes
• Nice example CQRS implementation in C# (by... Greg Young)

Visual Studio Breaking on Exceptions for MSTests

At home I don't have ReSharper installed (shock!) and I use MSTests (wtf?!) for some small old projects. Annoyingly VS would break on exceptions when running tests; given that these tests rely on exceptions being thrown it becomes a tad annoying.

Turns out I'm a massive idiot and I was just running the tests with the wrong keyboard shortcut - doh! Here are the shortcuts I was using and the ones I am now using:

• Ctrl r + Ctrl t => Run tests in current context (debug mode)
• Ctrl r + t => Run tests in current context
• Ctrl r + Ctrl a => Run all tests in solution (debug mode)
• Ctrl r + a => Run all tests in solution

T-SQL Simple Search & Replace on Table Content

This was thrown together in an attempt to save some time having to edit a hefty amount of text content. Simple amend the below script with your table, field, text you want replaced, what you want to replace it with and hey ho - you've hacked your content!

DECLARE @TABLE  VARCHAR(30) 
DECLARE @FIELD VARCHAR(30) 
DECLARE @WHERE VARCHAR(100) 
DECLARE @FIND VARCHAR(100) 
DECLARE @REPLACEWITH  VARCHAR(100) 
DECLARE @query VARCHAR(8000) 

SET @TABLE = 'MyTableName' 
SET @FIELD= 'MyField'
SET @FIND =  'text i want replacing' 
SET @REPLACEWITH = 'new text' 

BEGIN     
SET @query  = 
'UPDATE ' +  @TABLE +      
' SET ' +  @FIELD + 
'= REPLACE(CONVERT(varchar(8000),'     + 
@FIELD + '),''' +  @FIND + ''',''' + 
@REPLACEWITH  +''')'   

EXECUTE (@query) END 
GO

XP2011 International Agile Conference (Madrid)

I'm lucky I work for an employer that recognises the value in sending the development team members to conferences ever year. Since I started at Esendex, I've been to a number of conferences over the years, most of them focused on the coding aspect of development; new frameworks, IDEs, tools etc. This year I was hungry for something more focused on the management practices and processes in agile software development, something that I felt I need to add to my skillset as an aging developer (I may not look it, but I'm 27!).

The team at work had been discussing conferences and XP2011 came up, when I investigated the conference program it sounded promising; just what I'd been looking for infact. Add to the mix that it was in beautiful Madrid and I was sold.

Conference format

The conference is designed to bring agile researchers and practitioners together. The first and last days are dedicated workshops and tutorials for discussing hot topics in the world of agile. The 2 days between are the core conference days and the programme offers 5 parallel sessions; 3 focused sessions plus invited talks and reports from industry. They also offer time for OpenSpace on the Thursday, for attendees to create their own sessions/discussions.

What was it like?

Good, very good. I wasn't sure what to expect to be honest, but the venue and food was great, people were friendly and everything seemed well organised. The session and workshop speakers were generally very good and knew their subject material well. You'd see the speakers attending lots of other sessions and discussions as attendees, they were approachable too, giving the conference that bit more of a community feel to it. Everyone seemed friendly, approachable and open to discussing their own agile perspectives and challenges they've faced; which I particularly found value in. One of the most enjoyable aspects of the conference were the sessions with lightning talks (10-20 min condensed talks). The speakers that I found particularly interesting were Ken Power, Esther Derby, Emily Bache, David Anderson, Mike Hill and Liz Keogh. Expect to see some posts related to the more interesting topics I attended at the conference over the coming weeks.

The social events were also well organised and there were plenty of complimentary drinks and food for everyone. The Wednesday night geek disco was a particular highlight :)

Any complaints? Well, a few of the sessions based on research papers were presented by the researchers themselves, who seemed a little inexperienced at presenting and as a result the session wasn't enjoyable. You can have the most interesting material in the world but if you don't deliver it well, you don't engage me or the rest of the attendees and it's a bit of a waste of time.

Conclusion

I enjoyed it and I'd recommend it to others looking for a conference of this type. The attendees are  passionate about the subject matter and it's infectious. If the subject material for XP2012 (Sweden) grabs me, I may well return to satisfy my agile appetite.

Getting to grips with Git

Git is a world apart from Team Foundaation Server and people seem to be frightened at a first attempts with it, as it can seem complex. Stick with it though and you'll soon see past that and how simple it can be. Git is a distributed source control system, so it allows you to work disconnected from the main repository and still commit. This 'commit' is to your local repository. You should commit often. Once happy that all your tests pass should you push to the remote repository.

1. clone - get the source code from your remote git repository
2. Make your changes
3. commit - commits your changes LOCALLY (not the remote git repository)
4. push - commits your changes to the REMOTE repository

For me, working disconnected is the number one advantage for Git over TFS. Number 2 is the ability to create branches. Essentially just a seperate work stream you can use for working on seperate features. Once happy, you then merge your branch into the master branch (which is the main repository).

1. clone - get the source code from the remote git repository
2. branch branch-name - create a branch branch-name
3. Make your changes on your branch
4. commit - commits your changes LOCALLY (not the remote git repository)
5. push origin branch-name - commits your changes to the REMOTE repository
6. checkout master - switch to the master branch
7. merge branch-name - merge the feature branch into the master
8. commit and push !!!

I'd strongly recommend that you use the git bash until you get to grips with the git comands, then you can make use of the helper tools.
At the moment I'm still working with the bash but I occasionally use TortoiseGit for windows explorer integration and Git Source Control Provider for Visual Studio GUI integration.

jQuery Document Ready Shorthand

You should always start your jQuery using the ready() function. The handler passed to .ready() function is guaranteed to be executed after the DOM is ready.

There are a few ways to start your jQuery function, but all are equivalent:

$(document).ready(function() {
  // Do some magic
});

$().ready(function() {
  // A bit quicker to write
});

$(function() {
  // Nice shorthand that I prefer to use
});

jQuery Sytax Error - Unrecognized Expression

A recent upgrade to jQuery 1.5 from 1.2 highlighted a problem with jQuery expressions which resulted in a number of javascript errors...

Syntax error, unrecognized expression: [@attributename=value]' when calling ...

Some digging highlighted that attribute selectors, since jQuery 1.3 and above no longer use the @ syntax. This actually had been deprecated since version 1.1.4, but only from version 1.3 is it no longer supported.

Old way (pre jQuery 1.3)...

$("#holder input[@value=contacts2]");

New way (post jQuery 1.3)...

$("#holder input[value=contacts2]")

Git First Impressions

So far, so good :) Took me just under half a day to set up a git repository, import the existing project source and update our CI server. GitHub works well for creating your repositories and walks you through the process. It's very quick, especially when you comapare it to TFS - it's knocked 40% off our build time for a fairly hefty web app. It's not just the speed thats impressive though, having a complete history and full revision tracking capabilities, not dependent on network access or a central server is something as a TFS slave user, I've been longing for. I still think there's lots to get my head round, which will make things interesting next week when I have some real work to do on this project; so watch this space.

Here are some useful resources if you are just getting to grips with Git:

• Git Homepage
• GitHub - repository managment
• Git for the lazy - nice and concise
• Handy help docs from GitHub - including generating SSH keys so you can upload to GitHub
• A useful Git cheat sheet

Git Bandwagon

I've being feeling the peer pressure of late to try out Git as source code repository for our the app team's projects. Today I'm going to give it a try with one of our bigger web apps. I'm hoping that it won't take me too long to get my head round the way Git works and that it results in the project build time dropping on our CI server. Well it had better - time is money!

Git is an extremely fast, efficient, distributed version control system ideal for the collaborative development of software.

Removing XmlDocument Whitespace C#

I've recently been working on matching certain API calls with XML data pulled from an XML file for testing purposes. I noticed there was a large amount of white space left in the XML when pulled from the resourced XML file; which is something I didn't want.

I thought setting the XmlDocument.PreserveWhitespace property to false would remove this for me, but it just seems to remove the preceding and trailing white space; making it similar to the string.Trim() method. I needed to use something akin to string.Replace() (this replaces a specific substring with another substring), but more powerful. Here comes the Regex.Replace function to the rescue, which is a bit like string.Replace() on steroids! Regex.Replace() allows replacement of text using regular expressions, something which can make make complex replacements a piece of cake.

Here is the code to replace white space in XML or any XML dialect (such as HTML - or XHTML):

// Remove inner Xml whitespace
Regex regex = new Regex(@">\s*<");
string cleanedXml = regex.Replace(dirtyXml, "><");

Team Foundation Server Admin for Dummies

Occasionally I'll need to administer my team's users and projects in TFS, which usually results in me having to annoy Jonathan our TFS guru at Esendex and beg him to help me. It's not just me that finds it difficult to administer TFS users and projects though; Microsoft don't make it easy for us developers at all.

Luckily, some nice people over at Attrice Corp have made it easy. Introducting Team Foundation Server Sidekicks, a GUI tool for TFS admin tasks.

You're welcome :)

Memory stream is not expandable

An upgrade to .Net 4 for an MVC app highlighted an issue with the MemoryStream class we were using in the framework. Seems that unless you use the default constructor, the stream cannot be expandable. If the stream expands you'll get a lovely System.NotSupportedException: Memory stream is not expandable error thrown at you like a ninja star.

var ninjaStarStream = new MemoryStream(buffer); // Potential error
var fluffyStream = new MemoryStream(); // Stream is expandable :)

Consider yourself warned!

Monodroid

Now this looks interesting... .Net development for Android using Monodroid

I signed up to be put in the queue to be part of testing (so I can get the download), but some how stumbled upon the hidden download when browsing the site! #win

P.S. Does anyone want to but me a Nexus S?

Firefox is King!

Internet Explorer loses its place as Europe's number one browser for the first time

...perhaps this will give Microsoft a kick up the arse!? Perhaps not.

IE's market share dropped to around 37.52 percent in December 2010 while Firefox's market share remained fairly consistent at 38.11 percent :)