Archive for the 'C#' Category

The War On Nulls

As .NET developers, we’ve all seen this exception hundreds of times: “System.NullReferenceException – Object reference not set to an instance of an object”. In .NET, this exception occurs when trying to access a reference variable with a null value. A null value means the variable does not hold a reference to any object on the heap. It is one of the most frustrating and prolific errors that we programmers encounter. But it needn’t be this way! We can prevent this error by following a few simple rules. But first, a little history…

The null reference was invented by Tony Hoare, inventor of QuickSort, one of the world’s most widely used sorting algorithms. In this introduction to his talk at QCon 2009, Tony describes the impact the null reference has had on software:

I call it my billion-dollar mistake. It was the invention of the null reference in 1965. At that time, I was designing the first comprehensive type system for references in an object oriented language (ALGOL W). My goal was to ensure that all use of references should be absolutely safe, with checking performed automatically by the compiler. But I couldn’t resist the temptation to put in a null reference, simply because it was so easy to implement. This has led to innumerable errors, vulnerabilities, and system crashes, which have probably caused a billion dollars of pain and damage in the last forty years. In recent years, a number of program analysers like PREfix and PREfast in Microsoft have been used to check references, and give warnings if there is a risk they may be non-null. More recent programming languages like Spec# have introduced declarations for non-null references. This is the solution, which I rejected in 1965.

So obviously null references have caused quite a lot of damage. But neither Tony or null references are to blame. It’s the careless use of null references that has made them as damaging and prolific as they are.

I can’t think of a single reason why you would need to use null references as part of your system design. Here are some tips for preventing null references in your system.

Never use null references as part of the design

Business logic should not be based around testing for null references. If an object requires an empty state, be explicit about it by creating an empty representation of the object. You can then check if the object is in an empty state by comparing the current instance to an empty instance.

Here is an example of some code that uses a generic interface called ICanBeEmpty to support an empty representation of a Customer object. An extension method called HasValue() allows us to check if an object represents an empty instance.

public class Customer : ICanBeEmpty<Customer>
{
    private int id;
    private string name = string.Empty;
    //...
 
    public bool Equals(Customer other)
    {
        return this.id == other.id;
    }
 
    public static Customer Empty
    {
        get { return new Customer(); }
    }
 
    Customer ICanBeEmpty<Customer>.Empty
    {
        get { return Empty; }
    }
}
 
public interface ICanBeEmpty<T> : IEquatable<T>
{
    T Empty { get; }
}
 
public static class Extensions
{
    public static bool HasValue<T>(this ICanBeEmpty<T> obj)
    {
        return obj.Equals(obj.Empty);
    }
}

Don’t accept null references as parameters

Guard statements are often used to check for null references in methods. If you design your system not to pass nulls, you won’t need guards to check for null in your methods. But when you can’t guarantee input to your public methods, then you need to be defensive about null references.

Don’t return null references

A call to a method or property should never return a null reference. Instead, return an empty representation of an object, or throw an exception if a non-empty value is expected.

Fail fast if a null reference is detected

Design-by-contract technologies, such as Spec#, have declarations that can check for null references at compile time. You can also use an aspect-oriented programming (AOP) solution, such as PostSharp, to create custom attributes that ensures an exception is thrown if any null references are passed in, or returned by a method at runtime. By throwing an exception as soon as a null reference is detected, we can avoid hunting through code to find the source of a null reference.

public class CustomerRepository
{
    [DoesNotReturnNull]
    public Customer GetCustomer(int id)
    {
        //...
        return Customer.Empty;
    }
 
    [DoesNotAcceptNull]
    public void SaveCustomer(Customer customer)
    {
        if (customer.HasValue())
        {
            //...
        }
    }
}

Wrap potential sources of null references

If you are using a third-party service or component where you might receive a null reference, then wrap the call in a method that handles any null references to ensure they don’t leak into the rest of the system.

Always ensure object members are properly instantiated

All object members should be instantiated when an object is created. Be careful with strings in C#, as these are actually reference types. Always set string variables to a default value, such as string.Empty.

Nullable value types are ok

The nullable value types introduced in C# 2.0, such as int? and DateTime?, are better at handling null references as you have to explicitly cast them to a non-null value before accessing them. Be careful with using the Value property on a nullable type without first checking if the variable has a non-null value using the HasValue property. You can use GetValueOrDefault to return a default value if the variable is null.

By limiting the use of null references and not letting them leak into other parts of the system, we can prevent the troublesome NullReferenceException from ruining our day.

User Interface Code And The Open/Closed Principal

The Open/Closed Principal (OCP) is a fundamental object-oriented design principal that states:

“Software entities (classes, modules, functions, etc.) should be open for extension, but closed for modification.”

This means that we should be able to add new behavior to a software entity without altering its code.

Most UI code I have seen, including Model-View-Presenter (MVP) and Model-View-Controller (MVC) implementations, clearly violate the open/closed principal.

When developing UI code, we tend to create presentation classes in terms of views, or sections of the UI. To add new behavior to the UI, we need to modify these presentation classes. This increases the risk of breaking existing functionality.

UI development can be notoriously complex because of the many interactions and state changes that can occur. To manage this complexity we should ensure that our classes have only one reason to change.

Instead of grouping presentation classes into visible sections of the UI, maybe we should be creating a series of discrete presentation behaviors that react to events raised from the view.

These behaviors can be added and removed without affecting other behaviors on the page.

Behavior-Driven Development (BDD) advocates creating tests based on individual behaviors, so why not create our presentation classes in the same way? The tests then correspond directly to a single discrete unit, rather than a behavior that occurs within a larger structure, e.g. a presenter class.

Each behavior has a name that describes the behavior it represents. Using a descriptive name provides instant documentation of the UI behavior. It should be easy for someone to understand what the UI does simply by looking at the behavior names. If a problem occurs, it should be easy to identify and isolate the affected behavior.

Implementing Presentation Behaviors

I have created a simple music finder application that demonstrates an implementation of presentation behaviors.

Download the sample code here.

The sample application contains a single page, represented by an IFindMusicView interface. The behaviors respond to events raised by this View and update the View accordingly.

A typical behavior can be defined as:

Given… a particular scenario

When… an event occurs

And… all conditions have been met

Then… do something

And… do something else

Each behavior is implemented as a class that derives from a base class with an IBehavior interface. This interface contains two methods: When() and Then().

The When() method contains code that registers the behavior with a certain event on the page. The Then() method contains the code that responds to the event if all conditions have been met. The “Given” aspect is implemented by the class constructor, which takes the view and any associated dependencies.

   1: public class when_song_list_is_empty_disable_select_button : FindMusicBehavior
   2: {
   3:     public when_song_list_is_empty_disable_select_button(IFindMusicView view)
   4:         : base(view)
   5:     {
   6:     }
   7: 
   8:     public override void When()
   9:     {
  10:         View.AfterSongsLoaded += (sender, e) => Then();
  11:     }
  12: 
  13:     public override void Then()
  14:     {
  15:         View.IsSelectButtonEnabled = View.SongsList.Count > 0;
  16:     }
  17: }

 

The behaviors are instantiated by the View, but an Inversion Of Control container could be used to register the behaviors at run-time. The View then wouldn’t need to know anything about the behaviors that are implemented for it. We could then drop-in and drop-out behaviors without needing to change the existing code.

Further Considerations

Although this is an interesting concept, I am yet to implement it on a large-scale application. There are several areas I need to investigate further, such as:

  1. Can one behavior extend the functionality of another behavior?
  2. Can parallel behaviors remain independant from one another?
  3. How does this work with MVC frameworks? Are the behaviors triggered by actions?
  4. How well does this scale?

Feedback

Does this concept make sense? Does it sound practical? Could it potentially solve some of the issues we face with developing complex UI code? Any feedback would be greatly appreciated.

Enjoy!

Legacy Code Can Be Fun!

No, I’m not being ironic; I’m actually enjoying working with legacy code!

I have recently inherited a large, legacy code-base. There are hardly any tests and it’s full of dependencies between components. The application is critical to the day-to-day operation of the business and requires ongoing feature additions, support and maintenance.

By “legacy code”, I am referring to the definition as ”code without tests and in need of refactoring”. This is described by Michael Feathers in his excellent book Working Effectively with Legacy Code. This post uses techniques from this book. If you are working with a legacy code base, buy this book! It will make your life much easier.

As developers, most of us have had to endure the frustration of working with code that is unclear and untested (if you haven’t, you don’t know what you’re missing ;-) . It can be painful having to make changes without knowing the affect the change could have on the application. We usually apply Change-and-Pray Development, that often results in hard-to-find problems slipping through to production. Over time, constant tacking-on of changes further degrades the quality of the code-base and can result in a Big Ball of Mud.

But fear not, there is a way to make changes to legacy code much easier, and – dare I say, fun! Continuous refactoring and testing can help to turn a bug-ridden, untested and tangled application into a slick, streamlined code-base that just gets better over time.

Introducing Changes To Legacy Code

One of the most difficult things about working with legacy code is introducing changes. Without tests to back us up, it is hard to know if a change has worked and hasn’t broken anything unexpectedly.

For this demonstration, I have created a sample class that reflects some common issues when introducing changes to legacy code. The code is intentionally simple to clarify the refactoring. Real-world legacy code is not usually so simple, but these techniques still apply.

Here we have a DocumentManager class.

public class DocumentManager
{
    public static Document GetDocument(int documentId)
    {
        DocumentDAL documentDAL = new DocumentDAL();
        Document document = documentDAL.Get(documentId);
        if (document != null && !File.Exists(document.Path))
        {
            throw new FileNotFoundException("Document file was not found");
        }
        return document;
    }
    public static List<Document> GetAllDocuments()
    {
        DocumentDAL documentDAL = new DocumentDAL();
        List<Document> documents = documentDAL.List();
        foreach (Document doc in documents)
        {
            if (doc != null && !File.Exists(doc.Path))
            {
                throw new FileNotFoundException("Document file was not found");
            }
        }
        return documents;
    }
    public static void SaveDocument(Document document)
    {
        File.WriteAllText(document.Path, document.Contents);
        DocumentDAL documentDAL = new DocumentDAL();
        documentDAL.Save(document);
    }
    public static void DeleteDocument(int documentId)
    {
        Document document = GetDocument(documentId);
        DocumentDAL documentDAL = new DocumentDAL();
        documentDAL.Delete(document);
        File.Delete(document.Path);
    }
    public static string GetFileName(Document document)
    {
        return Path.GetFileName(document.Path);
    }
}

The purpose of this class is to manage the persistence of documents to a database and file system.

I have come across this type of class many times; it is mostly composed of static methods that call an underlying data access layer. The class is doesn’t have any unit tests and contains direct calls to the file system and a data access class.

My goal is to add some validation code into the Save method of the DocumentManager class and to create unit tests to verify the new functionality.

Firstly, I’m going to create a test harness so I know my changes won’t adversely affect the existing functionality.

Creating a Test Harness

Creating a test harness allows us to make changes while ensuring the code still functions as expected.

Let’s create a new test fixture called DocumentManagerTests.

[TestFixture]
public class DocumentManagerTests
{
}

Instantiating the Class

Now that we have a test fixture in place, let’s create a test that instantiates the class we want to test.

[Test]
public void Can_Create_Instance_Of_DocumentManager()
{
    DocumentManager documentManager = new DocumentManager();
    Assert.That(documentManager, Is.InstanceOfType(typeof(DocumentManager)), "Should create an instance of DocumentManager");
}

Our code doesn’t compile, as DocumentManager is a static class and can’t be instantiated. We could just test the members of the static class, but we need an instance so any dependencies can be passed into the constructor. We will come to this shortly, but for now, let’s remove the static keyword from the class declaration so we can instantiate the class.

public class DocumentManager
{ ...

After making the class non-static our test compiles and passes! The assertion isn’t very meaningful, but at least we know we can instantiate the class without an error.

Breaking Dependencies

The reason legacy code seems difficult to test, is that it often contains internal calls to other components or sub-systems. This makes it difficult to isolate a piece of code for testing.

So why break dependencies and not just test everything? Well, firstly you end up testing the integration between components, rather than a single unit of code. Calling external components can cause our tests to run slowly, or to fail due to an problem with a sub-system.

To test the Save method in isolation, we need to extract calls to the DocumentDAL and File classes.

These are fairly safe structural refactorings, so we should be able to make the changes without affecting functionality.

To remove the dependency on the DocumentDAL class in the Save method, we can use a technique called Dependency Injection.

Dependency Injection

Dependency Injection involves passing a reference to the dependent object into the class constructor. Here, we can use an Extract Interface refactoring to decouple the DocumentDAL implementation from its interface. Using an interface allows us to provide a mock implementation for testing.

public class DocumentDAL : IDocumentDAL
{ ...

We can now “inject” the dependency into the DocumentManager constructor.

public class DocumentManager
{
    private readonly IDocumentDAL documentDAL;
    public DocumentManager(IDocumentDAL documentDAL)
    {
        this.documentDAL = documentDAL;
    } ...

Our test doesn’t compile, as we need to pass an implementation of IDocumentDAL into the constructor. I am using Rhino Mocks to mock an IDocumentDAL implementation.

[Test]
public void Can_Create_Instance_Of_DocumentManager()
{
    MockRepository mocks = new MockRepository();
    IDocumentDAL documentDAL = mocks.CreateMock<IDocumentDAL>();
    DocumentManager documentManager = new DocumentManager(documentDAL);
    Assert.That(documentManager, Is.InstanceOfType(typeof(DocumentManager)), "Should create an instance of DocumentManager");
}

To remove the Save method’s dependency on the file system, we can use another dependency-breaking technique called Subclass and Override Method.

Subclass and Override Method

First, we use Extract Method to move call to the file system into a separate virtual method.

protected virtual void WriteFile(Document document)
{
    File.WriteAllText(document.Path, document.Contents);
} 

We need to change the Save method to call the new WriteFile method. Because WriteFile is an instance method, we need to make the Save method non-static. Note that any code that calls this method will also need to be changed to use the non-static method.


public void SaveDocument(Document document)
{
    WriteFile(document);

    DocumentDAL documentDAL = new DocumentDAL();

    documentDAL.Save(document);
}

Now we can create a subclass of DocumentManager called TestingDocumentManager and override the WriteFile method. This allows us to stub-out the calls to the file system by the Save method for use in our tests.

public class TestingDocumentManager : DocumentManager
{
    public TestingDocumentManager(IDocumentDAL documentDAL)
        : base(documentDAL)
    {
    }
    protected override void WriteFile(Document document)
    {
    }
}

Next, we update our test to use the TestDocumentManager with the overridden WriteFile method.

[Test]
public void Can_Create_Instance_Of_DocumentManager()
{
    MockRepository mocks = new MockRepository();
    IDocumentDAL documentDAL = mocks.CreateMock<IDocumentDAL>();
    DocumentManager documentManager = new TestingDocumentManager(documentDAL);
    Assert.That(documentManager, Is.InstanceOfType(typeof(DocumentManager)), "Should create an instance of DocumentManager");
}

Ideally, we could remove the dependency on the file system altogether by injecting an interfaced wrapper for the File class, but that would distract us from the focus of our test. Subclass and Override Method is a very useful technique for quickly getting legacy code under test. We can refactor the code at a later time if required.

Testing the Method

To test the Save method we need to remove the instantiation of DocumentDAL and use the field that was set in the constructor.

 

public void SaveDocument(Document document)
{
    WriteFile(document);

    documentDAL.Save(document);
}

Now we can test the functionality of the Save method.

[Test]
public void Can_Save_Document()
{
    MockRepository mocks = new MockRepository();
    IDocumentDAL documentDAL = mocks.CreateMock<IDocumentDAL>();
    DocumentManager documentManager = new TestingDocumentManager(documentDAL);
    Document document = TestHelper.GetDocument();
    Expect.Call(() => documentDAL.Save(document));

    mocks.ReplayAll();
    documentManager.SaveDocument(document);
    mocks.VerifyAll();
}

Running this test shows us that the behaviour of Save is working as expected. It provides a safety net for making further changes. If a change affects the core behaviour of the Save method, this test should fail.

Making the Changes

Now that we have a test harness in place, we can implement our new changes using Test Driven Development (TDD).

First, we write a test to verify the changes we want to make.

[Test]
public void Cannot_Save_Invalid_Document()
{
    MockRepository mocks = new MockRepository();
    IDocumentDAL documentDAL = mocks.CreateMock<IDocumentDAL>();
    DocumentManager documentManager = new TestingDocumentManager(documentDAL);
    Document document = TestHelper.GetInvalidDocument();
    try
    {
        documentManager.SaveDocument(document);
        Assert.Fail("Should throw an exception for invalid document");
    }
    catch(Exception ex)
    {
        Assert.That(ex, Is.InstanceOfType(typeof(ValidationException)), "Should throw a ValidationException");
    }
}

The code compiles and the test is failing as expected. We now need to implement the code to make the test pass.

public void SaveDocument(Document document)
{
    if(!document.IsValid)
    {
        throw new ValidationException();
    }
    WriteFile(document);
    documentDAL.Save(document);
}

The test passes! We now run all the tests to ensure our change hasn’t broken the existing functionality.

Refactoring

With our tests passing, we can now clean things up a bit. We are free to make changes on any code that has been tested.

We could also take the time to further improve the design of the DocumentManager class, such as separating the File Access responsibilities into another class. The amount of refactoring you do depends on how much time you have. If you can spend some time tidying things up around the code you have changed, the better the code will be for the next person who needs to change it. You don’t have to do everything at once. Generally, code that is changed often should become continuously refactored and improved over time.

Summary

This is just one of many techniques you can use to refactor and test legacy code. Many more examples can be found in Working Effectively with Legacy Code. This has been a very simple example, but I have used these same techniques to make big changes to some very dodgy pieces of code.

Refactoring and testing legacy code can help to improve a legacy code-base over time. It feels like you’re writing new code, rather than just tacking onto existing code. It is also a great feeling walking away from implementing a change, knowing that the surrounding code is now cleaner and more robust than before.

Implementing The Circuit Breaker Pattern In C# – Part 2

In my previous post, I discussed an implementation of The Circuit Breaker Pattern as described in Michael T. Nygard’s book, Release It! Design and Deploy Production-Ready Software. In this post, I will talk about several additions and improvements I have made to the initial implementation.

Service Level

The Circuit Breaker Pattern contains a failure count that keeps track of the number of consecutive exceptions thrown by an operation. When the failure count reaches a threshold, the circuit breaker trips. If an operation succeeds before the failure threshold is reached, the failure count is reset to zero. This works well if the service outage causes multiple consecutive failures, but if the threshold is set to 100 and the service fails 99 times, then one operation succeeds, the failure count is reset to 0, even though there is obviously a problem with the service that should be handled.

To deal with intermittent service failures, I have implemented a “service level” calculation. This indicates the ratio of successful operations to failed operations, expressed as a percentage. For example, if the circuit breaker has a threshold of 100 and an operation fails 50 times, then the current service level is 50%. If the service recovers and 25 operations succeed, then the service level will be 75%. The circuit breaker will not completely reset after a single successful operation. Each successful operation increments the service level, while each failed operation decrements the service level. Once the service level reaches 0%, i.e. the ratio of failed operations over successful ones have reached the threshold, the circuit trips.

A ServiceLevelChanged event allows the client application to be notified of service level changes. This could be used for monitoring performance, or for tracking service levels against a Service Level Agreement (SLA).

The threshold value determines the circuit breaker’s resistance to failures. If a client makes a lot of calls to a service, a higher threshold will allow it more time to recover from failures before tripping. If the client makes fewer calls, but the calls are expensive to the service, a lower threshold will allow the circuit breaker to trip more easily.

Ignored Exception Types

Sometimes a service will throw an exception as part of the service logic. We might not want these exceptions to affect the circuit breaker service level. I have added an IgnoredExceptionTypes property, which holds a list of exception types for the circuit breaker to ignore. If an operation throws one of these exceptions, the exception is thrown back to the caller and is not logged as a failure.

 CircuitBreaker cb = new CircuitBreaker();
 cb.IgnoredExceptionTypes.Add(typeof(AuthorizationException));

Invoker Exceptions

If the operation invoker throws an exception that was not caused by the operation itself, then the exception is thrown back to the caller and is not logged as a failure.

Threading

As mentioned in a comment by Søren on my last post, it is likely a circuit breaker would be used in a multi-threaded environment, therefore it should be able to function property when multiple threads are executing operations.

The failure count is now updated atomically using the System.Threading.Interlocked.Increment and System.Threading.Interlocked.Decrement methods. This ensures the failure count variable is locked while being modified by a thread. Other threads wanting to update the failure count must wait until it is released.

While this does not guarantee the circuit breaker is completely thread-safe, it does prevent problems with multiple threads executing operations and tracking failures. I have to confess I’m not an expert at multi-threaded application designs, so if anyone has any further suggestions on how to make the circuit breaker more thread-safe, I would love to hear them!

For more information implementing threading, see the .NET Framework Threading Design Guidelines.

Download

Download the circuit breaker code and unit tests (VS 2008).

I hope you find these enhancements helpful. Does providing a service level make sense? How can I improve multi-threading support? If you have any comments or suggestions, please let me know your thoughts.

Implementing The Circuit Breaker Pattern In C#

When developing enterprise-level applications, we often need to call external services and resources. These services could be a network location, database server, or web service. Whenever we call a service, there is a chance that a problem with the network or the end-service itself could cause a service failure. One method of attempting to overcome a service failure is to queue requests and retry periodically. This allows us to continue processing requests until the service becomes available again. However, if a network or service is experiencing problems, hammering it with retry attempts will not help the service to recover, especially if it is under increased load. Such a pounding can cause even more damage and interruption to services. If we know there could potentially be a problem with a service, we can help take some of the strain by implementing a Circuit Breaker pattern on the client application.

Circuit breakers in our home prevent a surge of current from damaging appliances or overheating the wiring. They work by allowing a certain level of current to enter the system. If the current exceeds the threshold, the circuit opens, stopping the current from flowing and preventing further damage. Once the problem has been fixed, the circuit breaker can be reset which closes the circuit and allows electricity to flow again. The Circuit Breaker patten uses the same concept by stopping requests to a resource if the number of failures exceed a certain threshold.

The Circuit Breaker pattern is described in Michael T. Nygard’s book, Release It! Design and Deploy Production-Ready Software. The pattern has three operational states: closed, open and half-open.

In the “closed” state, operations are executed as usual. If an operation throws an exception, the failure count is incremented and an OperationFailedException is thrown. If the failure count exceeds the threshold, the circuit breaker trips into the “open” state. If a call succeeds before the threshold is reached, the failure count is reset.

In the “open” state, all calls to the operation will fail immediately and throw an OpenCircuitException. A timeout is started when the circuit breaker trips. Once the timeout is reached, the circuit breaker enters a “half-open” state.

In the “half-open” state, the circuit breaker allows one operation to execute. If this operation fails, the circuit breaker re-enters the “open” state and the timeout is reset. If the operation succeeds, the circuit breaker enters the “closed” state and the process starts over.

You can download the circuit breaker code and tests here. If you have any comments or suggestions, I would love to hear them!

Update: I have posted a new article that contains a number of additions and improvements to the circuit breaker code.

For more information on this pattern and many other ways to improve software stability, capacity and operational ability, I highly recommend the book Release It! Design and Deploy Production-Ready Software by Michael T. Nygard.

Circuit breaker class diagram