Monday, May 16, 2011

Implementing read-only mode in ASP.NET Using Inversion of Control, Reactive Programming, & the Visitor Pattern

In this post I wanted to focus primarily on Inversion of Control and a little on Reactive Programming (Rx) since I think the two go hand in hand. I really like these functional style approaches to programming. IoC is very reactive in nature, because the relying party ultimately ends up reacting to the input you created for it (details to follow). These 2 concepts are very mundane in nature, but serve as the core building blocks of some of our most widely used functions and libraries. If my suggestions leave you with any doubt or skepticism, feel free to pay one of my favorite guys a visit.

You're sure to have come across IoC if you're into Domain Driven Design (DDD) like myself. I'd suggest picking up a copy of the book with the bridge. Don't worry. You won't have any problem finding it...trust me. It's been around for 5 years by now. Yea...it's that good.

Inversion of Control/Rx examples

  • how NUnit runs your tests for you
  • how ASP.NET MVC invokes your controller for you, conveniently populating the arguments you requested like route values, models, etc.
  • how jQuery abstracts away AJAX for you and allows you to be ignorant as to how the response was generated and where it came from
  • how WCF invokes your service/host for you
  • how a DI Container creates concrete instance for you as opposed to you doing it yourself
  • how Windows Forms and ASP.NET listen for events for you and allow you to react to them
  • how the Windows and the CLR unite to provide your program with command line arguments via the string[] args parameter to your command line program's Main method. You react to the arguments, but not  once have you ever had to worry about where they came from or how they got there. It's all handled by the runtime.


Let's get into some code shall we?

Recursion....Important but BOOORRRIIIINNNGGGG!!

public static class AdubbExtensions {
    public static void ToReadOnly(this Page page) {
        page.ToReadOnly(new DefaultReadOnlyModeControlVisitor());
    }

    public static void ToReadOnly(this Page page, IReadOnlyControlVisitor visitor) {
        Action<IEnumerable<Control>> recursor = null;

        recursor = controls => {
            foreach (var control in controls) {
                if (control is BulletedList) continue;

                control.IfIs<ListControl>(l => {
                    var placeHolder = new PlaceHolder();

                    l.Items
                    .Cast<ListItem>()
                    .ToList()
                    .Select(i => new ListItemControl(placeHolder, i))
                    .ForEach(visitor.Visit);

                    /* swap the ListControl with a placeholder control so it's children will be rendered. otherwise the ListControl would ignore our child controls that 
                     * we added to the Controls collection. it doesn't read that property when rendering. it reads the Items property which we're not even dealing with. */
                    l.SwapWith(placeHolder);
                });

                control.IfIs<TextBox>(visitor.Visit);
                control.IfIs<CheckBox>(visitor.Visit);

                // for ListControl, the Controls collection will be empty. the actual controls are contained within the Items property.
                recursor(control.Controls.Cast<Control>().ToList());
            }
        };

        recursor(page.Form.Controls.Cast<Control>().ToList());
    }

    public static int Index(this Control control) {
        var index = -1;
        if (control.Parent == null) return index;

        foreach (var c in control.Parent.Controls) {
            index++;

            if (ReferenceEquals(c, control))
                return index;
        }

        throw new InvalidOperationException("Could not find control in parent's control tree.");
    }

    public static void SwapWith(this Control old, Control @new) {
        var index = old.Index();
        var parent = old.Parent;

        parent.Controls.Remove(old);
        parent.Controls.AddAt(index, @new);
    }

    public static void IfIs<T>(this object target, Action<T> action) where T : class {
        var wannaBe = target as T;

        if (wannaBe != null) action(wannaBe);
    }

    public static void ForEach<TType>(this IEnumerable<TType> target, Action<TType> action) {
        foreach (var element in target)
            action(element);
    }
}

internal class DefaultReadOnlyModeControlVisitor : IReadOnlyControlVisitor {
    const string LineBreak = "<br />";
    const string SelectableItemReadOnlyFormat = "<b>{0}{1}{2}</b> {3}{4}";
    const char OpenCurly = '{';
    const char ClosedCurly = '}';
    const char X = 'X';
    const char Underscore = '_';

    public void Visit(TextBox textBox) {
        textBox.SwapWith(new Literal { Text = string.Format("{0}{1}", textBox.Text, LineBreak) });
    }

    public void Visit(CheckBox checkBox) {
        checkBox.SwapWith(new Literal { Text = string.Format(SelectableItemReadOnlyFormat, OpenCurly, checkBox.Checked ? X : Underscore, ClosedCurly, checkBox.Text, LineBreak) });
    }

    public void Visit(ListItemControl listItem) {
        listItem.SwapWith(new Literal { Text = string.Format(SelectableItemReadOnlyFormat, OpenCurly, listItem.Selected ? X : Underscore, ClosedCurly, listItem.Text, LineBreak) });
    }
}

public interface IReadOnlyControlVisitor {
    /// <summary>
    /// Converts a <see cref="TextBox"/> to read-only mode.
    /// </summary>
    /// <param name="textBox">The text box to convert to read-only mode.</param>
    void Visit(TextBox textBox);

    /// <summary>
    /// Converts a <see cref="CheckBox"/> to read-only mode.
    /// </summary>
    /// <param name="checkBox">The check box to convert to read-only mode.</param>
    void Visit(CheckBox checkBox);

    /// <summary>
    /// Converts a <see cref="ListItemControl"/> to read-only mode.
    /// </summary>
    /// <param name="listItem">The list item to convert to read-only mode.</param>
    void Visit(ListItemControl listItem);
}

/// <summary>
/// A wrapper for ListItem since it's not a control.
/// </summary>
public class ListItemControl : Control {
    public ListItemControl(Control parent, ListItem listItem) {
        Selected = listItem.Selected;
        Text = listItem.Text;

        parent.Controls.Add(this);
    }

    public bool Selected { get; private set; }
    public string Text { get; private set; }
}

The beauty of the design is that we handled the recursive part of the code. That's the part that no one wants or even cares to deal with and rightfully so. I picked up this concept while reading Real World Functional Programming. Thomas P talks about how to implement routines like Sum, Max, and Min by encapsulating recursively iterating a list and accepting a function that knows how to do the rest. The client never has to worry about writing the loop. They just provide a function that abstractly accepts 2 values, does something with them, and returns the result. In the case of Sum, you'd provide the + operator as a function by wrapping it in parenthesis. Then you'd have a function like aggregate/reduce (it's really called fold in F#) that accepts the + operator. So a client could call

let seed = 0
let ten = Seq.aggregate seed [1; 2; 3; 4;] (+)

let seed' = 1
let twentyFour = Seq.aggregate seed' [1; 2; 3; 4;] (*)

This approach is backed by this blog post and a must have in any functional style language. It's a lot more verbose to define in C#, but definitely works. You're probably thinking what I'm thinking. Isn't the Aggregate function available in LINQ? Yup. Sure is. And I use it all the time.

In my case, you should only have to worry about reacting to a particular type of control. You inverted control over to me so that you can declaratively tap into the processing of the control tree. That's kind of funny when you think about it. You mean I'm going to give this person a reference to myself and I can't even control when or how many times I'm invoked? That's the beauty of it my friends. It's what IoC is all about. The same thing happens in ASP.NET MVC. When have you ever been in control of when your controller was invoked? You're not. That's the job of the action invoker. You just have to write code. A simple yet powerful concept.

I simply pluck controls from the tree, and if they match, I tell you about it. You're kind of the subject in this case. This is similar to how IQueryable and LINQ Providers work. You write code that knows how to handle each type of expression. Then .NET notifies your expression tree visitor when that particular type of expression shows up. Also when you implement a query provider, you have no control over when your code is executed. .NET will invoke your provider accordingly once the client makes calls to Where, Select, OrderBy, etc. Lastly there's the aforementioned Reactive Framework (Rx) in .NET. It's pretty cool as well with IObserver and IObervable. Their counterparts over in LINQ are IQueryable and IQueryProvider. In both cases .NET has conveniently implemented extesion methods that make use of these 2 heavy weight abstractions. You write code, and the extension methods provided by .NET determine when it will be executed.

I can imagine the next time those Microsoft guys find a standard and generic way of executing code. They'll be some other IX and IXable interface tandem. I have to them credit. They always find ways to formulate the perfect marriage between husband and wife. I wonder if IQueryable and IQueryProvider will be producing offspring in the near future. The world may never know.

Honey, we Have a Visitor...

I'm sure that some of you were attracted to this post to find out how I made usage of the visitor pattern. I used the visitor pattern to handle each type of control. That's exactly what that pattern was created for. You make some high level class that knows how to handle concrete instance of an inheritance hierarchy (2 points for polymorphism). I find it rare that I have a legitimate purpose for using it here, but it worked to perfection in this particular scenario. I created a default implementation of my IReadOnlyControlVisitor interface, but clients are allowed to swap that out if need be. For the record, RadioButton is a CheckBox via inheritance. So I kind of killed two birds with one stone on that regard (lucky me).

Why Make a Separate Control for ListItem?

I had to treat the ListItem object with a special case. Firstly, anytime I come into contact with a ListControl, I immediately drill down into its children. Why should clients have to write the same old loop over and over. All it cares about is the individual items. Secondly, ListItem is not a control, so I needed to make a wrapper for it that encapsulates whether its selected or not. At that point, I can treat it just like any other control and swap it, find its index, etc.

Adding flavor with SwapWith, IfIs<T> and Index

The 2 extension methods, SwapWith, and Index, are 2 pretty clever utilities I implemented. They're both pretty simple and straight forward. SwapWith is just a function if Index. IfIs<T> is an idea I got from a pal and decided to implement myself. I'm sure my implementation matches his line for line. We can both agree that we were fed up writing that POTC (Plain Old Type Cast). So we implemented a more declarative and functional implementation that's not as noisy as the de facto imperative check.

Two can Play that Game

Just to flex the design a little, I came up with a separate visitor that expresses how simple it is for us to customize our implementation. We're wiping out the DefaultReadOnlyModeControlVisitor with a more naive one.

internal class RainbowControlVisitor : IReadOnlyControlVisitor {
    public void Visit(TextBox textBox) {
        textBox.SwapWith(GetDiv("red", "Wacky Red"));
    }

    public void Visit(CheckBox checkBox) {
        checkBox.SwapWith(GetDiv("blue", "Wacky Blue"));
    }

    public void Visit(ListItemControl listItem) {
        listItem.SwapWith(GetDiv("green", "Wacky Green"));
    }

    static Control GetDiv(string color, string text) {
        var red = new WebControl(HtmlTextWriterTag.Div);

        red.Style.Add(HtmlTextWriterStyle.Color, color);
        red.Controls.Add(new Literal { Text = text });

        return red;
    }
}

Getting in Trouble

There are two ways to make this implementation blow up. The first is due to LINQ and that lazy bastard IEnumerable<T> (I love you). Since I'm modifying the incoming control collection, I have to be done enumerating it by the time it arrives. Put simply, the code will fail without a call to ToList which forces eager evaluation. Secondly code nuggets (<%...%>) will make her blow. You can remedy that situation by following this stackoverflow post. There was another good post out there by Rich Strahl, but I can't seem to locate it. The simplest solution is to wrap any code using code nuggets in a PlaceHolder control. All better now?

Samples Anyone?

<%@ Page Title="Home Page" Language="C#" MasterPageFile="~/Site.master" AutoEventWireup="true"
    CodeFile="Default.aspx.cs" Inherits="Default" %>

<asp:Content ID="HeaderContent" runat="server" ContentPlaceHolderID="HeadContent">
</asp:Content>
<asp:Content ID="BodyContent" runat="server" ContentPlaceHolderID="MainContent">
    <h2>
        Welcome to ASP.NET!
    </h2>
    <p>
        To learn more about ASP.NET visit <a href="http://www.asp.net" title="ASP.NET Website">www.asp.net</a>.
    </p>
    <p>
        You can also find <a href="http://go.microsoft.com/fwlink/?LinkID=152368&clcid=0x409"
            title="MSDN ASP.NET Docs">documentation on ASP.NET at MSDN</a>.
    </p>

    <asp:TextBox Text="Antwan As A Literal" runat="server" />
    <asp:RadioButtonList ID="buttonList" runat="server" />
    <asp:CheckBoxList ID="checkBoxList" runat="server" />
    <asp:RadioButton runat="server" Checked="true" Text="Antwan checked me homie!!"  />
    <asp:CheckBox runat="server" Text="R.I.P. to Bone of Cali Swagg" />
</asp:Content>

public partial class Default : Page {
    protected void Page_Load() {
        if (IsPostBack) {
            Response.Write("Why would you bind twice with view state enabled? Don't be silly.");
            return;
        }

        var foods = new List<string> { "pizza", "pineapples", "macaroni" };
        var dances = new List<string> { "cali-duggie", "detroit-jit", "atlanta-shoulder lean" };

        buttonList.DataSource = foods;
        checkBoxList.DataSource = dances;

        buttonList.DataBind();
        checkBoxList.DataBind();

        this.ToReadOnly();

        // skittles
        // this.ToReadOnly(new RainbowControlVisitor());
    }
}

Before


After


Skittles



Conclusion

And that's it. We began by handling the mundane recursive part of our implementation to alleviate the burden on our clients. No one should have to repeatedly implement that code. This set the stage for IoC. It gave us the ability to serve controls to the client in a reactive and convenient fashion. We made use of the visitor pattern to handle each type of control we wanted to convert to read-only mode. We provided clients with a default implementation but gave them the ability to override that implementation by providing their own version of IReadOnlyControlVisitor. Lastly we handled the special case for ListItems since they do not inherit from the base Control class provided by ASP.NET.

Thursday, May 5, 2011

Synchronizing Files With F# and the FileSystemWatcher

So I needed a way to automate change tracking on a set of directories and have those changes merged to another set of directories. In my case, I'm dealing with directories that have a similar make up. That is to say, they contain the same files, folders, etc. Just in different locations. They're essentially clones of one another. The team I'm currently working on calls these packages. I certainly don't agree with the way they implemented it and all the duplication, but I'm not going to manually copy my changes to 2 other directories all day long. So I came up with a simple utility to do the work for me. It's not fully polished yet, but I wanted to get my initial implementation up online. If you actually tried to use the program, things would work, aside from the IOException thats generated after subsequent saves due to the host process somehow maintaining a lock on the files. Now normally it's the developer's fault, and it probably is in my case, but based on the very nature of the function I'm calling, and what it promises to do for me, I doubt it. I'll resolve it in the coming days though. Lastly, I'm working purely with code (.cs, .aspx, .ascx, etc.) so I can get away with calling File.ReadAllText. I'm not even going to think about binary. Maybe it'd still work. Who cares...lol. Enough talking already. Here's the code.

Iteration I - 4 April, 2011

// Learn more about F# at http://fsharp.net
open System.Xml.Linq
open System.Reflection
open System.IO
open System.Linq
open Microsoft.FSharp.Control
open System.Threading
open System

let pathAttributeName = "path"
let xs n = XName.Get(n)
let wildcard = "*.*"

let workflow = async {
    printfn "Started listening at %A..." DateTime.Now

    while Console.ReadLine() <> "q"
        do
            let doc = (Assembly.GetExecutingAssembly().Location |> Path.GetDirectoryName) + "\Synch.config" |> XDocument.Load
            let config = doc.Root
            let root = config.Element("root" |> xs)
            let rootDir = root.Attribute(pathAttributeName |> xs).Value

            let mapdirs (e : XElement) =
                e.Elements("add" |> xs)
                |> Seq.map((fun (e : XElement) ->
                                let dir = [|rootDir; e.Attribute(pathAttributeName |> xs).Value;|] |> String.Concat
                                dir))

            let directories = config.Element("directories" |> xs)
            let masters = directories.Element("masters" |> xs) |> mapdirs
            let slaves = directories.Element("slaves" |> xs) |> mapdirs

            let directoryWatchers = masters
                                    |> Seq.map((fun d -> 
                                                    new FileSystemWatcher(d, EnableRaisingEvents = true, Filter = wildcard)))

            directoryWatchers 
            |> Seq.iter((fun w -> 
                            w.Changed.Add((fun e -> 
                                            let merge fp fn =
                                                let targetDir = Path.GetDirectoryName fp
                                                let content = fp |> File.ReadAllText

                                                slaves
                                                |> Seq.iter ((fun d -> 
                                                                    let fileName = [|d; "\\"; fn;|] |> String.Concat

                                                                    if File.Exists fileName then
                                                                        try
                                                                            File.WriteAllText(fileName, content)

                                                                            printfn "Merged %s to %s at %A %s" fp fileName DateTime.Now Environment.NewLine
                                                                        with 
                                                                            | :? IOException as e ->
                                                                                printfn "Antwan said he'd handle it later. He's eager to get his post up now!!"
                                                                    else
                                                                        printfn "File %s did not exist in directory %s. No merge required. Aborting...%s" fileName d Environment.NewLine
                                                                    ))

                                            e.Name |> merge e.FullPath))))

    printfn "Stopped listening at %A..." DateTime.Now
}

let start() = 
    workflow |> Async.RunSynchronously

do start()

And here's the configuration file I use. No it's probably not the most intuitive xml file you've ever seen, but it works for me. I called it Synch.config and placed it in my bin/Debug directory.

<watch>
 <root path="C:\Users\A-Dubb\Documents\" />
 <directories>
    <!-- Directories I'll be working in -->
    <masters>
      <add path="TestDir" />
    </masters>
    <!-- Directories I want my work merged to -->
    <slaves>
      <add path="TestDirII" />
    </slaves>
 </directories>
</watch>

It's nothing too complex. I just listen for changes in the master directories and merge them to the slave directories. Pretty cool though. It's definitely a good candidate for a Windows Service. I cheated with a while loop to force the main thread to wait on me without exiting the program. There are numerous ways to achieve that behavior as well, but it was quick and painless. I bet you something like DropBox makes use of a similar construct like FileSystemWatcher to keep your files in synch between machines. I'll upload the patch to resolve the IOException once I have time to delve into it.

Wanna get her up and running quickly? You got it. Just download Funtastic. It's a lightweight F# editor. Basically just a wrapper around F# Interactive. She's quite handy though.

For now, adios my friends.

Iteration II - 5 April, 2011

Ok. So I figured out what the problem is. First off, my exception handling code is in the wrong place. It should be concentrated on the attempt to read the file that was actually changed. Not the files that need to be patched. Number 2, since I'm subscribing to the Changed event, it gets triggered just by me simply reading the file. It's cause the file's metadata get's changed by the OS upon reading it (LastAccessedDate). So the Changed event happens so fast (probably nanoseconds) that as I'm reading the file the first time around, I attempt to read it again. Don't believe me? Open up once of your tracked files in Notepad++ and watch it get logged to the console. Even better, upon running the application, you'll notice that you always see the same file get merged to each directory twice. So instead of seeing 2 sets of output, you see 4. I'll have to find a way to suppress notifications for reads. I did try opening the file with FileAccess.Read and FileShare.Read. That didn't work 100% of the time but did seem to be a lot better than what I had before. I also like how ReadAllLines and ReadAllBytes are more high level. I don't have to worry about managing streams, disposing them, reading them, etc. The problem is, you don't have control over access permissions when consuming the file because of the defaults .NET sets for you. I'd never have a source file that's over 2 gigs, but that the most you can load in memory with my current approach because of Int.MaxValue. Maybe the guys at Microsoft know a way around that with their implementation. Who knows? Lastly, I'm working with raw bytes now since that's the fundamental makeup of every file whether it be binary or text based. So I take back my statement from earlier. I kind of do actually care now. I thought I'd have to make some fancy factory that knows how to read and write each file based on it's extension. That'd be one of three things: Either an infinite switch block, a jam packed dictionary, or a regex longer than the Mississippi. Anyway, here's my current revision. You're probably starting to think I'm trying to obsolete git by now. Forgive me. I just want an immediate view of how many times I took a swing at this thing. Don't worry. I'll call it a strikeout at 3.

// Learn more about F# at http://fsharp.net
open System.Xml.Linq
open System.Reflection
open System.IO
open System.Linq
open Microsoft.FSharp.Control
open System.Threading
open System

let pathAttributeName = "path"
let xs n = XName.Get(n)

let workflow = async {
    printfn "Started listening at %A..." DateTime.Now

    while Console.ReadLine() <> "q"
        do
            let doc = (Assembly.GetExecutingAssembly().Location |> Path.GetDirectoryName) + "\Synch.config" |> XDocument.Load
            let config = doc.Root
            let root = config.Element("root" |> xs)
            let rootDir = root.Attribute(pathAttributeName |> xs).Value

            let mapdirs (e : XElement) =
                e.Elements("add" |> xs)
                |> Seq.map((fun (e : XElement) ->
                                let dir = [|rootDir; e.Attribute(pathAttributeName |> xs).Value;|] |> String.Concat
                                dir))

            let directories = config.Element("directories" |> xs)
            let masters = directories.Element("masters" |> xs) |> mapdirs
            let slaves = directories.Element("slaves" |> xs) |> mapdirs

            let directoryWatchers = masters
                                    |> Seq.map((fun d -> 
                                                    new FileSystemWatcher(d, EnableRaisingEvents = true, IncludeSubdirectories = true)))

            directoryWatchers 
            |> Seq.iter((fun w -> 
                            w.Changed.Add((fun e -> 
                                            let merge fp fn =
                                                let targetDir = Path.GetDirectoryName fp
                                                
                                                try
                                                    use fs = File.Open(fp, FileMode.Open, FileAccess.Read, FileShare.Read)
                                                    
                                                    let size = fs.Length |> int
                                                    let buffer = Array.zeroCreate<byte> size
                                                    
                                                    fs.Read(buffer, 0, size) |> ignore
                                                    
                                                    slaves
                                                    |> Seq.iter ((fun d -> 
                                                                    let fileName = [|d; "\\"; fn;|] |> String.Concat

                                                                    if File.Exists fileName then
                                                                        File.WriteAllBytes(fileName, buffer)

                                                                        printfn "Merged %s to %s at %A %s" fp fileName DateTime.Now Environment.NewLine
                                                                    else
                                                                        printfn "File %s did not exist in directory %s. No merge required. Aborting...%s" fileName d Environment.NewLine
                                                                    ))
                                                with
                                                  | :? IOException as ioe ->
                                                        printfn "exception occured %s %s" ioe.Message Environment.NewLine
                                               
                                            e.Name |> merge e.FullPath))))

    printfn "Stopped listening at %A..." DateTime.Now
}

let start() = 
    workflow |> Async.RunSynchronously

do start()

I'll be back for my last strike later.

Iteration III - 5 April, 2011 3:54 PM

Ok. I spent a few minutes looking around at something I completely ignored to start out with. This line allows you to filter your events. You can filter events using F#, but this is even simpler. It still doesn't work for me though, because as soon as I open the file, that in and of itself is considered a change.

new FileSystemWatcher(d, EnableRaisingEvents = true, IncludeSubdirectories = true, NotifyFilter = NotifyFilters.LastWrite)))

I'm throwing in the flag for now, but you have to admire my persistence. I'm sure I could throw in a Sleep call in between reads, but I don't want to hack anything right now. It was kind of fun heuristically playing with the FileSystemWatcher. At least I'm fully aware of its potential limitations. Cool :).

Source can be found here.