Site Meter
 
 

Review: Open-source components used in learn.knockoutjs.com

Over the last two months I’ve been using my wife-approved weekend coding time to build learn.knockoutjs.com, a set of interactive in-browser coding exercises to help people learn the basics of knockout.js in a convenient and fun way. If you haven’t already seen it, grab a hot drink and treat yourself to 15 minutes of enjoyable learning right now!

image

Since this was a one-person weekend project, I’ve been using all possible means to minimise the total development time. A key part of that has been using existing open source components so I can avoid reinventing stuff wherever possible.

In the end, I was surprised to realise just how many open source projects went into building this single-page application, and how high-quality and easy-to-use most of them were. So, I’d like to give them a bit of recognition by listing them here and giving you a mini-review of each. Maybe you’ll benefit from a few of these in your own projects.

1: Knapsack

Knapsack (NuGet: knapsack) is Andrew Davey’s JavaScript / CSS dependency manager for ASP.NET. It lets you define a series of JS/CSS references like this:

Html.ReferenceScript("Scripts/Lib/knockout-1.3.0.latest.js");
Html.ReferenceScript("Scripts/Custom/bindings.js");
Html.ReferenceScript("Scripts/Custom/appViewModel.js");
Html.ReferenceStylesheet("Content/App/boilerplate.css");
// ... etc ...

… and then emit the actual <script> / <link rel="stylesheet"> tags like this:

@Html.RenderScripts()

What’s the point? Well, in debug mode, it just emits one <script> tag per referenced script, as usual, so you can use your browser’s JavaScript debugger as always. But in release mode, it emits just one <script> tag pointing to a special URL that returns all of the JavaScript code combined into a single file, minified using the MS Ajax Minifier library. So, if you view source on learn.knockoutjs.com, you’ll see only one or two <script> tags even though I’m actually using dozens of JS files.

It also sets all the right cache/etag headers so the browser can avoid making unnecessary HTTP requests, but doesn’t risk using out-of-date scripts because the dynamic script URL changes whenever you change the referenced script files (the URL includes a hash of your JavaScript source code).

I’ve been very happy with knapsack, and would definitely recommend it. The one caveat I discovered is that it automatically references all JavaScript files in any folder that you point it at, which is surprising if you intentionally didn’t reference one of the files in that folder. That aside, if you can organise your JS/CSS files into folders to match how they should be combined, you’ll probably be very happy with Knapsack too.

Note: Knapsack will soon be changing its name to something else due to a cease-and-desist.

2: YamlSerializer

As part of my minimum viable product kind of philosophy for the site, I didn’t create any kind of editor for the tutorials, and I didn’t even store them in a database. The simplest editor is a text editor, and the simplest data store is files on disk. So, the tutorials are written in YAML – a human-editable structured text format.

To parse these YAML files, I used YamlSerializer (NuGet: yamlserializer), a .NET library written by Osamu Takeuchi. So, I have .NET classes like this:

public class Tutorial
{
    public string Title { get; set; }
    public virtual ICollection<tutorialStep> Steps { get; set; }
    // ... various other properties ...
}
 
public class TutorialStep
{
    public string Instructions { get; set; }
    // ... other properties ...
}

… and YAML files like this:

title: Introduction
steps:
 - instructions: |
    ##Welcome!
    Blah blah blah, here's what to do...
 - instructions: |
    ##Doing the next thing
    Do the following things...

… and YamlSerializer parses Tutorial/TutorialStep instances like this:

Dictionary data = new YamlSerializer().Deserialize(allText);
return new Tutorial {
    Title = (string)data["title"],
    Steps = (from stepDict in (Dictionary)data["steps"]
             select new TutorialStep {
                 Instructions = ((string)stepDict["instructions"]).Trim(),
                 // ... other properties ...
             }).ToList(),
    // ... other properties ...
};

It’s maybe not quite as pretty as it could be with a more generics-oriented API (or maybe even a dynamic one?) but it’s robust, fast enough (though I do cache the parsed result), and I had it working in minutes. So I’m satisfied with that!

3: Ninject

image

I didn’t strictly need to use Dependency Injection in this project, but since it takes exactly 10 seconds to set up thanks to NuGet, I did it anyway. My favourite DI container is Ninject (NuGet: Ninject.MVC3), a very easy-to-configure product from Nate Kohari. The MVC 3 version hooks itself into the MVC 3 Dependency Injection extensibility point automatically (internally, it uses David Ebbo’s WebActivator to make itself run when your app first starts).

In this site, it’s only used to supply ITutorialRepository instances to my one-and-only HomeController. Really I could have just hardcoded the dependency, as there is only one tutorial repository (YamlTutorialRepository), but hey it genuinely did take less than one minute to set up the DI and it made me feel less like I was just hacking things together…

4: CodeMirror

image

See those lovely in-browser code editors with context-specific syntax highlighting, the ones that even work on IE6? That’s CodeMirror, a JavaScript library from Marijn Haverbeke.

This was not particularly easy to get working just as I wanted – I hit a range of browser-specific performance problems mainly because of my compulsive need to put rounded corners and dropshadows on every element on the page. Nonetheless, it’s a completely vital component, the entire site couldn’t exist without it, and it’s an impressive technical achievement in its own right, so I’m willing it to give it plenty of slack.

The one alternative code editor I tried was Ace – it looks very promising, but had serious layout issues in IE (e.g, if you resize the font). In the future Ace might overtake CodeMirror to become the smartest and best-performing in-browser code editor, but it’s not there yet.

5: Amplify.js

image

Did you notice that if you work on one of the tutorials, then leave the tutorial (possibly even the whole site), and then later come back, the site will offer to restore your code and your position in that tutorial? How can this be, given that you didn’t even register for an account? It’s HTML 5 local storage, of course.

To implement this, I used Amplify.js (NuGet: amplifyjs), a JavaScript library from appendTo. Of all the JavaScript libraries I used, this one was the easiest to get working. It was completely trivial, in fact – just about three lines of code in total.

Whenever you run your code, the site saves your state:

amplify.store("tutorialState." + tutorial.Id, { view: ..., viewModel: ..., stepIndex: ... });

Similarly, whenever you load a tutorial, the site checks if you have saved state:

var savedState = amplify.store("tutorialState." + id);

… and if so, offers to restore it. The site doesn’t care what browser you’re running, because Amplify.js figures out what local storage options exist and uses whichever one is going to work for you.

6: json2.js

Amplify.js needs your browser to support JSON.stringify, the JavaScript API for serializing an object to JSON. All decent browsers do, but just in case you’re running IE 7 (What kind of web developer are you? Shame on you!), I reference json2.js (NuGet: json2) which provides an implementation of JSON.stringify. Json2.js is the work of legendary JavaScript guru and JSON inventer, Douglas Crockford.

7: Highlight.js

In the tutorial instructions, there are lots of little snippets of code showing you what to do. For the syntax highlighting here, I used highlight.js from Ivan Sagalaev.

There are dozens of code-highlighting JavaScript libraries, and I’ve no reason to say this one is better than the rest. I chose this one simply because it lets me “highlight” a certain DOM element at an arbitrary time of my choosing, and doesn’t get fussy about doing things when the page first loads. Also it tolerates existing HTML markup in the element you’re highlighting, which is how I was able to put special yellow backgrounds on selected bits of code to emphasise the key points.

Highlight.js was pretty easy to work with. I made a Knockout custom binding to display the tutorial steps, and whenever it updates the UI, it highlights any code blocks like this:

$('pre code', element).each(function () {
    hljs.highlightBlock(this, '    ' /* tab = four spaces */)
});

To customise the colour scheme, you just edit its CSS file. Pretty convenient.

8: Showdown.js

I said the tutorials are written in YAML, but that’s just how the .NET objects are represented as text. To represent the actual markup within the instructions (e.g., the headings, links, code blocks, etc.), I used Markdown, a simple text-formatting DSL invented by John Gruber. So I’ve got “markdown” syntax within my YAML files, like this:

 - instructions: |
    ##How to write markdown
    To emphasise text, use *asterisks*. To include a code snippet, just indent it like this:
 
        var blah = "this is some JavaScript"
 
    You can put bits of text into code font `using backtick quotes`.

That markdown text gets sent to the client (in JSON form, of course – so many text-based formats!). To render it as HTML in the browser, I use showdown.js, a Markdown-to-HTML conversion library from John Fraser. This is pretty straightforward – I have a Knockout custom binding like this:

ko.bindingHandlers.markdown = {
    update: function (element, valueAccessor) {
        var markdownValue = ko.utils.unwrapObservable(valueAccessor());
        var htmlValue = markdownValue && new Showdown.converter().makeHtml(markdownValue);
        $(element).html(htmlValue || "");
    }
};

… and then I can bind elements to show markdown data like as follows, and it will refresh the display whenever the markdown text changes:

<div data-bind="markdown: someMarkdownValue"></div>

9: jQuery and friends

image

Obviously I used jQuery. I won’t bother discussing the core library because everyone knows about that, but I will mention the jQuery plugins I used:

  • jQuery UI, not for any of the visual elements, but merely for its “draggable” API. (You can drag to resize the panes, see?). I didn’t need any of its buttons, menus, etc.
  • jQuery Impromptu, to display the overlaid prompts (e.g., for choosing a tutorial or for explaining that your code will be overwritten when you click “Help, it’s not working!”). To be honest, this is the only component I regretted using. It’s is probably great if you use it for its intended use case (which includes things like multi-step wizards), but all I wanted was to move an existing DOM element in front of a gray background, lightbox style. jQuery Impromptu tries to do all manner of extra things that that screwed around with my DOM and its default animation was horribly stuttery on Chrome and Safari. In the end, I butchered its source code almost beyond recognition to get the desired behaviour with tolerable performance. It would have been much quicker for me to write my own version from scratch…
  • jQuery Templates (used internally by Knockout, and soon to be replaced with JsRender)
  • jQuery Address (used internally by knockout.address for the hash-based navigation)

10: … and of course, Knockout.js

icon

Not surprisingly, the UI is all orchestrated with knockout.js (NuGet: knockoutjs), so I have a very tidy MVVM architecture for the client-side code. There are objects to represent each of the major UI elements (panes, draggable splitters, code editors, prompts, etc.), and Knockout takes care of keeping the visible UI in sync with that whole object graph.

But I don’t need to explain this – you can learn much more from the tutorials

Summary

Building an application by smashing together a lot of different open-source projects is pretty quick. The end result is not always a perfectly harmonious symphony, as each component brings its own coding styles and conventions, but it works.

What I’ve learned about components is kind of predictable: the most useful ones are

  • Focused. Just solve one problem, please.
  • Self-contained. I’m not going to accept a tall stack of dependencies; it’s too risky.
  • Clearly documented. First I want examples, then a systematic list of API functions.
  • Available on NuGet Smile

It wasn’t difficult to find components to suit my requirements. It’s great to see the open source ecosystems are thriving, both on .NET and JavaScript.

Knockout.js 1.2.0 released

Earlier today I published the final 1.2.0 build of knockout.js. This is a big release with stacks of new goodness since 1.1.2:

  • Changed the license to MIT (more permissive than the previous license, MS-PL)
  • Added useful new bindings: event, attr, html
  • Enhanced some existing bindings: options, checked
  • Enhanced templating features (now supports jquery.tmpl 1.0.0pre as well as previous versions, template polymorphism, passing additional data via options)
  • Various performance improvements (e.g., now precompiles and caches templates, and ensures all data/subscriptions associated with DOM elements are cleaned immediately when those elements are removed) plus bugfixes
  • Better compilation style (ensures Closure Compiler no longer adds any extra variables to global namespace)
  • Various new extensibility points requested by plugin authors
  • Minor syntactical improvements (e.g., added zero-arg constructor for observable arrays)
  • All documentation updated to match 1.2.0

Thanks very much to the many people who have contributed ideas, pull requests, bug reports, and support to make this release possible! I’m really enjoying the community that’s growing around this.

Hang on, what’s Knockout.js?

In case you’ve never heard of it, here’s tekmaven’s explaination: Smile

image

(By the way, for Visual Studio developers: Install-Package knockoutjs)

More specifically, Knockout is a JavaScript library for building rich, responsive user interfaces. It uses automatic dependency tracking and declarative bindings to update your UI whenever your data model changes, letting you cleanly structure your code around the MVVM pattern.

No, that’s just marketing blather… show me something real

OK then, to learn more:

MvcScaffolding: Scaffolding custom collections of files

This blog post is part of a series about the MvcScaffolding NuGet package:

  1. Introduction: Scaffold your ASP.NET MVC 3 project with the MvcScaffolding package
  2. Standard usage: Typical use cases and options
  3. One-to-Many Relationships
  4. Scaffolding Actions and Unit Tests
  5. Overriding the T4 templates
  6. Creating custom scaffolders
  7. This post: Scaffolding custom collections of files

Previously you’ve seen how to modify the output from existing scaffolders (by overriding their T4 templates) and how to create custom scaffolders that take arbitrary parameters and render new templates of their own.

But what if, instead of generating just a single file, you want to generate multiple files that all work together? Well, it’s not surprising that you can do so.

  • You can put multiple Add-ProjectItemViaTemplate commands in your PowerShell file. You can generate as many different files, using as many different templates, as you want.
  • You can invoke one scaffolder from another. For example your custom scaffolder logic can contain “Scaffold Controller” or “Scaffold View” instructions to generate controllers and views using the built-in logic.

In this final post in the current MvcScaffolding series, I’ll show how some of these ideas can be tied together to build something useful.

Example: An Ajax-powered Controller

I showed an example of doing this in my MvcConf talk (video here) a couple of months ago, when I built a custom ControllerWithAjaxGrid scaffolder. This custom scaffolder produces a controller (obviously), but instead of having the usual index/edit/details/delete views, it just has a single view that contains an Ajax-powered grid from which the user can view, add, edit, and delete records. Videos are hard to reuse code from, so for your convenience I’ll reproduce the code in this blog post.

To see it in action, download the demo project, and then scaffold an Ajax-grid controller for the included StockItem model by executing “Scaffold ControllerWithAjaxGrid StockItem” in the console:

SNAGHTML2e143ef5

Run the controller by launching the application and navigating to /StockItems:

image

It uses jqGrid to do all the client-side stuff. By the way, if you’re wondering how to make jqGrid do different things like sorting or filtering, don’t ask me, I have no idea – talk to the experts! Let’s keep this post focused on the scaffolding side of things.

How ControllerWithAjaxGrid Works

I won’t list all the code for ControllerWithAjaxGrid and its template – you can download the runnable demo project for that. As an outline of how it works, ControllerWithAjaxGrid.ps1 contains the following instruction to create the controller itself:

Scaffold MvcScaffolding.Controller $ModelType -Project $Project `
    -CodeLanguage $CodeLanguage -OverrideTemplateFolders $TemplateFolders `
    -NoChildItems -Force:$Force

This inherits all the normal controller-scaffolding logic (e.g., knowing which folder to output the class file into) from MvcScaffolding.Controller, and merely overrides location where the T4 template is found (using –OverrideTemplateFolder). I was then able to put a ControllerWithContext.cs.t4 file in my custom scaffolder’s folder and hence override the code that gets generated. My custom controller template returns JsonResult data to support the Ajax grid. For example, here’s the template for its GetData action:

public JsonResult GridData(int rows, int page)
{
    var count = _context.<#= modelNamePlural #>.Count();
    var pageData = _context.<#= modelNamePlural #>.OrderBy(x => x.<#= Model.PrimaryKey #>).Skip((page - 1) * rows).Take(rows);
    return Json(new {
        page = page,
        records = count,
        rows = pageData,
        total = Math.Ceiling((decimal)count / rows)
    }, JsonRequestBehavior.AllowGet);
}

Similarly, my custom controller doesn’t need all the usual views – it only needs one that contains the JavaScript code to display the grid. I generate that view using a “Scaffold View” command in ControllerWithAjaxGrid.ps1, again using the –OverrideTemplateFolders parameter to make the View scaffolder load the T4 template from ControllerWithAjaxGrid’s folder.

One interesting thing about the view template is that it gets a list of all the properties on your model class:

<# var properties = viewDataType.VisibleMembers().OfType<codeProperty>(); #>

… then later uses this to generate some JavaScript code relating to each property:

<script type="text/javascript">
    jQuery("#ajaxGrid").jqGrid({
        ...
        colNames: [<#= string.Join(", ", properties.Select(prop => "'" + prop.Name + "'")) #>],
        ...
    });
</script>

Finally, my custom controller needs to be sure that there is actually a data context class that it can use for data access. This is easy to arrange: I just put the following command in ControllerWithAjaxGrid.ps1:

Scaffold DbContext $ModelType -DbContextType ((Get-Project $Project).Name + "Context")

You can see I’m using the convention that the data context class should be called ProjectNameContext, but you could change that if you wanted.

For the full code including the T4 templates, download the demo project.

Conclusion

You could use the same techniques to create other custom controller types with different combinations of views, data access classes, and so on.

If you’re unfamiliar with PowerShell then it might take a bit of learning, but once you’ve got a custom scaffolder to suit your project’s requirements, you can commit your /CodeTemplates folder into source control, and then everyone on your team can quickly generate controllers in the style you’ve defined, supplying whatever custom parameters you’ve declared.

For more information about creating custom controllers, see the tutorial in my previous blog post and the list of useful cmdlets for custom scaffolder scripts.