Site Meter
 
 

Category: Uncategorized

Knockout Projections – a plugin for efficient observable array transformations

knockout-projections is a new Knockout.js plugin that adds efficient “map” and “filter” features to observable arrays. This means you can:

  • Write myObservableArray.map(someMappingFunction) to get a new, read-only array-valued observable containing the mapped version of each input item.
  • Write myObservableArray.filter(someFilterFunction) to get a new, read-only array-valued observable containing a subset of the input items.

When the underlying myObservableArray changes, or whenever any observable accessed during the mapping/filtering changes, the output array will be recomputed efficiently, meaning that the map/filter callbacks will only be invoked for the affected items.

The point of all this: it can scale up to maintain live transforms of large arrays, with only a fixed cost (not O(N)) to propagate typical changes through the graph of dependencies.

Trivial mapping example

To illustrate the mechanics in an obvious way, consider this underlying array:

var numbers = ko.observableArray([1, 2, 3]);

Now if you’ve referenced knockout-projections, you can write:

var squares = numbers.map(function(x) { return x*x; });

Initially, squares will contain [1, 4, 9]. It’s observable, so you can use squares.subscribe to get notifications when it mutates, or you can bind it to a DOM element (e.g., foreach: squares) in KO’s usual way.

Now if you transform the underlying array:

numbers.push(8);

… then squares updates (to [1, 4, 9, 64]) and only calls your mapping function for the new item, 8.

Any transformation is permitted, e.g.:

numbers.reverse();

This has the effect of reversing squares (to [64, 9, 4, 1]), again without remapping anything.

If you remove item(s):

numbers.splice(1, 1);

… then squares is updated (here, to [64, 1]) without remapping anything.

In summary, anyObservableArray.map is an efficient, observable equivalent to Array.prototype.map. Also, it can be arbitrarily chained (and combined in chains with filter) to produce graphs of transformations.

Trivial filtering example

The filter feature works exactly as you’d expect, given the above. For example,

var evenSquares = squares.filter(function(x) { return x % 2 === 0; });

Initially, evenSquares will contain just [64]. When you mutate the underlying array,

numbers.push(4); // evenSquares now contains [64, 16]
numbers.push(5); // evenSquares doesn't change
numbers.push(6); // evenSquares now contains [64, 16, 36]

Again, it responds to arbitrary transformations:

numbers.sort();  // evenSquares now contains [16, 36, 64]

A more realistic use case

Typically you won’t just be playing around will small collections of numbers. Most Knockout apps work with collections of model objects – sometimes very large collections.

Here’s a simple model object:

function Product(data) {
    this.id = data.id;
    this.name = ko.observable(data.name);
    this.price = ko.observable(data.price);
    this.isSelected = ko.observable(false);
}

Many KO applications involve fetching a large collection of model objects and exposing that from a viewmodel:

function PageViewModel() {
    // Some data, perhaps loaded via an Ajax call
    this.products = ko.observableArray([
        new Product({ id: 1, name: 'Klein Burger', price: 3.99 }),
        new Product({ id: 2, name: 'Mobius Fries', price: 1.75 }),
        new Product({ id: 3, name: 'Uncountable Chicken Chunks', price: 3.59 }),
        new Product({ id: 4, name: 'Mandelbrot Salad', price: 2.40 }),
        ... etc ...
    ]);
}
 
ko.applyBindings(new PageViewModel());

Now this might be bound to the UI:

<ul data-bind="foreach: products">
    <li>
        <input type="checkbox" data-bind="checked: isSelected" />
        <strong data-bind="text: name"></strong>
        (Price: £<span data-bind="text: price().toFixed(2)"></span>)
    </li>
</ul>

This is all fine, and Knockout is already good at efficiently (i.e., incrementally) updating the UI when the products array changes. But what if you want to track which subset of products is “selected”?

The traditional approach would be something like:

this.selectedProducts = ko.computed(function() {
    return this.products().filter(function(product) {
        return product.isSelected();
    });
}, this);

This works (using Array.prototype.filter, not the Knockout-projections filter function). However, it’s inefficient. Every time the products array changes, and every time any of their isSelected properties changes, it re-evaluates the isSelected property of every product. It has to do so, because it has no built-in understanding of what you’re doing, so it can’t be clever and incremetally update the earlier selectedProducts array.

However, if you use Knockout-projections filter function, e.g.:

this.selectedProducts = this.products.filter(function(product) {
    return product.isSelected();
});

… it’s both syntactically cleaner, and way faster for large arrays: it now updates the selectedProducts array incrementally whenever either products changes or any of the isSelected values changes.

Similarly you might want to output the names of the selected items. So, you could chain on a new selectedNames property, and perhaps furthermore chain on selectionSummary:

this.selectedNames = this.selectedProducts.map(function(product) {
    return product.name();
});
 
this.selectionSummary = ko.computed(function() {
    return this.selectedNames().join(', ')  || 'Nothing';
}, this);

Now when products changes, or when an isSelected or name property changes, the effect will propagate out incrementally through the whole dependency graph with the minimum of callback-invoking, meaning a snappy user experience even with large data sets and low-end mobile devices.

Licence and origin

knockout-projections is open source under the Apache 2.0 license. It was built as a tiny component in a very large project I’m working on at Microsoft, and thanks to my boss (and his boss, and probably multiple people up the chain) we’ve decided to open-source it. Hopefully my team will be producing plentiful nuggets of OSS goodness for you as we go about our work :)

Just a clarification for the avoidance of any confusion, Knockout.js itself remains MIT licensed and is run by the KO community and the KO core team members (which includes me personally).

A standing desk with (mostly) Ikea parts

Most developers by now are familiar with the standing desk trend. This post isn’t about the pros/cons, suggestions about how long you should stand for, or anything like that. Let’s just say that like me you want to give it a go, but you’re not inclined to spend many hundreds of pounds on a brand new high-end desk until you’re sure you really will use it for the next five years or so.

Many others have built their own standing desks before, even with Ikea parts. So this is hardly revolutionary. But it’s a good looking, inexpensive design, that others may find interesting.

Finished result

This standing desk sits on top of your existing sitting desk, and is designed to take up roughly half of a typical desk width. So you’ll have a half-sitting, half-standing desk, and can easily move between the two postures:

Desk from front

Pros:

  • Transportable. Can easily move it around between desks/workplaces (unscrew the legs by hand, carry it over)
  • Good ergonomics. Gives a lot of room around your keyboard and mouse, and keeps the monitor a pleasing distance from your face, compared with alternatives.
  • Cheap(ish), versus commercial pre-built standing desks, yet looks professional.

Cons:

  • Not height-adjustable, unless you count taking a hacksaw to the legs.
  • Only wide enough for a single large monitor
    • To be fair, there’s nothing to stop you putting on a wider shelf. It’s easily strong enough to take a second large monitor. It would just look a bit more imposing.
    • However, since I switched back to having a single large monitor a few years ago, this is exactly how I want it. Less neck-swivelling = more comfort for me personally.
  • Cheaper DIY alternatives exist
  • Requires a trip to Ikea. But hey, meatballs, right?

Ingredients

Many of the following components can be substituted with alternatives, depending on what you have available. For example, there’s probably no real reason to use Sugru if you already have some strong glue such as Araldite. And any similar-sized piece of wood will do in place of the Ekby Laiva shelf.

From Ikea:

Order online:

Total cost: from £41.24 to £70.99 depending on optional parts. But I do recommend some kind of mat. And I assume you have tools (a drill, a hacksaw) and a range of small screws, since the ones that Ikea include are too big.

Method

When you see the list of parts, it’s pretty obvious how to construct it. So I’m not going to list every screw, cut, or measurement – I’ll just give an outline with some tips. Here’s another viewpoint on the desk to clarify its arrangement:

Desk at angle

Getting the right height

Start here because it’s the most important bit. I’ve read that your standing desk surface should be at roughly elbow-height, but I like mine about 8-10cm below that. You will need to account for the height of the underlying desk and thickness of the table top (obviously), and perhaps the height of your keyboard or anti-fatigue mat if either are big. You might want to first construct a mock-up by balancing the table top on some boxes or whatever.

When you’re sure of the height, measure the Adils legs from the circular bracket at the top downwards, and subtract from your measurement about 1cm to account for the feet you’ll attach later. Now hacksaw the legs to length, and retain the adjustable feet for later.

Screw the circular Adils leg brackets into the underside of the table top now, but don’t screw the legs into them just yet.

Fitting the monitor stand

Be careful when planning how far back to mount the monitor stand to ensure the monitor’s centre of gravity will end up within the desk’s four legs. For me, the desk’s size and the angle of the Capita brackets result in a pretty ideal natural distance from my face to the monitor, but individual preferences vary.

The Capita brackets are a bit baffling because there are so many possible ways of attaching them. I’d recommend first drilling the two 9mm(ish) holes in your table top, loosely bolting on the Capita brackets, and then marking out where the screw holes naturally land on the underside of the Ekby Laiva shelf.

Then drill pilot holes into the Ekby Laiva shelf, bolt the Capita brackets tightly to the table top, and finally screw the Ekby Laiva shelf to the top of the Capita brackets with some small screws.

Finishing the legs

You can now screw in the Adils legs to their brackets, and ta-da – your standing desk is usable. However the bottom end of the legs is just roughly sawn metal, which will make a mess of whatever desk it stands on.

Fortunately you can take the feet off the sawn-off parts of the Adils legs and attach them to the bottom of your new desk’s legs. You can stick them on with strong glue such as Araldite. Instead, I used rolled-out circles of Sugru (see photo below), partly just because I was in a “hack your stuff” mood, and partly because it let me compensate for some irregularities in my unskilled hacksawing:

Desk feet with Sugru

Review

I’ve only been using this for a few weeks now, but the desk feels great to use – it’s sturdy, spacious, and looks good in my office. My elbow pain (the reason I did this in the first place) was gone after the first two days. I tend to stand through the morning and sit in the afternoon, though maybe I should reverse the order to minimise post-lunch lag.

After the first couple of multi-hour standing-working sessions, it was clear I couldn’t just stand on carpet in socks (or shoes) for hours. Your feet will probably tell you the same. I stood on a foam pillow for the next few days, and it was hugely better. For a better, more long-term solution, follow the advice of standing-mat devotees everywhere and get a nice gel mat to stand on, such as the Dandy one I mentioned above. These mats are a bit pricey, but feel so much better.

Rich JavaScript Applications – the Seven Frameworks (Throne of JS, 2012)

A week ago was the Throne of JS conference in Toronto, perhaps the most interesting and different conference I’ve been to for a while. Quoting its website:

It’s no longer good enough to build web apps around full page loads and then “progressively enhance” them to behave more dynamically. Building apps which are fast, responsive and modern require you to completely rethink your approach.

The premise was to take the seven top JavaScript frameworks/libraries for single-page and rich JavaScript applications — AngularJS, Backbone, Batman, CanJS, Ember, Meteor, Knockout, Spine — get the creators of all of them in one location, and compare the technologies head to head.*

Disclaimer: I was there to represent Knockout, so obviously I’m not neutral. In this post my focus is on what the creators said about the scope and philosophy of their technologies, and not so much on whether I agree or disagree.

* Yes, I know that’s eight frameworks, not seven. This part was never fully explained…

TL;DR Executive Summary

  • For many web developers, it’s now taken for granted that such client-side frameworks are the way to build rich web apps. If you’re not using one, you’re either not building an application, or you’re just missing out.
  • There’s lots of consensus among the main frameworks about how to do it (Model-View-* architecture, declarative bindings, etc. — details below), so to some extent you get similar benefits whichever you choose.
  • Some major philosophical differences remain, especially the big split between frameworks and libraries. Your choice will deeply influence your architecture.
  • The conference itself was stylish and upbeat, with a lot of socialising and conversations across different technology groups. I’d like to see more like this.

Technologies: Agreement and Disagreement

As each SPA technology was presented, some fairly clear patterns of similarity and difference emerged.

Agreement: Progressive enhancement isn’t for building real apps.

All the technologies follow from the view that serious JavaScript applications require proper data models and ability to do client-side rendering, not just server rendering plus some Ajax and jQuery code.

Quote from Jeremy Ashkenas, the Backbone creator: “At this point, saying ‘single-page application’ is like saying ‘horseless carriage’” (i.e., it’s not even a novelty any more).

Agreement: Model-View-Whatever.

All the technologies made use of model-view separation. Some specifically talked about MVC, some about MVVM, and some specifically refused to define the third piece (just saying it’s models, views, and some kind of application thing that makes them work together). The net result in each case was similar.

Agreement: Data binding is good.

All except Backbone and Spine have a built-in notion of declarative data binding in their views (Backbone instead has a “bring your own view technology” design).

Agreement: IE 6 is dead already.

In a panel discussion, most framework creators said their IE support focus was limited to version 7+ (in fact, Ember and AngularJS only go for IE8, and Batman requires an ES5 shim to run on IE older than v9). This is the way of things to come: even jQuery 2 is set to drop support for IE older than v9.

The only stalwarts here appear to be Backbone and Knockout which support IE6+ (I don’t know about Backbone’s internals, but for KO this means transparently working around a lot of crazy edge-case IE6/7 rendering and eventing weirdnesses).

Agreement: Licensing and source control.

Every single one is MIT licensed and hosted on GitHub.

Disagreement: Libraries vs frameworks.

This is the biggest split right now. You could group them as follows:

Libraries Frameworks

Backbone (9552)

Knockout (2357)

Spine (2017)

CanJS (321)

Ember (3993)

AngularJS (2925)

Batman (958)

Meteor (4172) — unusual, see later

Numbers in brackets are a point-in-time snapshot of the number of GitHub watchers, as a crude indicator of relative influence.

What does this mean?

  • Libraries slot into your existing architecture and add specific functionality
  • Frameworks give you an architecture (file structure, etc.) that you are meant to follow and, if you do, are intended to handle all common requirements

By far the most passionate advocate of the framework model is Ember, whose creator Yehuda Katz is formerly of the Rails and SproutCore projects (similar philosophy). His argument was that anything less is just not ambitious enough and isn’t seriously advancing the state of the art. The counter-argument is that libraries are more focused, and hence can be easier to learn, adopt, customize, and help minimise project risk because your architecture isn’t so deeply tied to a specific external project. Based on my conversations, I’d say the audience was split and supported both sides of this debate.

Note that AngularJS is arguably somewhere in between library and framework: it doesn’t require a particular layout of files at development time (library-like), but at runtime it provides an “app lifecycle” that you fit your code into (framework-like). I’m listing it as a framework because that’s the terminology the AngularJS team prefers.

Disagreement: What’s flexible, what’s integrated.

Each technology has different levels of prescriptiveness:

Views

URL routing

Data storage

AngularJS

Built-in DOM-based templates (mandatory)

Built-in (optional)

Built-in system (optional)

Backbone

Choose your own (most used handlebars.js, a string-based template library)

Built-in (optional)

Built-in (overridable)

Batman

Built-in DOM-based templates (mandatory)

Built-in (mandatory)

Built-in system (mandatory)

CanJS

Built-in string-based templates (mandatory)

Built in (optional)

Built in (optional)

Ember

Built-in string-based templates (mandatory)

Built-in (mandatory)

Built-in (overridable)

Knockout

Built-in DOM-based templates (optional, can do string-based too)

Choose your own (most use sammy.js or history.js)

Choose your own (e.g., knockout.mapping or just $.ajax)

Meteor

Built-in string-based templates (mandatory)

Built-in (mandatory?)

Built-in (Mongo, mandatory)

Spine

Choose your own string-based templates

Built-in (optional)

Built-in (optional?)

As expected, whenever a library leaves a decision open, they argue it is better overall to guarantee composablity with arbitrary 3rd-party libraries. And the obvious counter-argument is that integration can be more seamless if built-in. Again, based on my conversations, the audience was split and opinions went in all directions — usually based on how much other technology stack an individual was wedded to.

Quote from Tom Dale of Ember: “We do a lot of magic, but it’s good magic, which means it decomposes into sane primitives.

Disagreement: String-based vs DOM-based templates

(As shown in the above table.) For string-based templates, almost everyone used Handlebars.js as the template engine, which seems to dominate this space, though CanJS used EJS. Arguments in favour of string-based templates include “it’s faster” (debatable) and “theoretically, the server can render them too” (also debatable, as that’s only true if you can actually run all of your model code on the server, and nobody actually does that in practice).

DOM-based templates means doing control flow (each, if, etc.) purely via bindings in your actual markup and not relying on any external templating library. Argument include “it’s faster” (debatable) and “the code is easier to read and write, because there’s no weird chasm between markup and templates, and it’s obvious how CSS will interact with it“.

In my view, the strongest argument here came from the AngularJS guys who stated that in the near future, they expect DOM-based templating will be native in browsers, so we’ll best prepare ourselves for the future by adopting it now. AngularJS is from Google, so they are already working on this with Chromium and standards bodies.

Disagreement: Levels of server-agnosticism

Batman and Meteor express explicit demands on the server: Batman is designed for Rails, and Meteor is its own server. Most others have a goal of being indifferent to what’s on your server, but in practice the architecture, conventions, and some tooling in Ember leans towards Rails developers. Ember absolutely works on other server technologies too, though today it takes a little more manual setup.

The technologies — quick overview

Here’s a rundown of the basic details of each technology covered:

Backbone
  • Who: Jeremy Ashkenas and DocumentCloud
  • What:
    • Model-View in JavaScript, MIT licensed
    • Most minimal of all the libraries — only one file, 800 lines of code!
    • Extremely tightly-scoped functionality — just provides REST-persistable models with simple routing and callbacks so you know when to render views (you supply your own view-rendering mechanism).
    • The best-known of them all, with the most production deployments on big-name sites (perhaps easy to adopt because it’s so minimal)
  • Why:
    • It’s so small, you can read and understand all of the source before you use it.
    • No impact on your server architecture or file layout. Can work in a small section of your page — doesn’t need to control whole page.
    • Jeremy seems to exist in a kind of zen state of calm, reasonable opinions about everything. He was like the grown up, supervising all the arguing kids.
  • Where: GitHub and own site
  • When: In production for nearly 2 years now
Meteor
  • Who: The Meteor development group, who just raised $11.2 Million so they can do this full-time
  • What:
    • Crazy amazing framework from the future, barely reminiscent of anything you’ve ever seen (except perhaps Derby)
    • Bridges a server-side runtime (on Node+Mongo) with a client-side one, so your code appears to run on both, including the database. WebSockets syncs between all client(s) and server.
    • Does “live deployments” every time you edit your code – client-side runtimes are updated on the fly without losing their state
    • Makes more sense if you watch the video
    • Like everyone I spoke to at the event, I really want this to succeed — web development needs something this radical to move forwards
  • Why: You’ve had enough of conventional web development and now want to live on the bleeding edge.
  • Where: GitHub and own site
  • When: It’s still early days; I don’t know if there are any production Meteor sites yet except built by the core team. They’re totally serious about doing this, though.
Ember
  • Who: Yehuda Katz (formerly of jQuery and Rails), the Ember team, and Yehuda’s company Tilde
  • What:
    • Everything you need to build an “ambitious web application”, MIT license
    • Biggest framework of them all in both functionality and code size
    • Lots of thought has gone into how you can decompose your page into a hierarchy of controls, and how this ties in with a statemachine-powered hierarchical routing system
    • Very sophisticated data access library (Ember.Data) currently in development
    • Intended to control your whole page at runtime, so not suitable for use in small “islands of richness” on a wider page
    • Pretty heavily opinionated about files, URLs, etc., but everything is overridable if you know how
    • Design inspired by Rails and Cocoa
    • Tooling: They supply project templates for Rails (but you can use other server platforms if you write the code manually)
  • Why: Common problems should have common solutions — Ember makes all the common solutions so you only have to think about what’s unique to your own application
  • Where: GitHub and own site
  • When: Not yet at 1.0, but aiming for it soon. API will solidify then.
AngularJS
  • Who: Developed by Google; used internally by them and MIT licensed.
  • What:
    • Model-View-Whatever in JavaScript, MIT licensed
    • DOM-based templating with observability, declarative bindings, and an almost-MVVM code style (they say Model-View-Whatever)
    • Basic URL routing and data persistence built in
    • Tooling: they ship a Chrome debugger plugin that lets you explore your models while debugging, and a plugin for the Jasmine testing framework.
  • Why:
    • Conceptually, they say it’s a polyfill between what browsers can do today and what they will do natively in a few years (declarative binding and observability), so we should start coding this way right now
    • No impact on your server architecture or file layout. Can work in a small section of your page — doesn’t need to control whole page.
  • Where: GitHub and own site
  • When: In production now (has been at Google for a while)
Knockout
  • Who: The Knockout team and community (currently three on the core team, including me)
  • What:
    • Model-View-ViewModel (MVVM) in JavaScript, MIT licensed
    • Tightly focused on rich UIs: DOM-based templates with declarative bindings, and observable models with automatic dependency detection
    • Not opinionated about URL routing or data access — combines with arbitrary third-party libraries (e.g., Sammy.js for routing and plain ajax for storage)
    • Big focus on approachability, with extensive documentation and interactive examples
  • Why:
    • Does one thing well (UI), right back to IE 6
    • No impact on your server architecture or file layout. Can work in a small section of your page — doesn’t need to control whole page.
  • Where: GitHub and own site
  • When: In production for nearly 2 years now
Spine
  • Who: Alex MacCaw
  • What:
    • MVC in JavaScript, MIT license
    • Worked example originally written for an O’Reilly book grew into an actual OSS project
    • Is a kind of modified clone of Backbone (hence the name)
  • Why: You like Backbone, but want a few things to be different.
  • Where: GitHub and own site
  • When: It’s past v1.0.0 now
Batman
  • Who: the team at Shopify (an eCommerce platform company)
  • What:
    • MVC in JavaScript, almost exclusively for Rails+CoffeeScript developers, MIT licensed
    • Most opinionated of them all. You must follow their conventions (e.g., for file layout and URLs) or, as they say in their presentation,”go use another framework
    • Full-stack framework with pretty rich models, views, and controllers and routing. And observability mechanism of course.
    • DOM-based templating.
  • Why: If you use Rails and CoffeeScript, you’ll be right at home
  • Where: GitHub and own site
  • When: Currently at 0.9. Aiming for 1.0 in coming months.
CanJS
  • Who: the team at Bitovi (a JavaScript consulting/training company)
  • What:
    • MVC in JavaScript, MIT licensed
    • REST-persistable models, basic routing, string-based templating
    • Not widely known (I hadn’t heard of it before last week), though is actually a reboot of the older JavaScriptMVC project
  • Why: Aims to be the best of all worlds by delivering features similar to the above libraries while also being small
  • Where: GitHub and own site
  • When: Past 1.0 already

Summary

If you’re trying to make sense of which of these is a good starting point for your project, I’d suggest two questions areas to consider:

  • Scope. How much do you want a framework or library to do for you? Are you starting from blank and want a complete pre-prepared architecture to guide you from beginning to end? Or do you prefer to pick your own combinations of patterns and libraries? Either choice has value and is right for different projects and teams.
  • Design aesthetic. Have you actually looked at code and tried building something small with each of your candidates? Do you like doing it? Don’t choose based on descriptions or feature lists alone: they’re relevant but limited. Ignoring your own subjective coding experience would be like picking a novel based on the number of chapters, or a spouse based on their resume/CV.

Despite the differences, I’d argue there is one killer feature all the above technologies share: the idea of Model/View separation. This is a classic design pattern that predates the web itself by about 20 years. If you’re building even the most basic kind of web application UI, you’ll almost certainly benefit from applying this on the client.