Site Meter
 
 

Improve scalability in ASP.NET MVC using Asynchronous requests

ASP.NET applications run with a fixed-size thread pool. By default, they have 200 (or 250? I forget…) threads available to handle requests. You can make the thread pool bigger if you want, but it’s not usually helpful: more contention and overhead from thread switching will eventually actually reduce the server’s throughput. But what if most of your threads are actually sitting around doing nothing, because they’re actually waiting for some external I/O operation to complete, such as a database query or a call to an external web service? Surely things could be more efficient…

Well yes, actually. Ever since Windows NT 4 we’ve had a notion of I/O Completion Ports (IOCP) which are a mechanism for waiting for I/O to complete without causing thread contention in the meantime. .NET has a special thread pool reserved for threads waiting on IOCP, and you can take advantage of that in your ASP.NET MVC application.

image

The IHttpAsyncHandler, first introduced in ASP.NET 2.0, splits request processing into two. Instead of handling an entire request in one thread (expensive I/O and all), it first does some processing in the normal ASP.NET thread pool, as per any normal request, then when it’s time for I/O, it transfers control to the I/O thread, releasing the original ASP.NET thread to get on with other requests. When the I/O signals completion (via IOCP), ASP.NET claims another, possibly different thread from its worker pool to finish off the request. Thus, the ASP.NET thread pool doesn’t get ‘clogged up’ with threads that are actually just waiting for I/O.

Adding asynchronous processing to ASP.NET MVC

The MVC framework doesn’t (yet) come with any built-in support for asynchronous requests. But it’s a very extensible framework, so we can add support quite easily. First, we define AsyncController:

public class AsyncController : Controller
{
    internal AsyncCallback Callback { get; set; }
    internal IAsyncResult Result { get; set; }
    internal Action<iasyncResult> OnCompletion { get; set; }
 
    protected void RegisterAsyncTask(Func<asyncCallback, IAsyncResult> beginInvoke, Action<iasyncResult> endInvoke)
    {
        OnCompletion = endInvoke;
        Result = beginInvoke(Callback);
    }
}

It’s just like a normal Controller, except it manages some internal state to do with asynchronous processing, and has the RegisterAsyncTask() method that lets you manage transitions across the gap between the two halves of IHttpAsyncHandler processing. Note: it doesn’t implement IHttpAsyncHandler itself; that’s the job of the IRouteHandler we have to set up. Unfortunately I had to reproduce most of the code from the framework’s MvcHandler, because I couldn’t just override any individual method and still get at the controller:

public class AsyncMvcRouteHandler : IRouteHandler
{
    public IHttpHandler GetHttpHandler(RequestContext requestContext)
    {
        return new AsyncMvcHandler(requestContext);
    }
 
    class AsyncMvcHandler : IHttpAsyncHandler, IRequiresSessionState
    {
        RequestContext requestContext;
        AsyncController asyncController;
        HttpContext httpContext;
 
        public AsyncMvcHandler(RequestContext context)
        {
            requestContext = context;
        }
 
        // IHttpHandler members
        public bool IsReusable { get { return false; } }
        public void ProcessRequest(HttpContext httpContext) { throw new NotImplementedException(); }
 
        // IHttpAsyncHandler members
        public IAsyncResult BeginProcessRequest(HttpContext context, AsyncCallback cb, object extraData)
        {
            // Get the controller type
            string controllerName = requestContext.RouteData.GetRequiredString("controller");
 
            // Obtain an instance of the controller
            IControllerFactory factory = ControllerBuilder.Current.GetControllerFactory();
            IController controller = factory.CreateController(requestContext, controllerName);
            if (controller == null)
                throw new InvalidOperationException("Can't locate the controller " + controllerName);
            try
            {
                asyncController = controller as AsyncController;
                if (asyncController == null)
                    throw new InvalidOperationException("Controller isn't an AsyncController.");
 
                // Set up asynchronous processing
                httpContext = HttpContext.Current; // Save this for later
                asyncController.Callback = cb;
                (asyncController as IController).Execute(new ControllerContext(requestContext, controller));
                return asyncController.Result;
            }
            finally
            {
                factory.DisposeController(controller);
            }
        }
 
        public void EndProcessRequest(IAsyncResult result)
        {
            CallContext.HostContext = httpContext; // So that RenderView() works
            asyncController.OnCompletion(result);
        }
    }
}

It handles requests by supplying an AsyncMvcHandler, which implements IHttpAsyncHandler. Note that during the first half of the processing, i.e. during BeginProcessRequest(), it makes a record of the current HttpContext object. We have to restore that later, during EndProcessRequest(), because we’ll be in a new thread context by then and HttpContext will be null (and that breaks various ASP.NET facilities including WebForms view rendering).

Using AsyncController

It’s now very easy to handle a request asynchronously. Define a route using AsyncMvcRouteHandler, instead of MvcRouteHandler:

routes.Add(new Route("Default.aspx", new AsyncMvcRouteHandler())
{
    Defaults = new RouteValueDictionary(new { controller = "Home", action = "Index", id = "" }),
});

Then set up an AsyncController. In this example, we’re calling an external web service using the WebRequest class and its BeginGetResponse() method. That uses an IOCP, so won’t consume an ASP.NET worker thread while it waits:

public class HomeController : AsyncController
{
    public void Index()
    {
        WebRequest req = WebRequest.Create("http://www.example.com");
        req.Method = "GET";
 
        RegisterAsyncTask(cb => req.BeginGetResponse(cb, null), delegate(IAsyncResult result) {
            WebResponse response = req.EndGetResponse(result);
            // Do something with the response here if you want
            RenderView("Index");
        });
    }
}

And that’s it. The web request gets set up and started in the first half of the async processing model, then the thread gets released to serve other requests. When the web request signals completion, ASP.NET takes a different thread from its pool, and gets it to run the code inside the anonymous delegate, inheriting the request context from the first thread so it can send output to the visitor.

Just remember that you should only use async requests when you’re waiting for some operation on an IOCP. Don’t use it if you just going to call one of your own delegates asynchronously (e.g. using QueueUserWorkItem()) because that will come out of the ASP.NET worker thread pool and you’ll get exactly zero benefit (but more overhead).

22 Responses to Improve scalability in ASP.NET MVC using Asynchronous requests

  1. Pingback: Links Today (2008-04-07)

  2. Joann Everman

    One correction. The threadpool size is increased to 250 in .NET 2.0.

  3. This is quite cool stuff! It’s actually the “regular ASP.NET” asynchronous processing applied to MVC.

  4. Really Cool. Just a small correction the default Size of the ThreadPool in .NET V1.0 was 25. But from v2.0 it has been increased to 200 for the ASP.NET.

  5. Pingback: Reflective Perspective - Chris Alcock » The Morning Brew #69

  6. John

    I fail to see how this improves scalability. I think you may be confusing “improved performance” with “improved scalability”..

    While moving to an Async Controller would most definitely increase my per server capacity, I don’t see how it has anything to do with how my application scales (whether that’s out or up).

    By using the same logic, I might come to the conclusion that writing my webapp in assembler would yield the most scalable solution.

  7. Steve

    John, I appreciate your point, that you could look at this and say that the main benefit is increasing per-box throughput (i.e. performance gains rather than scalability gains), but I’d also say you can look at it another way: if you *don’t* use async processing, you’re introducing a genuine barrier to scalability.

    That is, the fixed threadpool size means that no matter how much you improve the hardware in one server, its throughput for I/O bound requests won’t increase beyond a fixed level. Of course you can increase the threadpool size, but then you have to pay with increased thread contention, so scalability is limited.

    When you *do* use async processing, that limitation is lifted and the hardware is used to 100% of its capacity. That means you can double the hardware specs and get a doubling in throughput (scalability).

  8. Pingback: Wöchentliche Rundablage: Silverlight 2, ASP.NET MVC, .NET 3.5, jQuery, CSS, Powershell | Code-Inside Blog

  9. Pingback: Weekly Links: Silverlight 2, ASP.NET MVC, .NET 3.5, jQuery, CSS, Powershell | Code-Inside Blog International

  10. mark

    his is very good post but how to implement that dodnt get it can plese the example to download

  11. Steve

    Hi Mark – the above code was designed for an earlier version of the MVC framework (Preview 2) and won’t work in the latest or newer versions without adaptation.

    You might want to hang on to see whether the final release has native support for async requests. Also, consider carefully whether it will actually benefit your application, since it’s rare to be in a situation where it actually will.

  12. Salman Rafique

    I tried to implement this code in my MVC project. But it always gives the error

    The controller for path ~/Home.mvc/Index’ could not be found or it does not implement the IController interface.
    Parameter name: controllerType

    I am using MVC preview 3. Actually my goal is to use SmtpClient’s SendAsync() method in MVC.
    Is there any help regarding this.

  13. Tim

    This also does not seem to work in Preview 4. Sometime after BeginProcessRequest and the start of my action, it gives this error:

    [NullReferenceException: Object reference not set to an instance of an object.]
    System.Web.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute() +8674600
    System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously) +155

    Not much to go on.

  14. Tim

    My last comment was not quite correct. The error occurs between BeginProcessRequest and EndProcessRequest. My action seems to excecute fine when called from BeginProcessRequest.

  15. Steve

    Hi Tim. Sorry about that. This code was written for some earlier version of ASP.NET MVC (Preview 2?) so I wouldn’t expect it to work automatically with Preview 4. I’m not sure what changes are needed but you have the source code so you can investigate.

    Sorry that’s not very helpful. Unfortunately I can’t keep all the code on my blog up-to-date as MVC changes every few weeks. When we get the RTM version of MVC then things will settle and I might update some of these older posts.

    Good luck!

  16. Hi Steve,

    This is a really cool post. However, I have a question about how to make ASP.NET MVC to invoke implementation of IHttpAsyncHandler instead of IHttpHandler.

    I did experimentation with your sample code. The class AsyncMvcHandler implements interface IHttpAsyncHandler which implements IHttpHandler. So, it has three methods to implement:

    public void ProcessRequest(HttpContext httpContext)
    public IAsyncResult BeginProcessRequest(HttpContext context, AsyncCallback cb, object extraData);
    public void EndProcessRequest(IAsyncResult result)

    I set breakpoint to the three methods. And only ProcessRequest get invoked. If BeginProcessRequest/EndProcessRequest doesn’t get invoked, it is not asynchronous.

    Can you make it clear that how to make ASP.NET MVC to treat AsyncMvcHandler as IHttpAsyncHandler?

    Thanks.

  17. Maxim

    Hi, Steve.
    Actually I’ve got the same situation, in AsyncMvcRouteHandler only ProcessRequest being invoked, maybe you can help us ?

  18. Hi Steve & Maxim,

    About the AsyncMvcRouteHandler only got ProcessRequest method invoked issue, I believe it is a ASP.NET MVC bug.

    If the URI is kind of “default” like “http://website/”, only ProcessRequest is invoked. If the URL is in full mode like “http://website/Home/Index”, then ASP.NET can invoke BeginProecessRequest.

  19. Does not look like this made it in the MVC-2 release. Is it still in the futures release? Great article by the way.

  20. Pingback: Measuring the Performance of Asynchronous Controllers « Steve Sanderson’s blog

  21. Srini

    Steve,

    I am sorry, I know this is a wrong spot to ask this question…

    I bought your MVC 3 book and downloaded the source code from Apress.com ; but in those sample I am not finding the database that you used to build your sample code of the book.

    Could you guide me please where I have to download this related database from?

    Thanks
    Srini G

  22. SeePlusPlus

    Would you recommend calling an external REST web service from async controller (in mvc 3) or implementing that logic by under IHttpAsyncHandler? I have to post some data to external service via javascript from browser and i am using httphandler right now to pass of that request from JS and get response back that way.