Progressive Download Support in ASP.NET Web API

(Update 2014-12-10: The code is now in GitHub and a package is available as a NuGet Package. See the announcement about it.)

Some time ago I read an interesting Stack Overflow question asking about streaming videos to iOS devices from ASP.NET MVC. In researching the answer, I learned about parsing HTTP request headers and constructing responses. In general for iOS devices to support playing video a web server needs to understand requests with the RANGE header and respond appropriately with 206 (PartialContent) and a CONTENT-RANGE header which more or less repeats the original request range.

I originally wrote an answer to this effect but also wanted to test it out to make sure it worked in ASP.NET MVC. While successful, it was not particularly elegant. After getting some feedback and requests, I wrote a second version. Surprisingly I still get pings on this from time to time and it made me wonder if something more integrated has shown up in ASP.NET MVC since my original attempts. Until recently I hadn’t had a need for this myself until a couple days ago I got the urge to stream video content from my Windows laptop to my TV and thought this might be a good time to revisit the library. What I wanted was an iOS compliant video server so I could stream videos to my Apple TV via my phone.

I knew I had a working library for ASP.NET MVC that I could have up and running in a few minutes but I was more curious if there had been any advancements in supporting partial range requests more seamlessly. A few brief searches turned up the ByteRangeStreamContent class and the HttpRequestMessage’s RANGE header used in ASP.NET Web API which looked promising.

System.Net.Http.HttpRequestMessage

In Web API there are a few ways you can construct public methods. The main route I’ve always used was to return instances of my POCO objects directly and let Web API take care of the rest. I had seen references to HttpRequestMessage and HttpResponseMessage classes for access to request and response headers but I’d had no need to use them until now.

One of the first problems I had to solve in the ASP.NET MVC library was the parsing of the request headers for the RANGE header. This header can have a few different formats which need to be taken into account. It can be a single range of start and end byte (200-2000), a start with open end (200-) which means to start at the 200th byte and stream whatever is left, it can ask for the last bytes of a stream (-1000) and it can combine multiple range requests (200-2000, -500). It turns out we don’t need to do this ourselves anymore as the HttpRequestMessage.Headers.Range property parses this for us. All we have to do is check to see if it’s null (no RANGE header was present) or if there are 1 or more Ranges.

System.Net.Http.HttpResponseMessage

Once the RANGE is parsed, we need to create an appropriate HTTP response. First, if there was a RANGE header, we need to respond with 206 (PartialContent). We also need to add a header (CONTENT-RANGE) which reflects the request header’s RANGE content. (There are some variations allowed here but this isn’t important right now.) We also have to set the CONTENT-LENGTH to the number of bytes we’ll be sending back. With that done, we can grab our requested resource and seek to the requested bytes and transmit what was expected. Pretty straight forward. Even for multiple ranges, the process is the same except we instead return multipart messages with a boundary and independent CONTENT-LENGTH values for each range.

The HttpResponseMessage will let us set these headers and even construct an object for the Content property to use when the message is relayed. But now we no longer need to worry about all this processing with the use of the ByteRangeStreamContent class. Given a Stream of what is requested, the HttpRequestMessage.Headers.Range object and the media type of the resource, all this will be taken care of for us with a few exceptions. See below for the requirements of using this class.

System.Net.Http.ByteRangeStreamContent

Serving the resource is easy. Assuming you can retrieve your resource in a compatible format for the object, using it is as simple as passing the Stream, Range (HttpRequestMessage.Headers.Range) and media type to the constructor and then assigning the object to the Content property of your HttpResponseMessage. Easy as that.

Well, that assumes a range is actually specified. If not, you should try using a StreamContent object instead. Here though you have to be careful how you set the header for the content type. I found that specifying the type as part of the HttpRequestMessage.CreateResponse call results in an error which looks like is due to Web API trying to format your output using a stream formatter registered to handle that type. See caveats below for more information. Basically just specify your media type in HttpResponseMessage.Content.Headers.ContentType.

The Code (or: What You Really Wanted)

In your Web API controller, your action can be very simple.

[Route("api/video/{filename}")]
public class VideoController : ApiController
{
  public HttpResponseMessage Get(string filename)
  {
    var decodedFileName = Uri.UnescapeDataString(filename);
    var vidFile = File.OpenRead(Path.Combine(@"D:\Videos\", decodedFileName) + ".m4v");

    return new ProgressiveDownload(Request).ResultMessage(vidFile, "video/mp4");
  }
}

The ProgressiveDownload class takes care of deciding if the request is a RANGE or full request as well as handling bad requests and responding appropriately.

public class ProgressiveDownload
{
  public bool IsRangeRequest
  {
    get
    {
      return _Request.Headers.Range != null &&
      _Request.Headers.Range.Ranges.Count > 0;
    }
  }

  public HttpResponseMessage ResultMessage(Stream stream, string mediaType)
  {
    try
    {
      if (IsRangeRequest)
      {
        var content = new ByteRangeStreamContent(stream, _Request.Headers.Range, mediaType);
        var response = _Request.CreateResponse(HttpStatusCode.PartialContent);
        response.Content = content;

        return response;
      }
      else
      {
        var content = new StreamContent(stream);
        var response = _Request.CreateResponse(HttpStatusCode.OK);
        response.Content = content;
        response.Content.Headers.ContentType = MediaTypeHeaderValue.Parse(mediaType);

        return response;
      }
    }
    catch (InvalidByteRangeException ibr)
    {
      return _Request.CreateErrorResponse(ibr);
    }
    catch (Exception e)
    {
      return _Request.CreateErrorResponse(HttpStatusCode.BadRequest, e);
    }
  }

  public ProgressiveDownload(HttpRequestMessage request)
  {
    _Request = request;
  }

  HttpRequestMessage _Request;
}

ByteRangeStreamContent Requirements

The ByteRangeStreamContent class has only a few requirements.

  • A System.IO.Stream object containing the requested resource is needed.
  • The Stream needs to support seeking.
  • The Stream needs to represent the full resource (possibly just enough to cover the last requested byte).
  • The HttpRequestMessage.Headers.Range must be non-null.
  • The Range header’s Ranges property must have at least 1 defined range.
  • The requested ranges can’t be for more bytes than the resource stream contains.

While the library I wrote was a little more flexible with respect to the resource object types, the tradeoff of supporting only Stream is worth it to use classes seen, used, tested and reviewed by a much larger audience. This limitation is pretty minor too as your response will already need to be transformed into a stream anyway so whatever you’ve got must be able to support that. The only quirk is that usually response streams don’t need to support seeking which is a requirement for ByteRangeStreamContent.

I also point out the resource stream needs to be complete. This is because the library will be seeking over the resource according to the requested ranges in order to generate the output. A change request I got for my library was to support reading only the necessary data and sending that out rather than opening a stream for the full data. I wasn’t sure what this would buy until the user pointed out they are reading their resource data from a WCF stream which does not support seeking and would need to read the whole stream into a MemoryStream in order to allow the library to generate the output.

That limitation still exists in this specific object but there is a workaround. Instead of using a ByteRangeStreamContent, you could instead use a ByteArrayContent object instead. Since the majority of RANGE requests will be for a single start and end byte, you could pull the range from the HttpRequestMessage, retrieve only the bytes you need and send it back out as a byte stream. You’ll also need to add the CONTENT-RANGE header and set the response code to 206 (PartialContent) but this could be a viable alternative (though I haven’t tested it) for users who do not want or can’t easily get a compliant stream object.

Even if you can’t get a compliant stream and you have to support multiple range requests, you can still satisfy this without much more work by using the MultipartContent class. When instantiated, you simply add HttpContent-derived objects to it and let it generate the message boundaries for you. You will still need to add the CONTENT-LENGTH header and I assume this will need to be on the MultipartContent object itself as this header should be in the main response headers, not on the parts themselves (unlike CONTENT-LENGTH which needs to be specified for each part).

Just note I haven’t tried this at all since I am just serving local files which are easy to serve. File.OpenRead returns a FileStream object which supports seeking and the request ranges usually cover most of the file (unless I specifically skip around) which doesn’t lend itself to optimizing by reading only specific bytes.

Caveats

One strange thing I encountered while writing a demo library was the curious responses generated when I specified the HttpContent in the HttpRequestMessage.CreateResponse call. When I pass in the content here, the output isn’t generated correctly but if I just create the response and assign the content to the Content property, it works correctly. I don’t know if I’m misusing the CreateResponse method or if there’s a bug hidden in there.

return _Request.CreateResponse(HttpStatusCode.PartialContent, content);

This results in a strange response of content type application/json and a length of 122 bytes. However, this works correctly (where “content” is a ByteRangeStreamContent object):

var response = _Request.CreateResponse(HttpStatusCode.PartialContent);
response.Content = content;
return response;

I also didn’t put much effort in to checking the HTTP specifications for the exact error codes I should be using when a bad range request is made. I did see there was a special overload of CreateErrorResponse which takes an InvalidByteRangeException object which the ByteRangeStreamContent will throw so I’ve been letting that take care of setting the status code (which I assume is just 416 (RequestedRangeNotSatisfiable). As for other request errors, I just return a 400 (BadRequest) response and assume things will work out. I would double check that before I used it on the open web but as this is purely a home project on my local network, I’m not too worried about being a bad citizen. If you’re going to make a public endpoint, you should always check to make sure you’re satisfying the HTTP specifications.

Going Further

These classes make it very easy to respond to a range request appropriately and indeed I’ve been using basically this same code for days to great success on iOS 7 and 8. Watching Chrome request the resources shows it too is generating a RANGE request presumably not because of low memory but to see if it’s possible to allow seeking within the video stream without pre-loading the whole video. So what else could there possibly be to do?

Modern media players have support for requesting different versions of video files based on resolution of the device and bandwidth available. iOS devices and many JavaScript HTML 5 Video players can smartly request different resources in a smooth way that doesn’t affect the user’s viewing experience. This lets a device such as a phone request a lower resolution video for its smaller screen or for browsers to drop to lower resolution videos if the network bandwidth can’t support a full resolution stream without excessive buffering. You can see this latter case with Netflix which usually starts its stream with a lower quality video segment and steps up from there if the device can support the bandwidth and use the resolution.

One other thought: the request headers can have a lot of other values that could affect how you can respond to the request. For instance a browser could issue any number of conditional headers to ask if the resource has changed since a certain date or if the ETAG has changed for example. This doesn’t change how you handle responding to valid requests but you or the client browser might employ caching which could help you reduce processing and bandwidth if your resource hasn’t changed. This should be relatively easy to accomplish by creating a Resource Retriever class that does this checking and comparing for you and returning either the resource for streaming or a status code like 304 (NotModified) which you can send back to the browser. For my purposes this will almost never be useful as browsers tend not to cache large video files and mobile devices won’t have the memory for it. That’s just a long way of saying I haven’t written this but it’s an exercise I leave up to the user.

Happy Streaming,
-Erik

Advertisements

6 responses to “Progressive Download Support in ASP.NET Web API

  1. Pingback: Creating NUnit Tests in Visual Studio 2013 | Erik Noren's Tech Stuff

  2. Pingback: Web Api Progressive Download on GitHub and NuGet | Erik Noren's Tech Stuff

  3. Oh my god you are a life saver, thank you so much, this worked immediatley for me and my plans to stream my family videos.

    Thanks!!

  4. Pingback: Video Streaming for mobile clients via ASP.NET Web API 2 [Tutorial] - Eloy's blog - Site Root - StudentGuru

  5. George Lanes

    This will work with asp.net core 1.1?

    • The NuGet package definitely won’t. Even though ASP.NET Core can run on .NET Framework, there are library differences and missing functions this library used. Looking at the Issues I see this question was asked and it looks like the StaticFileHandler middleware supports range requests. If your media can be directly accessed by URL (http://server/path/file.mp4) then you should be all set. Otherwise it looks like support was being worked on but I’m not entirely sure where it ended up. You can follow the discussion on GitHub and see where it went. I honestly thought range request support was going to be an early priority given the goal for efficient and response applications; I’m sure if it’s not there in 1.1 then it’ll be along soon or at least all the necessary libraries to build the middleware will be. Feel free to ping me on Twitter if you want more extended discussion: @ErikNoren.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s