Tech Talk

Mobile video is growing like crazy. Zencoder customer PBS just announced that viewers watched over 88 million PBS videos on iOS devices in the month of November, 86 million for their PBS Kids app alone. That's a lot of video, and 89% growth since June. All of this video - like most mobile video today - was delivered using HTTP Live Streaming (HLS), an Apple-created, HTTP-based format for streaming H.264 and AAC video in an MPEG-TS container. HTTP Live Streaming works by segmenting a long video into short pieces, typically 10 seconds, and then providing a M3U manifest that lists each segment. The player reads the manifest and determines when to pull each segment in order to ensure seamless playback.  (If you want more in-depth info on HLS, check out our guide on best practices for iOS encoding.) The problem is that MPEG-TS is an inefficient format, especially at low bitrates. The MPEG-TS format can easily introduce 10%-15% of unnecessary overhead to a file compared to a format like MP4, which increases costs and decreases picture quality, MB-for-MB. And if you're deploying a video application to the App Store, HLS isn't optional - it's mandatory if you want to display video longer than 10 minutes. Apple has rejected countless applications from the App Store for not complying with this policy.

Last month, Zencoder encoded over 2 million videos. We're pretty sure this makes us the largest cloud transcoding service on the market. Some interesting facts: Our transcoding volume is more than 1% of YouTube's volume. We're doing a few percent, in fact. We think that's pretty cool. Over the last 12 months, we've grown by 15x. Since our launch 18 months ago, we've grown by more than 20% month-over-month. Based on public information, we believe we're the largest cloud-based encoding service around. By about 2x. Before we get ahead of ourselves, we're still small, all things considered. We're operating in a competitive, quickly changing industry. In other words, we're still at the beginning of the marathon. But now that we've been operating for 18 months, it's interesting to stop and reflect on what has contributed to our growth, for better or for worse. Looking back, here are the three things that most contributed to our success.

Every API service should include request rate limiting. Up until recently, Zencoder didn't have it. This was less than ideal; it left us exposed to various types of problems. The good news is that as of today, we have full rate limiting. Why rate limit API requests? Rate limiting is important for a few reasons. First, one of the golden rules of multi-tenant architecture is that one user's behavior shouldn't affect someone else. If you're using Dropbox to store files, and someone else uploads a few petabytes of data, you shouldn't get an error saying Dropbox doesn't have any space left for you. Or if your neighbor floods his cable internet with BitTorrent traffic, your own service shouldn't be degraded because of this.

Speed has been part of Zencoder's DNA ever since the first line of code was written. Having been in the video transcoding industry for a few years, we knew that speed was important to most customers. UGC sites don't want to make users wait for video. Professional publishers want to make sure that they can get content online as quickly as possible. And broadcasters want to publish breaking news immediately - one of the reasons that PBS moved to Zencoder. So that's why Zencoder is architected for speed. We've known for a long time that Zencoder is fast - we've optimized every corner of our system for speed, and anecdotally, customers tell us that we beat their previous on-premise systems, or competitive cloud services, by a wide margin. But we haven't really known how fast. So we decided to find a reputable third party to do a performance benchmark of cloud-based transcoding providers. CloudHarmony came quickly to mind. We'd seen their previous work benchmarking cloud computing servers and CDNs, and they seemed like a good fit. We reached out to them, and (full disclosure) Zencoder sponsored a project to benchmark leading cloud-based encoders.

At Zencoder we take pride in making it easy to integrate with our API. In addition to being meticulous about our documentation, we also have an API Request Builder to help developers quickly and easily integrate with Zencoder's API.  With the Request Builder, you can generate and execute example JSON API requests through an easy drop-down interface. Beyond the JSON, it's also handy for testing Zencoder's capabilities before fully integrating. And now it's even easier to test the full capabilities of our API with the Request Builder by adding support for multiple outputs as well some of our advanced settings.

One of the most commonly used features on Zencoder is notifications, which tells your application when outputs are transcoded, uploaded, and ready for viewing. Receiving these notifications is a pretty important part of the integration process, but as developers we often develop our sites on our own computers, accessing them at locations available only to the local machine, like localhost or 127.0.0.1, and Zencoder is unable to connect to them. At other times the development or staging server could be behind a firewall during the development process, cut off from the notifications that are needed to finish testing the application. Getting around this doesn’t typically leave many attractive options: open a port on the local machine and make the in-development site available online, then send notifications to an IP address that might change from day to day put the site on another server, going through the deploy process any time a change is made; or editing files live on the server mock everything and run scripts to post to notifications to the local server At Zencoder we don’t want to deal with this, and we doubt our users want to either. That’s where Fetcher comes in. Fetcher is a utility written in Ruby that runs on your local machine. It grabs your account’s notifications and posts them, one by one, to your locally running application.

Zencoder is up for a Streaming Media Europe "Reader's Choice" award.  We would greatly appreciate your help in securing it.  In return, we promise we'll continue to listen to you and keep trying to pump out a great product:  http://www.streamingmediaglobal.com/ReadersChoice/2011/Vote.aspx