Every so often, a new product or service has an immediate impact on improving my day-to-day personal and professional routines. Dropbox – immediately useful. Square – great for merchants but clunky for the end user. Uber – fantastic. If you're not familiar, Uber lets you easily book, track and pay for a private car service from your mobile phone. While the benefits may not be immediately apparent, this means:
- Automated billing to my credit card and electronic receipts – no more “broken” credit card readers (San Francisco)
- Ability to request a car and track its estimated time of arrival – no more waiting curbside during shift change (New York)
- A much higher degree of confidence that the car is clean and the driver is professional – no more disgruntled drivers and ill-smelling automobiles (Everywhere)
While the granular qualities of the service – speed, distance, overall fare – are on par with or may be bested by traditional alternatives (e.g., a taxi service), the Uber experience is its core value and keeps me a loyal customer.
Video QoS, or quality of service, can be viewed through the same lens--service is experience. Video QoS is one of the more common topics I've discussed with publishers over the past 12 months. Publishers want to better understand a myriad of metrics for both live streaming and on-demand video, including:
- Player load and start times
- Average buffering time and views affected by buffering
- Average available bandwidth and playback bitrate
- Failed video starts
Several companies specialize in enabling publishers to collect this data, often used to understand video delivery and playback performance. Consequently, the analysis tends to focus on CDN capabilities.
While it's important for publishers to optimize this step of video consumption, QoS should be redefined as quality of video experience, or video QoE. Measuring and monitoring the engagement experience should be prioritized to drive what many publishers aim to accomplish: increased views per user, overall viewing time and – for many publishers – ad inventory.
Playback Performance
Publishers typically hide the complexities of rendition selection from users (or rendition selection is controlled by the underlying video player, e.g., Apple's native video player for HTTP Live Streaming). Several recent reports state that publishers need to provide 2.5Mbps or higher video renditions to optimize video playback performance.
This simplistic approach overlooks a number of factors that need to be taken into consideration, though:
- Some publishers provide users direct control of rendition selection based on known limitations, e.g., their target audience is affected by bandwidth caps. As a result, lower renditions may actually be by preference, not by necessity.
- We are becoming a mobile society. Publishers should consider the proliferation of Wi-Fi in airports, trains, cafes, public spaces, hotels and related areas. While bandwidth may be free (or reasonably priced), bandwidth and latency (especially for mobile 3G) is often inconsistent.
It's straightforward to optimize for users that have access to ample bandwidth, low latency, and devices with sufficient processing power, but publishers should be optimizing for mobile experiences, for experiences where conditions aren't ideal.
Publishers should focus on collecting contextual information to better understand the conditions under which their users are consuming their content, especially bandwidth (e.g., average bandwidth, bandwidth variance over time) and local performance (e.g., dropped frames, average framerate, framerate variance over time). Publishers should not only analyze their encoded renditions to ensure that the gaps between bitrate tiers provide an optimal adaptive playback experience, but publishers should also give users the ability to affect their rendition quality, opting for lower bitrate playback with consistent playback vs. higher bitrate playback interrupted by frequent buffering.
Replay
I first noticed the replay button when DVRs were becoming commonplace, often used as a necessary adjustment when seeking to skip over commercials. The presence of the replay button in video players seemed to be an antiquated carryover until I recognized several recurring use cases (some that were also commonplace in recorded broadcast content):
- A “wow” moment, such as a slick snowboarding knuckle grind or the final Coney Island "splash" scene in Cloverfield
- Interruption due to buffering
- Poor audio quality/balance (often with films that were poorly remixed for digital or television broadcast)
- User limitations (e.g., viewing through laptop speakers or with loud background noise, distractions from multitasking)
Publishers should track user-initiated replay events (either explicitly via a replay button or implicitly via the scrubber) to understand whether users are reacting positively or negatively to systemic issues with their player or content.
Advertising
Publishers that have instrumented their players to measure QoS typically focus exclusively on their video content but not on their advertising content. While their content is the primary focus for their viewers, their content experience is inextricably tied to the advertising experience. Poor quality – and especially problematic playback – can directly impact the video experience.
- Ad creative should be optimized for delivery for all platforms (desktop, mobile Web, native applications) in regard to available renditions and bitrates, format and protocol
- Ad requests (and requests for the underlying assets) should be analyzed for latency or failures (which may result in prematurely ending video playback)
CDN Performance and Self-Healing
CDNs are often the primary suspect when publishers identify systemic issues with playback buffering and failures. As publishers analyze these types of issues, they should review all other contextual factors as well as physical assets (e.g., formats, encoding profiles for renditions).
- Whether a regional CDN strategy fits into a global CDN strategy (i.e., selecting regional providers to optimize for delivery to a specific geography)
- Whether their video platform strategy can automatically detect and respond to - “self-heal” - detected issues (i.e., switching protocols, switching formats, or switching CDNs based on identified systemic playback degradation or failures)
What's Next
In broadcast syndication, you'll rarely see film credits roll in their normal speed or size. Unlike in a theater, the broadcast version is likely edited for content, commercials and – most commonly – duration (to ensure that the program fits into a predetermined time slot). In addition, it is common practice is to explicitly promote "what's next" programming. If publishers have content with credits or end slates, they should measure whether users are pre-emptively disengaging when a video is nearing completion. Publishers should explore the effectiveness of proactively displaying on-page or in-player elements to promote the next video.
QoS measurement is a never-ending process. It's easy to become inundated with data, but if publishers define QoS as improving QoE, it will become easier to focus on the discrete mechanisms of the video experience that enable loyal viewers to do as little as possible to consume as much as possible.