Fetcher - Making it even easier to integrate with Zencoder

One of the most commonly used features on Zencoder is notifications, which tells your application when outputs are transcoded, uploaded, and ready for viewing. Receiving these notifications is a pretty important part of the integration process, but as developers we often develop our sites on our own computers, accessing them at locations available only to the local machine, like localhost or 127.0.0.1, and Zencoder is unable to connect to them. At other times the development or staging server could be behind a firewall during the development process, cut off from the notifications that are needed to finish testing the application. Getting around this doesn’t typically leave many attractive options:
  • open a port on the local machine and make the in-development site available online, then send notifications to an IP address that might change from day to day
  • put the site on another server, going through the deploy process any time a change is made; or editing files live on the server
  • mock everything and run scripts to post to notifications to the local server
At Zencoder we don’t want to deal with this, and we doubt our users want to either. That’s where Fetcher comes in. Fetcher is a utility written in Ruby that runs on your local machine. It grabs your account’s notifications and posts them, one by one, to your locally running application.

Using Fetcher

To get started with Fetcher, you’ll need to have Ruby installed. If you’re on OSX you’re already good to go. If you’re on Windows you can use the RubyInstaller to get up and running. Once Ruby is ready, use rubygems to install Fetcher from the command line.
$ gem install zencoder-fetcher
That’s all it takes to get Fetcher ready to run. All you need is your API key and you’re ready to start notifying.
$ zencoder_fetcher <API KEY>
This will get the last 50 notifications for your account and post each one to http://localhost:3000. This is great if that’s where you’re handling your notifications, but let’s assume we need them posted somewhere else.
$ zencoder_fetcher <API KEY> --url http://localhost:3000/zencoder/notifications_handler
Now we’re posting notifications to http://localhost:3000/zencoder/notifications_handler, and hopefully everything’s working well in your application. If it’s not you should probably take a look at that. When your notification handling is working we’ll look at one more thing Fetcher can do to make life easier. When Fetcher is run in the examples below it just grabs the last 50 notifications, posts them, and then quits. It’d be nice to run Fetcher in a loop, watching for new notifications to become available, posting them as they are available.
$ zencoder_fetcher <API KEY> --url http://localhost:3000/zencoder/notifications_handler --loop
Passing the --loop option tells Fetcher to keep running, checking for new notifications every 60 seconds and posting them as they become available. If 60 seconds doesn’t work for you, the --interval option will let you adjust the frequency as necessary.

Creating API Requests

If you’re using Fetcher, you need to create an API request that has a notification in it, and that notification needs a URL it can post to. If you don’t have a server running you probably don’t have a URL you can use, or at least not one you can post to without creating unnecessary traffic. Rather than using some random URL for your notifications, use
http://zencoderfetcher/
as the URL for notifications. Notifications sent to this URL will automatically be marked as successfully posted without actually going out, so there’s no unnecessary requests going to your servers that they’re not expecting. Everything else about the notification will work like normal, and Fetcher will grab it and post it to your local application.

Full Integration Achieved

With Fetcher, integration with the entire Zencoder process is easy. Job requests are made just like normal, and when they’re finished, Fetcher will grabs the notifications and passes them along. No more worrying about only being able to test on an external server or waiting for the deploy to QA to complete.