How i built fortmcmoney.com – Part 4 – Video & Audio

In this fourth post of the series, i will talk about my experience in building the video and audio player on fortmcmoney.com. I will discuss about some fails and problems that occurs during the process.

Video playback in Flash

Playing video is one the most common task for a Flash Developer. In fact, video playback in Flash is, in my opinion, one of the reason why Flash is still alive these days. Video playback in Flash as made some progress in the past years by introducing StageVideo. StageVideo is designed exclusively to display high resolution video in Flash using a memory chip, in your computer or mobile device, specially designed to decode and render high definition video.

If you don’t know about StageVideo, you should check this tutorial video by Lee Brimelow about StageVideo.

StageVideo in Fort McMoney

From the beginning of the project, i knew i would have to work with StageVideo to display our large collection of videos clips. Most importantly, playback had to be smooth and use as low battery power as possible on iPad. StageVideo was the only option.

On desktop only, a fallback to flash.media.Video as been enabled for computers not supporting StageVideo.

MP4, F4V AND IOS

StageVideo works best with MP4 and F4V video. Encode your video .H264, then output it to MP4 or F4V and it should play smoothly without any problem, until you try to use F4V on iOS.

F4V is a video format owned by Adobe which encapsulate a MP4 file and some additional metadata in a single file. It’s like zipping a video and text file together.

Since this format is owned by Adobe, playing F4V on iOS is not working. The only choice left was using MP4. You would ask: « And what’s wrong with using MP4 instead of F4V? ». Nothing’s wrong in using MP4. The only advantages of using F4V over MP4 was the ability to use CuePoints, which i strongly need in Fort McMoney.

MP4 and CuePoints

CuePoints are time coded keys in a video who trigger events in your ActionScript application. I’ve search the web a lot to find something about MP4 and CuePoints compatibility, but found nothing.

So, i wrote my own CuePoints loader and parser to make it work the way i needed. My CuePoints files are simple JSON files defining CuePoint name and time in second.

When a video file is requested, i load the JSON file associated to it using URLLoader and parse it. Then, on EnterFrame, i’m checking for video’s current time and trigger corresponding cuePoint event.

Problem when mixing StageVideo and Starling

StageVideo is, in the Flash display list hierarchy, always displayed under everything. This also include Starling layer, who always as opaque background and displayed on top of StageVideo.

This was a pretty big problem since i needed to switch Starling’s visibility on and off during video playback.

Even worse, i’ve found that when using StageVideo, on iOS, and tried to display Flash display object on top of Starling, the rendering of the Flash display list will draw opaque black background on your objects. You can read the details on this bug here: https://bugbase.adobe.com/index.cfm?event=bug&id=3186454

For components like the mini dashboard, which is sticked to screen’s bottom and displayed over Starling or Video player, it was a huge problem.

The only way i’ve found to resolve this problem was to have 2 mini dashboards working simultaneously. One using Starling display list and the other one using Flash display list. Syncing them was, and still, problematic during the various actions happening when playing the game. If your mini dashboard never hide by itself off the screen, my apologies. 🙁

Audio and voice over

There is a lot of audio ambiances and narrative voices over in Fort McMoney.

Audio ambiances are MP3 files loaded asynchronously from our CDN and looped in code. When moving in the exploration module, you can feel background’s audio changing depending on your position in the panoramic picture. This effect has been achieved by loading 3 MP3 (left, center and right), mixed by an professional sound man, and changing their volumes accordingly on EnterFrame.

Voice over are MP3 files loaded asynchronously from our CDN and played when ready. I keep track, in a cookie, of which voice over as been played to prevent playing them twice when coming back to a location you have previously visited.

The trickiest part of voices over was to correctly set files paths into CMS. As the project is available in french, english and german, the project count around 500 voice over MP3 files. It’s a lots of files to handle and our content team battled hard to keep track of all these files during iterations and various rounds of corrections.

Conclusion

I hope you enjoy reading my experience with video and audio on fortmcmoney.com and that my fails and tricks will help you on your next project.

In the next post of the series, i will talk how i handled internationalization in fortmcmoney.com.

Thanks for reading

Posted under Fort McMoney, Flash / AIR
Tagged as Starling, Stage Video, Fort McMoney, Case Study