Barcinski & Jeanjean

Picking up where we left off in our stereo photography article, we now have a total of 576 photographs we need to get into our website. In this article we will talk about the different approaches we tried and the solutions we eventually opted for.

Below a diagram describing the footage from the two photo shoots.

You have a number of options displaying those images in a website; Loading JPEGs individually, forget about it! You could embed the images on the timeline in an SWF. This really isn’t a very good option as your memory usage quickly grows beyond acceptable norms and really doesn’t perform very well due to the JPEG decompression it needs to do.

No, video is your best option. FLV embedded in an SWF with every frame encoded as a keyframe, that is. It handles memory usage beautifully and allows you to smoothly jump to any frame backwards or forwards.

Ok, so back to our diagram. Add to it the portfolio cases that need to be suspended around us and be masked when falling behind us and the obvious setup would be to have one video with the background and one video with an alpha channel of us and use the latter to mask the cases behind us.

But the problem with that is that video, especially at this resolution, is quite cpu intensive. So in the case of viewing it in anaglyph it would mean you end up with a total of four videos on stage and two of those with alpha channels, which makes them even heavier. The solution we opted for is to combine us and the background into a single video, leaving only two videos on stage when viewing in anaglyph and no alpha channels.

But then how do you achieve the masking of the cases? Well, we also exported PCT image sequences of the alpha channel from us keyed out. Those 144 PCT files per side were imported onto the timeline in the Flash IDE and converted into vector shapes using the Trace Bitmap function. To avoid RSI we automated this process by creating an JSFL script that loops through the timeline and traces every Bitmap it comes across. The result is two SWFs, one for left and one for right, with vector shapes defining the masking area for each single frame in the video. Click on the image below to see the left mask SWF used in the website. This method is way more performant and smaller in file size than encoding the images into JPEGs.

Each of the cases have a separate RenderLayer in Papervision3D. The mask SWF is also placed on its own RenderLayer with BlendMode set to Erase. Set the correct z of your mask layer and voila, you’ve got your masking.

In an attempt to reduce file size for the video we came up with the following idea; FLV is able to contain 4 channels; Red, Green, Blue and Alpha. When you view the website in normal mode, so not in anaglyph, the Red, Green and Blue channels of the right video are used. So that’s 3 channels, right? When viewing the website in anaglyph you use the same Green and Blue channels of the right video and only the Red channel of the left video. So while the left video contains all 3 channels, we only need one; the Red channel. In other words we only need a total of 4 channels; RGB of right and R of left. Meaning we could store those channels in a single FLV by encoding the RGB channels of right into their appropriate channels and encode the R channel of left into the unused Alpha channel.

This can easily be done in After Effects. Just make sure you encode the video using Straight Alpha and not Pre-multiplied Alpha. In After Effects this can only be set when using the Render Queue instead of simply using the Export Movie option.

Ok, so now we have just one FLV with all the data we need to render both normal view and anaglyph view. Great, right? Well, not exactly. The On2 codec really isn’t made with this use in mind. It uses most of the available bytes to encode the areas on the color channels that are going to be visible according to the alpha channel. Which really is a good thing when using the alpha channel for what it is supposed to be used for. Why spend precious bytes to nicely encode parts of the video that aren’t going to be visible anyway? But that kinda ruined our little plan.

Besides the poor encoding quality, this method would require to copy the necessary channels from the video onto a canvas. Verdict; much lower quality and less performant. So we went back to the two FLV setup. Shame for the redundant data being loaded but at least it performs well and looks good.

To keep video file size low, minimizing loading time, we started with a low bit rate video to be used when animating the panorama. As soon as you would stop scrolling a high quality JPEG was loaded and placed on top of the video. However, this caused a short freeze in the physics when you stopped scrolling. This was caused by Flash using all the cpu resources to decompress the JPEG into a bitmap once loaded. Also the transition between the low quality video and high quality JPEG was annoyingly obvious.

Since we couldn’t find a work around for the freeze and couldn’t live with it we decided to get rid of the JPEG stills. But that meant that we had to use way higher bit rates to encode the video to get acceptable quality at that resolution (1376×576). Once we found the appropriate bit rate and encoding settings we ended up with a video of roughly 16Mb per side! So we had to find a way to reduce the preloading time required to enter the panorama view.

The solution we came up with is illustrated in the following diagram:

We split the 144 frames circle into three separate SWFs. Requiring only the first set of 36 frames to be loaded before being able to grant access to the panorama view. While you can start exploring the panorama the second and third set continue loading in the background improving the smoothness of the turn as they load.

This solution reduced the required preloading time back to acceptable norms. To further reduce the annoyance of the wait we decided to add a little game to the preloader. Maybe more on that in a later post.