Someone recently asked me some questions about the Temporal Difference1 effect, which is a relatively new temporal image processing effect in Studio Artist 4.03 that was added after the initial 4.0 release and is not documented in the User Guide pdf. This post will discuss the Temporal Difference1 effect, and contrast it to the much simpler original Temporal Difference effect.
The 3 control panels in the gallery below correspond to the Editor settings for the 3 effects shown above. The first 2 are using the Temporal Difference1 effect. The third is using the much simpler Temporal Difference effect.
Temporal Difference
Let’s start by looking at the 3rd effected image and the third control panel, which shows off the simple Temporal Difference effect. Note that the Temporal Difference effect has no editable controls other than the standard generic Mix, Composite, and Composite Space parameters associated with every temporal image operation effect. All the Temporal Difference effect does is look at the difference between the current frame and the previous frame. So it provides a way to visualize any motion taking place between adjacent frames.
If your video camera is stationary, then the edge image generated by the Temporal Difference effect visualizes the edges of moving objects in the scene. If the camera is shaking or panning, then the Temporal Difference effect is going to pick up all of that movement in the entire scene, so in general it’s going to be a less useful effect for those kinds of source video. In general, you want to use it with stationary camera video shots.
The Temporal Difference is actually available in many different imager processing effects as parameter modulation options. Look at the Adaptive Filter ip op for example. The Modulation parameter contains a Src Movie Frame Difference option. This would be a black and white version of the Temporal Difference effect. So in the context of the Adaptive Filter ip op, you could adaptively modulate the Filter Size or Orientation based on the movement of objects in the scene. Making the filter more dramatic at the edges of moving object (or vice versa).
By routing the Temporal Difference output to the current Selection buffer, you can also do interesting things based on movement in the source movie. Like constraining paint strokes to only get drawn in areas of the canvas where there is object movement taking place when processing a video file with the paint synthesizer.
Temporal Difference1
The first 2 effected images at the top of the post and their respective first 2 Editor control panel screenshots in the gallery above both use the Temporal Difference1 effect. As you can see when you look the the Temporal Difference1 Editor control panel, there are a number of user adjustable parameters associated with this more complex temporal difference effect.
Like the much simpler Temporal Difference effect, Temporal Difference1 is looking at the difference between movie frames. But you can control how many frames are processed by the effect (Size), the spacing between the comparison frames (Frame Inc), the kind of different calculation used (Offset), how the computed frame differences are combined together to build the final effected output image (Output), and the direction from the current source movie frame index traveled to compute the specified frame differences (Symmetry).
The 2nd effected image above shows off a colored edge effect similar to the original Temporal Difference effect, but generated by processing the difference between multitple frames to compute the final effect output. The movement of the walking woman in my source movie takes place over 40 frames until she walks off the left side of the frame image.
I setup both the first and second effects above to process an input frame window that occupies a 40 frame time period, starting from the current source movie frame index (located at frame 1) and moving forward for 40 frames. Both effects only process 10 frames (Size), but the spacing between processed frames is set to 4 (Frame Inc), so Size times Frame Inc equales a 40 frame time period.
The first effect image above is using a different Output and Offset parameter setting, so rather than generating a colored edge effect associated with movement in the processed video frames it tries to overlay the movement on top of the base frame image.
The Temporal Difference Matte effect is a more sophisticated effect for overlaying or visualizing motion across multiple frames into a single static image. It’s another interesting temporal image processing effect to use if your source movie consists of a static camera view with motion taking place inside of that static view field.
Depending on what you are trying to achieve with your temporal processing, you might want to make sure you use a tripod to hold your video camera perfectly still when taking the original video shot. For other artistic effects, some camera shake might add to the artistic feel of the effect. Which approach is the right one depends on what you are trying to achieve artistically, as well what is going on in your source video footage.
