Today’s post continues our discussion from yesterday, where we used a source movie taken with a non moving camera that contains objects (pedestrians) moving within the static view scene. Yesterday we worked with the Temporal Scan Tracker to build SlitScan effects. Today we will look at some other temporal image processing effects that work well with static camera view source movies.
We’re still using the same source movie we used for yesterday’s post for today’s examples. It was taken out of the window of a car stopped at a traffic light. The camera does not move (except for some slight camera shake since it was hand held), and a series of pedestrians move within the view in 2 different directions over time.
Yesterday we shows how to configure a slit scan effect so that the movement of the slitscan aperture over time worked well to accent the pedestrian movement in a visually interesting way. Today we will focus on 3 different temporal image processing effects that work with motion in a movie in different ways. The first gallery image above was created with the Temporal Difference Matte effect. The second gallery image above was created with the Temporal Difference1 effect. The last 3 images in the gallery above were created using the Temporal Rank Filter effect.
All of these effects have different editable parameters that can be used to dramatically change the appearance of the resulting time based effect. This should be clear with the 3 examples shown for the Temporal RankFilter above. The 3rd, 4th, and 5th examples above were generated using the Median, Min + Max, and Median Max Distance settings for the Algorithm parameter in the Temporal RankFilter.
The Editor control panel for the first 3 effected gallery images above are shown in the gallery below. The last 2 effected images above use the same Editor parameter settings as the 3rd gallery image below, except that the Algorithm parameter was set to Min + Max, and Median Max Distance settings respectively as opposed to the Median setting shown below.
All of the effects shown above were created by using a processing window that spans 120 frames, and that starts at frame 1. We will explain how this time window of multitple frames used as input to the various temporal effects was configured below.
All 3 temporal effects have standard Size, Frame Inc (increment), and Symmetry parameters. Size refers to the number of different input frames used to generate the temporal effect. Frame Inc gives the spacing between input frames. So a Frame Inc value of 3 means that every 3rd input frame is used for input to the effect. The Symmetry value determines the positioning of the time window of frames used for processing in relationship to the current frame position of the source movie.
Note that the Source Movie (located in the Source Area palette) is set to frame position 1 for all 3 temporal effects. And that the total distance of the frame processing window we’re using in the movie is 120 frames in all cases (since it’s the Size times the Frame Inc values). We use 40 different input frames for the Temporal Difference Matte and Temporal Rank Filter effects, and 20 frames for the Temporal Difference1 effect. The Frame Inc spacing sets up all 3 effects to process a time window that spans 120 frames. The source movie has a total of 141 frames, so we are ignoring the last 21 frames of the movie.
We used a Symmetry setting of Forward for all of the effects above. Remember, the time window is based on the current frame position of the source movie. Since we are using a starting position at frame 1 we need to position the time window to move forward from the current source movie frame position. If our current source movie position was at frame 70 we could use the Center option for the Symmetry parameter.
All of these different effects are sensing motion in the source input frames for the effect in different ways, and generating their effect output based on analyzing the motion of objects in the video. Camera shake or other camera movement also is perceived as motion by these effects. You can use this to your advantage or it might totally ruin the effect you are going for, depending on what you are trying to achieve with a given effect.
If you want to just concentrate on the movement of objects with a totally static background, then you really should use a tripod when shooting your original source video. If your goal is to use one of these temporal effects to create a painterly look, then some camera shake might actually help you to achieve that goal.
As you can see, all of these different effects process motion in different ways. The Median temporal rank filter algorithm actually works to eliminate or reduce the effects of motion in the scene. The other effects or rank filter algorithms we used above are trying to incorporate motion taking place over the entire time window of input movie processing into the final output image. So the generated effect is visualizing the motion over time in different ways that can be seen in a static output image (depending on what effect or effect settings you are using). Motion over time is encapsulated into a single output image.
The Temporal Motion Rank, and Temporal Rank Neighborhood effects also work well with movie sequences taken by a static non moving camera that incorporate object movement within the static background video sequence. The best way to get a feel for what all of these different effects can generate is to try them out with different parameter settings. Note that I used the same time window size positioned in the input movie for all of the different effects above to better contrast the difference between the individual effects (as opposed to how changing the Size or Frame Inc parameters would change the visual appearance of the effect).
Obviously adjusting the time window’s size and overall temporal extent will change the visual appearance of the effected output as well, but to a large extent how you adjust those parameters is very dependent on what is going on in the source movie file you are processing. If there is a lot of fast movement in your source video sequence, you might want to have a fairly small time window for processing (small Size and a Frame Inc setting of 1). For other motion sequences, increasing the Frame Inc spacing to extend the input time window might be the key to capturing the details of movement that takes place over a longer period of time.
Again, experimentation is the best way to get a feel for what these effects can do with a particular source movie file. Again, we specifically chose a static non-moving camera video sequence, with a static background and objects moving in relationship to the static background. If we had used video taken while the car was moving we would have gotten very different effect output results for the different effects shown above.
In this example we generated static effected images from video input. But we could have also used these different temporal effects to process an entire movie sequence. Typically in the case of processing a complete movie file with a temporal effect to generate processed movie output, you want the time window you are processing to be fairly small relative to the overall movie length (the opposite of the approach we took above where our time window spanned essentially the entire short movie sequence).
