3d camera tracker ошибка

Background of problem.

Hi all,

Just recently started to enjoy the camera tracking and motion tracking features in AE, and it seems pretty simple to do. However I have some drone footage shot in 4K that I dropped into PP timeline that was 1080 sequence, it all is fine in PP, but when I replace with AE comp through dynamic link im not able to see all the image, so scale back to 50% in AE bit then get the error about layer

matching.

Successful solutions :

My first attempts where in native 4K both PP  and AE

this seemed to work fine, but as it takes soooo long to render, so I created a 1080 PP and then tried to replace with AE doing camera tracker and have the problem.

Questions :

So do I transcode the footage to 1080 first?

Will moving the camera create what i need in relation to seeing more image?

Dont really want to render out to 4k when no one will watch it on that and the majority of other footage is all 1080, in the PP project.

Thoughts ?

The 3D Camera Tracker in Adobe After Effects is a powerful tool that lets you analyse your footage and extract information about the movement of the real life camera used to film the scene. Assuming the tracking data is spot on, you can then place other visual elements in the 3D space of your scene and they will follow the movement of the camera correctly.

I have a full tutorial on how to use the 3D Camera Tracker in Adobe After Effects as a part of my 3D Integration VFX tutorial series.

As great as the 3D Camera Tracker is, there are a number of common problems that people encounter when trying to track their footage and this tutorial will explain what is going wrong and how you can fix it.

How the 3D Camera Tracker Works

Before we look into why the 3D camera tracker does not work, we first need to understand how it does work.

The 3D Camera Tracker works by analysing your footage frame by frame and detecting features that it can track. A feature is a clearly distinguishable visual shape in your clip that represents a fixed location in space. Moving objects like people and cars are not features and tend to interfere with the 3D camera Tracker.

The tracker will try to find as many features as possible in your footage and track their movement from frame to frame. This generates a cloud of 2 dimensional track points on your footage.
Using this cloud of 2D track points, the 3D Camera Tracker then tries to ‘solve the camera’. It will try to infer the 3D position of these track points based on how they move as a group and then derive the movement of the camera from this data.

In the screenshot below you can see that the 2D track points are all the same size as they have no 3D position. On the right side the size of the solved 3D track points represents how far they are from the camera.

Now that we understand the basics of how the 3D Camera Tracker works, let’s look at the most common scenarios of why it might fail!

3D Camera Tracker Fails To Solve Camera

One of the most common problems that people encounter is that the 3D Camera Tracker ‘just does not work’. You import a piece of footage, throw the 3D Camera Tracker onto the clip and then you get the following error message:

There are 2 likely reasons for why that is happening!

The Camera Movement is too fast

Have a look at the following image sequence, taken from my After Effects building destruction tutorial.

During the explosion, the cameraman dives for cover behind the concrete barrier and the camera’s movement is all over the place. Notice that the individual frames of the footage are extremely blurry and there are no features, no distinguishable shapes for the tracker to lock on to. Even if an accidental feature was found, the camera’s movement is so fast that the feature jumps around the screen, making it impossible for the 3D Camera Tracker to follow its movement.

If your footage includes very fast camera movement with blurry frames and undefined shapes, the 3D Camera Tracker will simply not be able to follow the movement of any features and therefore fail to analyse your footage. If it manages to analyse the footage and produce a cloud of 2D track points it might still fail to solve the camera if there are not enough 2D track points to infer the 3D camera movement from.

In this situation your only option is to re-shoot the scene or work around it. I ended up tracking the first half and the second half of my building destruction video separately because I could not track the middle part where the camera’s movement was too fast.

Moving Foreground Elements

Even if you footage is nice and crisp and there is no fast camera movement, elements in the foreground can interfere with its operation.

You will get the best results when you have no moving objects at all in your scene. That way the 3D Camera Tracker will not track the movement of any moving objects in your scene thinking that they are fixed features. If you have a scene with a large moving foreground element, it can easily invalidate your tracking data. Here is a screenshot of my fireworks tutorial:

The 3D Camera Tracker will pick up my movement and think it’s tracking the movement of a static element in your scene, a feature. To avoid this happening, you have to first mask out any large moving foreground elements to remove them from your footage.

Then pre-compose the layer including the masks and apply the 3D Camera Tracker to the resulting layer.

This way you are giving the tracker only the static geometry of your scene to track and it is much more likely to give you a good track.

But alas, even if all goes well with the tracking, there are yet more problems that you might encounter!

No Depth Information from a Tripod Pan Solve

If you’ve been using the 3D Camera Tracker a couple of times I bet that you have come across the following issue before:

‘No depth from a tripod pan solve’ shows up at the bottom of the target icon and the icon sits flat on your footage. You may also notice that all of your track points are the same size, indicating they are all equal distance from the camera. What is going on?

When you walk along the street you may notice that static objects at different distances seem to be moving at different speeds. The further they are away, the slower they seem to move. The closer they are, the faster they seem to move. This is called the parallaxing effect.

The 3D Camera Tracker uses this effect as a basis to figure out how far object are from one another in your 2 dimensional footage layer. However, if the camera does not move around the scene very much, e.g. it is mainly a panning movement, the tracker cannot derive any depth information because there are no parallaxing objects in your shot.

In those cases, the 3D Camera Tracker might determine that it is a panning shot and not give you any depth information with the warning ‘No depth from a tripod pan solve’.

But that isn’t a reason to panic as this warning is usually not a problem. Why?

Because the camera does not move around within the depth of the scene, you can attach objects to the ‘flat’ track points or use the 2D motion tracker I demonstrated in this Adobe After Effects tutorial to link other layers to your camera’s movement and it will almost always work just as good.

So don’t freak out when you see this warning, it’s simply telling you your scene does not contain any depth information – and for most tracking scenarios it won’t have to.

Finally, let’s look at some properly tracked footage and what you can do with it. And even here, there might be a few common problems that are worth exploring.

Correctly Tracked Scene

Here is a screenshot of a properly 3D tracked clip from my tutorial on how to blow up a motorcycle in Adobe After Effects.

Note that the track points are of different sizes, indicating that the 3D Camera Tracker was able to derive depth information from the scene and correctly place the track points at different distances from the camera. The smaller a track point appears, the further away it is from the camera.

With a properly tracked clip, you can place other elements into the 3D space of your scene and they will follow the camera’s movement as expected.

All you have to do is select the tracked layer, select the 3D Camera Tracker effect and hover your mouse over the scene. A red target marker will appear and it will be placed on a ‘flat’ surface in your 3D scene.

You can select a number of track points that define a plane in 3D space or select a single track point, right click and create different elements and an actual After Effects Camera.

This allows you to place Null markers, Solids or Text elements in your scene. Note that the creation of the camera is necessary (and therefore a part of any of the above options) to ensure that the 3D layers in your composition align with the movement of the camera you used to film the scene.

Here is a screenshot of some text I placed in the middle of the street. Remember to enable motion blur on the layer and the composition so that the placed element blends in more naturally when the camera is moving.

Cannot solve for this frame

Even if the analysis and solve phase was successful, you may encounter a few frames that give you the following error message:

If you look at the actual frame you will see that it is rather blurry and there are very few features for the 3D Camera Tracker to latch on to. Again, this is hard to fix afterwards, but can be avoided if you plan out how to shoot your footage.

I hope this tutorial gave you a solid overview of the 3D Camera Tracker in Adobe After Effects and showed you how to resolve some of the common problems that you may encounter when tracking your footage.

The 3D Camera Tracker in Adobe After Effects is a powerful tool that lets you analyse your footage and extract information about the movement of the real life camera used to film the scene. Assuming the tracking data is spot on, you can then place other visual elements in the 3D space of your scene and they will follow the movement of the camera correctly.

I have a full tutorial on how to use the 3D Camera Tracker in Adobe After Effects as a part of my 3D Integration VFX tutorial series.

As great as the 3D Camera Tracker is, there are a number of common problems that people encounter when trying to track their footage and this tutorial will explain what is going wrong and how you can fix it.

How the 3D Camera Tracker Works

Before we look into why the 3D camera tracker does not work, we first need to understand how it does work.

The 3D Camera Tracker works by analysing your footage frame by frame and detecting features that it can track. A feature is a clearly distinguishable visual shape in your clip that represents a fixed location in space. Moving objects like people and cars are not features and tend to interfere with the 3D camera Tracker.

The tracker will try to find as many features as possible in your footage and track their movement from frame to frame. This generates a cloud of 2 dimensional track points on your footage.
Using this cloud of 2D track points, the 3D Camera Tracker then tries to ‘solve the camera’. It will try to infer the 3D position of these track points based on how they move as a group and then derive the movement of the camera from this data.

In the screenshot below you can see that the 2D track points are all the same size as they have no 3D position. On the right side the size of the solved 3D track points represents how far they are from the camera.

Now that we understand the basics of how the 3D Camera Tracker works, let’s look at the most common scenarios of why it might fail!

3D Camera Tracker Fails To Solve Camera

One of the most common problems that people encounter is that the 3D Camera Tracker ‘just does not work’. You import a piece of footage, throw the 3D Camera Tracker onto the clip and then you get the following error message:

There are 2 likely reasons for why that is happening!

The Camera Movement is too fast

Have a look at the following image sequence, taken from my After Effects building destruction tutorial.

During the explosion, the cameraman dives for cover behind the concrete barrier and the camera’s movement is all over the place. Notice that the individual frames of the footage are extremely blurry and there are no features, no distinguishable shapes for the tracker to lock on to. Even if an accidental feature was found, the camera’s movement is so fast that the feature jumps around the screen, making it impossible for the 3D Camera Tracker to follow its movement.

If your footage includes very fast camera movement with blurry frames and undefined shapes, the 3D Camera Tracker will simply not be able to follow the movement of any features and therefore fail to analyse your footage. If it manages to analyse the footage and produce a cloud of 2D track points it might still fail to solve the camera if there are not enough 2D track points to infer the 3D camera movement from.

In this situation your only option is to re-shoot the scene or work around it. I ended up tracking the first half and the second half of my building destruction video separately because I could not track the middle part where the camera’s movement was too fast.

Moving Foreground Elements

Even if you footage is nice and crisp and there is no fast camera movement, elements in the foreground can interfere with its operation.

You will get the best results when you have no moving objects at all in your scene. That way the 3D Camera Tracker will not track the movement of any moving objects in your scene thinking that they are fixed features. If you have a scene with a large moving foreground element, it can easily invalidate your tracking data. Here is a screenshot of my fireworks tutorial:

The 3D Camera Tracker will pick up my movement and think it’s tracking the movement of a static element in your scene, a feature. To avoid this happening, you have to first mask out any large moving foreground elements to remove them from your footage.

Then pre-compose the layer including the masks and apply the 3D Camera Tracker to the resulting layer.

This way you are giving the tracker only the static geometry of your scene to track and it is much more likely to give you a good track.

But alas, even if all goes well with the tracking, there are yet more problems that you might encounter!

No Depth Information from a Tripod Pan Solve

If you’ve been using the 3D Camera Tracker a couple of times I bet that you have come across the following issue before:

‘No depth from a tripod pan solve’ shows up at the bottom of the target icon and the icon sits flat on your footage. You may also notice that all of your track points are the same size, indicating they are all equal distance from the camera. What is going on?

When you walk along the street you may notice that static objects at different distances seem to be moving at different speeds. The further they are away, the slower they seem to move. The closer they are, the faster they seem to move. This is called the parallaxing effect.

The 3D Camera Tracker uses this effect as a basis to figure out how far object are from one another in your 2 dimensional footage layer. However, if the camera does not move around the scene very much, e.g. it is mainly a panning movement, the tracker cannot derive any depth information because there are no parallaxing objects in your shot.

In those cases, the 3D Camera Tracker might determine that it is a panning shot and not give you any depth information with the warning ‘No depth from a tripod pan solve’.

But that isn’t a reason to panic as this warning is usually not a problem. Why?

Because the camera does not move around within the depth of the scene, you can attach objects to the ‘flat’ track points or use the 2D motion tracker I demonstrated in this Adobe After Effects tutorial to link other layers to your camera’s movement and it will almost always work just as good.

So don’t freak out when you see this warning, it’s simply telling you your scene does not contain any depth information – and for most tracking scenarios it won’t have to.

Finally, let’s look at some properly tracked footage and what you can do with it. And even here, there might be a few common problems that are worth exploring.

Correctly Tracked Scene

Here is a screenshot of a properly 3D tracked clip from my tutorial on how to blow up a motorcycle in Adobe After Effects.

Note that the track points are of different sizes, indicating that the 3D Camera Tracker was able to derive depth information from the scene and correctly place the track points at different distances from the camera. The smaller a track point appears, the further away it is from the camera.

With a properly tracked clip, you can place other elements into the 3D space of your scene and they will follow the camera’s movement as expected.

All you have to do is select the tracked layer, select the 3D Camera Tracker effect and hover your mouse over the scene. A red target marker will appear and it will be placed on a ‘flat’ surface in your 3D scene.

You can select a number of track points that define a plane in 3D space or select a single track point, right click and create different elements and an actual After Effects Camera.

This allows you to place Null markers, Solids or Text elements in your scene. Note that the creation of the camera is necessary (and therefore a part of any of the above options) to ensure that the 3D layers in your composition align with the movement of the camera you used to film the scene.

Here is a screenshot of some text I placed in the middle of the street. Remember to enable motion blur on the layer and the composition so that the placed element blends in more naturally when the camera is moving.

Cannot solve for this frame

Even if the analysis and solve phase was successful, you may encounter a few frames that give you the following error message:

If you look at the actual frame you will see that it is rather blurry and there are very few features for the 3D Camera Tracker to latch on to. Again, this is hard to fix afterwards, but can be avoided if you plan out how to shoot your footage.

I hope this tutorial gave you a solid overview of the 3D Camera Tracker in Adobe After Effects and showed you how to resolve some of the common problems that you may encounter when tracking your footage.