LA Noire - Why The Bodies Look Wrong

So Brendan McNamara has commented on what a lot of people have noticed about the dissonance between facial animation and body animation in LA Noire:
McNamara: why you thought LA Noire's characters were "dead from the neck down"

"That's because we had all this animation in the neck and all this animation in the face, but the clothes don't move. Once you get to the level that people can actually see that level of realism, then people expect to see clothes moving and the rest of the body moving in a way we can't replicate in video games."

I'm afraid I don't buy this at all. The problems stem from the way the performance is captured, and the fact two different techniques are being used to do this. Behind the Scenes of LA Noire

1. Body Capture
The body is captured in the traditional manner with actors working together in the studio. This is a proven technique, and is used by most AAA studios. There are different ways to get the facial information - Naughty Dog records only body performance, then hand animates the faces (using videos for reference). This creates a very stylized animation for the faces - not necessarily realistic. Compare this to Avatar, Heavenly Sword or TinTin, where facial performance and body performance are captured using the same technique - small dots that are tracked by the cameras. With the right density of information, a good solver and a decent facial rig, good results can be generated - but at this resolution there is the very real danger of the Uncanny Valley - the slightest error being magnified and ruining the whole effect (like the dead eyes of Polar Express).

2. Facial Capture
This is the unique and impressive tech part of LA Noire. The facial capture uses 3D scanning of the actual face, which is then mapped onto the character's face. In order to do this, the actor has to sit very still on a chair while a number of cameras are targeted at their face.

• This results in a very high resolution capture, but has several significant problems. 
• The actor must remain facing a certain direction, they cannot turn their head. 
• There is only one of these rigs at any one time, so actor's performances are captured in isolation. 
• They are required to act in a sitting position only. 

You can instantly see that this is going to have an impact on their performance. They cannot move around much, and they have no-one to reflect off.

3. Combining
This is where LA Noire falls down - every performance is effectively two performances. The body and face are combined, and this is why there is a dissonance.

It is very difficult to get ADR (Automated Dialog Replacement) to match when re-recording voice work (hence why it is often obvious, or is done when the character is not facing the camera). Effectively the entire performance is ADR'd across the whole game and this is why they don't match a lot of the time - humans are very good at picking up on human movements, so can instantly spot this difference.

No amount of clever work on the clothing will fix this issue - for this technology to really work, they need to be able to capture the whole performance at once.