“We are going to find out what digital cinema means. We are not there yet.”- Ang Lee
People like to talk a lot about the “future” of cinema. Is it high-frame-rate (HFR)? High-Dynamic Range (HDR)? 3-D? Virtual Reality (VR)? Augmented Reality (AR)? Multi-screen/Multi-View Environments?
It seems everyone is trying to create so-called “immersive cinema,” in order to figure out how to get people out of their home theaters and back into the Cineplex. As a themed entertainment designer, I’ve been creating immersive experiences in ride films and attractions for 20 years. I wouldn’t call any of the experiences “cinema,” though some of them may be “cinematic.” As a filmmaker and themed entertainment designer I see the fundamental difference between immersion and cinema. I first delved into these concepts in 2013 when I was on the Siggraph Panel and Presentation “From Big Screens to Big Screams.”
Cinema should suspend your disbelief and bring you into a world where you emotionally connect with the characters telling their story. Camera placement, editing, focus, staging, color, pacing, time, design and performance are all manipulated by the filmmakers to provide visual and emotional cues to tell that story in a very deliberate way. They can slow things down with slow motion, change perspectives, trick the audience with editing, and force you to see only what they want you to see. You watch the story unfold as they intend it. You are a spectator, not a participant.
Immersive experiences don’t want to suspend disbelief, they want to place you in a world, where you are one of the characters and it becomes your story. You are now a participant and not a spectator, so most traditional cinematic film-making techniques don’t apply, or don’t translate well. And why should they? If the story is happening to you there shouldn’t be a “camera.” You are the camera, free to look in whatever direction you choose, whenever you choose to, in real-time. There should be no motion blur, no cuts, no selective focus, no proscenium between you and the experience.
Sure, we’ve developed techniques for immersive ride films to direct your focus to a particular area or moment on-screen. We’re still trying to tell a story after all, but as an immersive experience it’s different for each person from their unique perspective. This is true whether you’re in an enormous projected environment like Kong Kong 360-3D, and Fast & Furious: Supercharged, or you’re experiencing a Virtual World with an Oculus. While it can still be an emotional ride, immersive experiences are more about the experience, and less about lasting emotional impact.
A major technical differentiating factor has been frame rate. Since sound was introduced to Cinema, films have been displayed at 24 frames per second. This was the lowest frame rate possible to maintain audio synchronization and allowed the studios to save money on film. This frame rate has since crept into the universal subconscious of film language. Its dreamlike blur helps draw us into the story world and suspends our disbelief. Peter Jackson’s The Hobbit which employed 48 fps in select locations, has shown us that most of us find it disconcerting to see “cinema” shot and projected at higher frame rates, even if it does help eye strain on stereoscopic imagery. Many describe it as fake or having a “soap opera” effect. The image becomes too clear, too real. Our subconscious notices that. We lose our suspension of disbelief, which takes us out of the story emotionally and intellectually.
Immersive experiences and ride films, however, benefit from higher frame rates. Ask any gamer. The more frames, the less motion blur, the more real everything appears. When you don’t want to suspend disbelief but place someone in the environment as if it’s real, this is what you need. This is why Universal’s attractions King Kong, Fast & Furious, and Harry Potter and Forbidden Journey are all projected at 60 fps. We wanted to simulate reality as closely as possible.
So if immersive experiences and cinema are so fundamentally different, why does everyone keep trying to make cinema more immersive? It’s simply about the question “What can we deliver at the cinema that you can’t get at home?” Most of these immersive techniques fail at least at first their attempts to transition from the theme parks to the multiplex. Why, because it means adopting a whole new visual language. VR is exciting for its interactivity, but when the viewer can look at anything how do you to tell a powerful and focused story? 3-D was initially a fad in the fifties. It attempted a resurgence a few times since then but didn’t really take hold until recently when technology matured and filmmakers began to use stereo space as a storytelling tool and not just a gimmick. Directors like Martin Scorsese and Ang Lee, took the extra dimension of depth and used it as they would color, blocking, and camera movement to provide additional subtext to the right story. The last bit is important. Certain stories work better with certain creative tools and techniques. In the hands of those who know how to wield it, 3-D is a powerful tool for story-telling that can add additional subtext and emotion to the story. Just like only certain films or sequences should be in black and white versus color or be silent or cacophonous. Imagine Schindler’s List in color or Gravity in 2D. The impact just isn’t the same.
3-D is now part of the cinematic language, but what about frame rates? Higher frame rates benefit 3-D by reducing eye strain but can take us out of the film. How do we combat that and create a new cinema experience worth getting off the couch? Well, Ang Lee and Doug Trumbull think they have the answer. Add even more frames. 120 fps to be exact.
Ang Lee and Doug Trumbull presented a demonstration at NAB of Ang Lee’s new film “Billy Lynn’s Long Halftime Walk.” The film was shot at 120fps and projected at 120 fps thanks to new laser projectors from my frequent collaborators at Christie Digital. While I didn’t make it to NAB this year to see it first hand, I’ve heard that it’s breathtaking. Despite the technical achievement, what I find truly interesting is what Ang Lee has to say about the process of making the film.
This is my virtual reality. I don’t want you to wander away. I want you to see my vision.
A lot of lighting and performance looks funny in higher frame rate. A lot of movie magic you have to re-think. I didn’t dare to put on make-up [on the actors], apart from the cheerleader, because you see everything.
This was a quest to go deeper into cinema. This type of dream requires a lot of collaboration. This requires a whole ecosystem. I hope movies can be entreating and spiritual. That is my dream. – Ang Lee
Ang Lee is talking about how he’s learning the language of HFR, and how just like with stereo imagery, slow motion, or other film techniques you need to learn to use it properly to add emotional subtext to enhance the story. Then and only then can the new technique succeed. (For a more in-depth transcript of the presentation check out this post from Celluloid Junkie.)
So what is the future of cinema? I envision a cinematic world where frame rates can ramp up or down dynamically in the middle of a film to suit a moment or scene for emotional impact. Where interactive experiences aren’t the whole experience, but a vital part of it. Where we use all of this wonderful new technology as well as our storytelling experiences of the past to tell new and wonderful stories of the future. So let’s stop worrying about the future of cinema, and just start making it. It’s the only way we’ll ever know.
For more on Billy Lynn’s Long Halftime Walk and the 120 fps demo check out these articles: