Photorealism In Video Games – A Worthy Goal

A response to Errant SignalPhotorealism

In his video Campster details why we should not be chasing photorealism in video games, I explain why we should.

Pipe Dreams

I have to chuckle every time someone uses this term in context to anything technological.

Not because the tem amuses me in itself (which it does, considering the context and etymology), but because it reminds me how history tends to not only repeat itself, but also how people still lack the imagination to extend their way of thinking past the here and now.

I will quote the first of Clarke’s three laws that has held true over the decades:

When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.

What baffles me is that Campster, a man of certain respectable age (i.e. older than me), sees Photorealism as a pipe dream, unachievable (or so fantastical and uneconomical as to be almost unachievable in our lifetime) while he has lived through the amazing technological revolution of video games in the past 20 years or so.

Ultima Underworld – The Stygian Abyss, the first ever full 3D first person RPG, 1992 (according to Wikipedia)

Witcher 2, most advanced graphics in an RPG as of 2012, sets computers on fire @Ultra

What can we expect in 2032?

Can you really, honestly, say that photorealism is unobtainable or uneconomical? A Witcher 2 level of fidelity was unobtainable and uneconomical in 1992, the level of fidelity of 2032 is unobtainable and uneconomical in 2012.

Campster brings up the hurdles that need to be taken: realistic mesh deformation, detailed animation, hair simulation, realistic cloth and physics interaction. They are all true, they are all hurdles that are hard to achieve, even harder to put together into a coherent whole, but claiming them to be inherently unreachable?

Inklings of solving those problems already are beginning to take shape:

Hair simulation:

Real-Time Eulerian Water Simulation:

Real-Time Soft-Body deformation:

Organically Grown Procedural Animation and Interaction:

Many more are working on other parts of photorealism.

Why, why are they all doing this? Do they want to produce the next Modern Warfare? Do they want to “pollute” our games with more brown filters?! Monsters!

Progress

The only way of discovering the limits of the possible is to venture a little way past them into the impossible.

~Clarke’s 2nd law of prediction

Wow this article certainly is Clarke-heavy.

The people working on photorealism are working on it because its the way forward, its progress.

You couldn’t have your Windwakers without dynamic shadows/lighting and depth of field/motion blur post processing effects, technology that was not designed for Windwaker, or artistically expressive games, but for photorealism and photorealistic games.

Developing techniques and solutions for 3D graphics is time and resource intensive, no singular artist, designer or developer can afford to do the RnD themselves. Not even the large publishers (except maybe Microsoft, Sony and Nintendo, hybrid technology companies) can afford to run their own graphics RnD department.

Every single one of the effects you showcase as being not photo-realistic, were developed because of photorealism. We take our ideas from nature, from photorealism. Neon glow effects and styles did not come from the ingenuity of the artist, they came from neon lights we see in real life.

You don’t see the old masters paint in the style of Tron, because the real life counterpart did not exist, the inspiration that made the style happen.

Photorealism is the single, most effective, driving factor for developing new technology and artistic games are benefiting directly from the solutions this research provides.

Video Game graphics are at the stage of medieval art, we are still trying to figure out our techniques for expression, finding the right tools and the right solutions. Only if we achieve photorealism can we move past it.

Abstract, expressionism, impressionism, Cubism, surreal, etc. art-styles only emerged after painting was firmly grounded in realism (or at least as close as the technology of a paintbrush can get to realism).

Medieval Uncanny Valley, Madonna de Santa Trinitá, Byzantium, 1285

Realism, 1854, Courbet

Photorealism, 2007, Baeder

Authenticity

No matter how artistically expressive your game is, our brains are wired a certain way, they see reality every day and expect things to behave in exactly (or approximately) the same way in simulations.

This is why we can tell that 3D graphics look “fake” and its the reason the Uncanny Valley exists. However, we can only take out certain aspects until something becomes less then simplified and becomes “fake” as well.

It has mostly to do with shadows, proportions, lighting and perspective, those are the things that our brain has an, lets say, intuitive understanding off and can immediately tell from fiction.

Color is far less problematic it seems, we still recognize black and white film as depicting photorealistic characters, and we don’t feel its somehow “off”. The problem starts with tonal changes and fake coloring when our brains reject the image as photorealistic.

Still photorealistic, tones are preserved, light and shadow is realistic. (c)Jenna Herman

Jenna Hermans photo fucked around with in Photoshop. The face loses its recognizable definition. Uncanny Valley.

3D graphics technology strives to be authentic, be it in Windwaker, Torchlight, or World of Warcraft.

Light casts shadows realistically, water reflects the environment, the perspective is realistic, the world is believable enough to our brain to not reject it outright but fantastical enough to entice us.

The most persuasive argument are the Toy Story movies. The enriched, higher fidelity and -physically realistic- lighting and shader models made Toy Story 3 the experience that it is and could not have been made with the distinctly less realistic technology of 1995.

Especially if you look at the fire-effects in the furnace at the ending, the dramatic physically authentic lighting with scattering, multiple dynamic sources, displacement and normal shaders. All this could not have been done unless someone worked on achieving the photorealistic effects in 3D graphics.

Subsurface Scattering and softbody meshes would not be present if it wasn’t for trying to make realistic skin (Shrek), refraction, caustics and photon scattering wouldn’t exist if someone didn’t want to simulate realistic water (Finding Nemo), realistic fur and hair simulation wouldn’t exist if it wasn’t for the desire to render authentic animals. etc.

All those techniques would not be here if photorealism didn’t motivate them to be created.

To ask to not pursue photorealism, is to ask to not have progress. Its to limit yourself to what we know and never take a step further into the unknown.

I do not advocate -only- photorealistic games, but the last time I checked, we didn’t have a problem with stylistically and artistically diverse games in the marketplace.

Sure, not very often from tripple-A publishers, but can you blame them? Tripple-A titles are the Blockbusters of the industry, they cater to what the majority of the public wants, and the majority of the public doesn’t watch experimental cinema, so why would they want to play it?

Advertisements

6 Responses to Photorealism In Video Games – A Worthy Goal

  1. Twiggles says:

    What I see as being an issue with striving towards photorealism is that, while it can and already has led to new, totally different aesthetic styles being developed, the vast majority of those style preference directly representational signs. In game X, a chair is designed into a room. We see that chair, and, through the myriad experiences with chairs we’ve had in our lives, associate those qualities with the chair we see in front of us, in the game. To ensure it is a chair, a “real” chair, I would need to be able to do all of the tests I have done in the past about chairs: can I pick it up to check its weight? Can I balance it on one leg? can I flip it over and put it on the table? These tests, since I’ve been able to conduct them successfully on prior chairs, would let me know that the in-game chair was a real one. Nothing less.

    To dial it back for a second, consider film. It would be strange to say that film isn’t “photorealistic,” but it clearly is. Most movie sets (due to green screen) don’t exist at all, and ones that do are primarily filled with props that do not have any of the properties of the things they represent. What allows movies to resonate with us on a real and relate-able level is that we have no control over the setting, the characters, or the action. With that in mind, we focus mainly on the actors/actresses, and if they deliver their dialogue “realistically” enough and convey their emotions “truthfully” enough, then by believing them we necessarily believe the world they occupy. Keanu Reeves was great in “The Matrix” because his perpetually-bewildered look (and his unremarkable, “anyman” appearance) was perfect for a man in the circumstances he was in.

    I agree with your idea about striving towards photorealism for technological ends, but for aesthetic ends photorealism carries with it its own bias, and depends upon a different sort of video game that has to be more limited in scope in order to be effective. To be “photorealistic” in a game is not simply a graphical issue: it necessarily encompasses every element of the game, from graphics to physics to ai to interface to, most importantly, our interactions.

    • tradamtm says:

      Why do we complain so much that CGI always looks fake in movies or when we can tell that a mask is a bit of plastic and not an actual face?
      It’s because we firstly ascertain visuals and the way we expect an object to handle that approximates something we are used to be dealing with in reality.

      I’ve been watching a lot of Wuxia Kung Fu movies and the swords are absolutely completely fake, i know this, i can see it and feel it from how they operate, but I’m still immersed.
      This has noting whatsoever to do with the performance of the actors or the dialogue (i watch them in Cantonese or Mandarin with subtitles and hance can not ascertain the value of acting since the language and mannerisms are so foreign)

      What you are suggesting is that because in games there is interaction we can’t achieve immersion or authenticity through photorealism, which I would object to.

      Also, I’m not entirely sure we can talk about photorealism by adding interaction to it. I mean we can stretch this definition of photorealism to infinity until we reach a point where “the matrix” or the holodeck is the only acceptable “photorealistic game”.

      Campster didn’t mention that in his video, but the easiest way to “discredit” photorealism would be to claim that controllers aren’t photorealistic since they rely on a physical metaphor to facilitate control instead of letting me move my arms and have tactile feedback of touch.
      (What about smell and taste?)

      However ultimately I feel that this (his) argument is insincere since its not what the word means or how it is used functionally today referring to games or CGI (what his rant was about).

      I see no no bias carried in photorealism for aesthetic ends any more than I see a bias carried by using fake swords in Kung Fu movies.

      Campsters rant was generally confused what he was addressing exactly, one time its the graphics with soft body collisions and movement, then its AI or physics.

      Abstracting a chair into a videogame doesn’t really matter if it’s done cell-shaded or realistically textured, with or without PhysX, its both recognized as fake, and always will be as long as we know its a simulation (view it on a screen, interact with a controller, etc.).

      Thats why we find it funny when a chair bugs out and rockets in to the sky Skyrim, because its not authentic behavior, but its not any worse than having a plumber go into a green pipe by squatting.

      What we ultimately strive for is authenticity (like I mentioned in my article) and authenticity becomes better the closer we get to emulating reality and it certainly will be achievable.

      • Twiggles says:

        I think my idea of “authenticity” and its ends may be closer to his, then, since he placed emphasis on the game as a “system” and how its authenticity, its ability to provoke sympathy (or any other emotional response) is based on how it creates its world, and then allows the player to interact in it. A game like Minecraft, for example, certainly isn’t authentic in its representation, but it certainly is in its sense of exploration and ability to evoke that primal urge we have to build things, to change the landscape around us. I’m not certain that a Minecraft that emulates reality would necessarily be better, not the least of which because it is partly those mind-boggling inconsistencies that make the game so much fun.

        In short, we should strive past photorealism, because it doesn’t even try to depict the thing as-is but rather how a camera would capture it. Some of the “realest” things we’re exploring in this decade (the existence of the Higgs-Boson and other subatomic particles) cannot even exist in the realm of photorealism. Things like x-rays and night vision are reduced for us into observable means, but they’re just approximations of what the “realism” is. I’m not doubting that some day we will achieve games that look like and react in accordance with what would be “photorealistic,” but by then our grasp of what is real will probably be much further advanced, meaning we’d still be behind.

        • tradamtm says:

          “I think my idea of “authenticity” and its ends may be closer to his, then, since he placed emphasis on the game as a “system” and how its authenticity, its ability to provoke sympathy (or any other emotional response) is based on how it creates its world, and then allows the player to interact in it. A game like Minecraft, for example, certainly isn’t authentic in its representation, but it certainly is in its sense of exploration and ability to evoke that primal urge we have to build things, to change the landscape around us. I’m not certain that a Minecraft that emulates reality would necessarily be better, not the least of which because it is partly those mind-boggling inconsistencies that make the game so much fun.”

          But Minecraft uses a more realistic (authentic?) representation than mining in World Of Warcraft where you click on a resource node to extract it without altering terrain.

          It uses realism, the physical removal of resource and world-shaping in the visual space. It does emulate reality in that aspect.

          The same way Amnesia or Penumbra use realism and physical interaction to open drawers rather than clicking on drawers and making them open by themselves.

          Both are completely valid ways to handle opening drawers, and both ways achieve the same goal, opening the drawer to check whats inside, but one is more authentic than the other because it emulates the physical metaphor closer.

          Other metaphors might involve just clicking the desk to check its contents or just running close to it to “loot” the desk.

          But you would agree actually physically looking inside the drawer is more authentic, no?

  2. Pingback: Virtual reality photogaphy : Photo, Réalisme et Jeux vidéo - le Blog by Sandro

  3. Pingback: Photorealism In Video Games – A Worthy Goal | liveware.problem | nathantowler

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: