Jam3 partnered up with Oculus, Electronic Arts and Respawn to create a web experience for Respawn’s Medal of Honor: Above and Beyond. The immersive site for Respawn Entertainment’s Medal of Honor brings the 3D feeling of Oculus VR gaming to players on 2D surfaces. Step into the shoes of an OSS agent during WWII in this new installment of the beloved AAA franchise the way it was always meant to be played: in 360º reality.
The WebGL experience turned the game’s painterly key art into an immersive 3D battlefield. By scrolling through the action, you can experience selected features and gameplay moments. By the end, you have a crystal clear idea of what to expect from the game: a stunning first-person experience, gripping moments of heroism, and innovative combat mechanics.
We conceived and developed a unique painterly look that channeled the drama of a WWII recruitment poster. But our immersive 3D poster dials up that drama considerably by pulling you right into the fray as you explore a moment frozen in time. After you’re dropped into combat, you can scroll deeper into the scene to uncover videos highlighting story fragments and game mechanics. Rather than bombard you with exposition, we let content from the game handle the heroics. We just set the scene with gorgeous visuals, rich atmosphere, and tactile UI elements inspired by period artifacts, like mission dossiers and telegraph messages.
Medal of Honor: Above and Beyond is the first game in the series to come to life in Virtual Reality. We were fortunate to have access to an early version of the game, which helped us find our creative direction. During our play through we captured locations and narrative moments we could potentially translate into story beats for the web experience.
One of the locations that stood out to us was the French Village of Dubuisson. Here the player assumes the role of a WWII resistance fighter.
With the location set, we worked to develop a visual style that would reflect the era and themes of the game, while remaining visually distinct from in-game graphics.
Our mood boards were inspired by the impressionistic style of the key art featured in the game. This painted texture mirrored iconic WWII propaganda posters that perfectly captured the tone of the game.
Our creatives painted a style frame to help our team to align on key elements the site would feature. It also helped us demonstrate to Oculus and Respawn what our vision for the website could be.
Impressionistic style rendering has been used in several games, and is well-documented in articles like this one by Disney Animation. We began to explore how we could use a painterly style effect in real-time using WebGL.
We liked the direction this treatment was heading in, but we weren’t sure how this effect could scale and how much detail we could push across a large-scale world.
Medal of Honor: Above and Beyond was still in development at the time of this project. Therefore, we weren’t able to use any of the game assets for the website. It became clear that creating the assets ourselves would be more time efficient and would give us greater control for the art direction we wanted to achieve.
Before any asset creation could take place, we drafted some plans for the town layout and camera path. This would inform what we needed to model and where we could create action moments.
Our creatives mocked up a camera path animatic in Blender to help visualize our story in 3D space. The main benefits of this approach were that it helped Jam3 get client approval, it informed our creative team on what to model, and it helped to visualize the pacing between narrative moments.
Exporting to WebGL
Our next challenge was to export our scene, compatible and optimized for mobile devices. There were a variety of ways we could do this including:
Export the entire scene including textures as a glb file
- Export the scene as an FBX file and load the textures separately
All of our textures were painted at very high resolutions, so the first approach wouldn’t be feasible. We also wanted to deliver two variations of each texture, based on the device’s capabilities. So we chose the latter approach.
The glTF format contains mesh nodes that map to materials and textures. Using the texture indices, we can look up which meshes use which texture.
We wrote a nodejs script to generate two data files. The first was mesh-to-material mappings, which would be used at runtime to apply the corresponding textures to the meshes.
The second script generates a configuration file so we can customize the resolutions we want per platform.
It became clear early on in development that PNG and JPG files weren’t going to be performant with the amount of textures that would be required for the entire scene.
Creating a 3D world at this scale, with this many textures, would require a different optimisation approach than WebGL experiences we’d developed in the past.
Basis universal texture format is a compressed texture format for GPUs. It’s a highly optimised and universal format that works across a variety of GPU’s which enables us to use higher resolution assets with a lower memory footprint. Threejs has built in support for it too, with the BasisTextureLoader.
Using basis universal compression for textures and the Draco compression for glTF gave us a huge reduction in file size and GPU memory. This allowed for both greater fidelity and faster loading times.
Colours were used to visually debug texture resolutions in the scene.
With the scene being completely static, we also disabled matrix calculations to speed up rendering performance.
Discovering game content
Collecting dossiers was a game objective used through several of the Medal of Honor games. We liked the idea of collecting intel on the enemy, so we used this mechanism to reveal game content to the user.
Hotspots would appear at key moments throughout the scene. Opening them would showcase footage from the game. We used threejs CSS3D renderer to anchor the UI hotspots to the world.
The site consists of multiple 3D scenes: preloader, home, intro, experience, and end. To create a seamless transition between these, we used a mask sprite sheet animation in our post-processing stack to blend one scene into another. We also Pre-render scenes for smooth transitions.
On the experience scene, the navigation timeline acts as a quick menu to jump from hotspot to hotspot. The transition mask effect didn’t account for this initially, as it required the same scene to be rendered twice. We modified our post-processing stack to account for this new behaviour.
Sound always plays a crucial role in immersion. We used threejs web audio capabilities to add positional sound to the scene. Sounds play around you, anchored to objects and also triggers as you scroll past. We created utilities to move and preview them in context. Each sound had its own config, allowing for greater customisation.
With the scene at 500k triangles, we needed to be conservative with post-processing effects to ensure that we wouldn’t need to re-render the scene multiple times. We created a minimal post-processing stack that turns on effects when required by the scene.
- Transition Pass (active during scene to scene transitions)
- Motion Blur (enabled for high graphics mode, active during scrolling)
- Depth of field (active when the user sees a hotspot)
FXAA, Vignette, Noise (active during the experience scene)
We experimented with a painterly pass too. This used the world depth position and triplanar mapping to distort textures in world space. The downsides to this approach were loss of performance and texture detail.
Threejs does a good job at sorting depth transparency in most cases. However, we noticed a few issues with transparency clipping on models. We manually set the render order for these transparent models to fix any visual glitches.
We always strive to deliver the best graphical experience for the end user but we also have to be realistic for the range of devices we try to support. Two graphics modes are available: high and normal. We use GPU Detect to profile and assign a tier to the gpu. This allows us to optimize the experience on a wider range of devices.
Other areas we optimise include:
- Particle count reduced
- Triangle count reduced on certain meshes
- No motion blur
- Lower resolution textures
- Mobile optimised videos
To pay homage to the series’ history, we decided to uphold and evolve the dossier visual language that originated in the first game. We wanted every element to feel like it was designed in the 1940s. This meant integrating textures that looked like real materials you could touch and feel.
Going overboard with paper texture can border on cheesy, so our aim was to find a balance that kept the UI feeling light, but still reminiscent of a physical briefing document. Bold brand fonts were grounded with light textured rules. We wanted the hotspots to feel like they were pulled out of a soldier’s notebook, so we designed them as quickly rendered pencil marks.
Given the design focus on the painterly style for the 3D scenes, the UI layer needed to match the more tactile look and feel of the experience.
We ruled out using pure SVG elements or coloured DOM elements to populate the design touches, instead choosing to use background images to create the textured look we were aiming for. We created an SVG button component with the paper-like texture and added the icon options as a mask to keep the button scalable and transparent.
Typewriters were considered an essential tool within the military during WWII. We chose DIN Next Slab as the global font choice for all the user interface copy, also within the game development diaries. Combining this font with paper-like textured mirrored WWII documents more authentically.
The motion chosen for the design elements also spoke to the skeuomorphic nature of the design choices. Such as stamps pressing into the page, lines being drawn, and character profile images flashing in as if it was a photograph developing. We achieved this with the use of GSAP library and the custom transition group library.
We also used a sprite sheet animation on some elements to give the motion a more unique feel. The animations were created in After Effects, translated to sprite sheets and loaded in individually instanced 2D canvas elements and attached animation triggers to DOM events.
Peter Hirschmann, the creative director of Medal of Honor: Above and Beyond and the original 1999 classic, said their goal was to make the experience feel “grounded and emotionally authentic”.
While technology can give us new and innovative ways to make and deliver experiences, the real experience is always the story. In this case, our work was grounded in the stories of those who fought and experienced the war first-hand. The story is the real hero and the technology is how we bring that story to life. The two should never be considered separately.
At Jam3 we take great pride in making sure the creative and technical choices we make work invisibly to enhance our storytelling and provide the most authentic web experience possible.
Medal of Honor: Above and Beyond is now available on Oculus Rift, Oculus Quest with Link, and SteamVR.
Jam3 is a Design and Experience Agency. We help the world’s top brands launch products and design new customer experiences in digital and beyond. Rather than make ads, we create relevance with consumers by attributing purpose and innovation in everything we do.
Our partners include Google, Spotify, Microsoft, Adidas, Facebook, Ford, Sephora, MTV and Disney Pixar. Modesty aside, our diverse work has received global recognition from Cannes to The One Show, and even the Pulitzer Prize. To learn more, visit us at jam3.com.