We know that sound in games greatly increases the immersion of game play especially in first person shooters and third person exploration type games. However what about top down or side scroller type games which are predominantly 2D? Like film and TV, playing video games happen on a screen and is viewed from a flat surface. Film creators know that to really engage the audience, having surround sound is vital and part of the story telling. Hence why the future of surround sound in interactive media like games shall become a standard that will extend to indie developed games.
For us to understand where surround sound is going with gaming we will need to take a step back and look at how surround sound came to be for traditional film media.
Brief History of surround sound
During the 1938 Walt Disney envisioned using a form of surround sound for his upcoming production called "Fantasia"
The Disney engineers created "Fantasound" which used 3 channels of audio three and one for tracking on the film itself that would be outputted from 3 speakers in the front of the theatre and two in the back. (Recommendations For Surround Sound Production, 2004)
Engineers operating the 9 channels in a theatre hall
At the same time the Disney company founded Fantasound and was a company that tried to bring surround sound to the masses by installing their surround system in theaters around America. At the same time of introducing surround sound they also invented:
Multi track recording
Panning
Overdubbing
All this was to create the required discrete channels that will later be played back in their proprietary sound systems.
By 1980's Dolby Laboratories developed and standardised a form of surround sound 5.1 where there was three speakers in the front and two in the back with a subwoofer. The researchers at Dolby determined that 6 discrete channels shall be used to create surround sound and is a precursor to today's 5.1 standard.
In 2010 the first 7.1 surround sound film "Toy Story 3" was played in cinema and have since became the standard for all surround sound in movie theaters. This popularized the 7.1 standard for almost all new movie titles. (Isaza, 2010)
Dolby Atmos was a natural extension of 7.1 which in the consumer market added at least 2 extra speakers overhead that allows spatial positioning of sounds in the room. The nomenclature reflected this change and is called Dolby Surround 7.1.2 and supports up to 128 discrete channels
Supports up to 128 simultaneous independent audio objects in a mix for rich, realistic, and breathtaking sound.
("Dolby Atmos for Home", n.d.)
Obviously for the average consumer this is not feasible to get 128 speaker just so that sounds can be spatially positions in 3D space. That is why there are speaker kits that simulate the Atmos experience.
So what does this mean for games?
Sound design and implementation of Dolby 7.1 games became a natural extension due to the effect of AAA titles becoming available on consoles. Reason being that if a customer has already invested in setting up a 7.1 home theatre system they would most liked want to hook up their console or gaming PC to the same sound system and try to use that system as much as possible to get the most out of the investment. That is why we see a trend in larger gaming companies developing audio that is mastered for 7.1 systems and will inherently follow closely along with the film adoption of sound technologies. And if they want to upgrade to Atmos they can do so by getting a new base unit that supports more channels and buying two extra speakers to hang from the roof. At time of writing there are only a few games that support Atmos ("Games in Dolby", n.d.) and it will be an interesting space to watch in the future.
So what games now use Dolby 7.1?
This is by no means an exhaustive list seen below but we see a trend that large game titles are being mixed for 7.1 surround especially with major titles that are driving games, flying, fighting, 3rd person shooters and 3rd person RPGs like "The Witcher 3".
Click on image to go to site list ("PC Gaming Surround Sound Round-up", 2011)
For the games to be experienced in this mode the consumer will need to have the relevant audio setup to enjoy the added surround experience.
But this opens the question, what about the average home consumer who will at best have a 2.1 setup or a good pair of headphones?
Enter Binaural Surround
For the average consumer buying a full immersive Dolby Atmos system, 7.1 or at the minimum 5.1 is out of the questions due to cost, size of room. That is why Dolby, DTS and a few other companies are some of the forerunners in developing consumer grade surround audio by creating HRTF* (Head Related Transfer Functions) to convert their current and future 5.1 and 7.1 mixes to Binaural mixes. Every audio company is trying to crack this code currently in trying to make "the" correct HRTF algorithm to work for as many people as possible.
* HRTF (Head Related Transfer Function) Which is basically function that is derived by calculating the mathematical model of the human head and the shape of an average human ear.
As with all good things there is still a catch, HRTF's are currently modeled on an "average" sized head space and ear shape. That's why binaural still sounds weird for some people due to the amplification and attenuation of certain frequencies this is explained better by Johansson:
Another challenge is that the sound processed by HRTFs often sounds unnatural. Certain pieces of the audio spectrum are inevitably amplified too much or too little. These inconsistencies can be easily perceived by a trained listener. A casual listener might not identify these colorations directly, but would hear something amiss, like the difference between a cheap audio speaker and an expensive one.
(Johansson, 2019)
So in summary the best way to experience the surround sound in binaural is to have your ears scanned by a laser scanner to create a custom HRTF algorithm for the 7.1 surround to decode into binaural information or to use a best fit solution which is what Dolby Atmos and DTS are trying to achieve.
How does Indie games benefit from this?
The heavy investment made by surround sound companies like Dolby and DTS have created a demand and almost an expectation from consumers to expect high quality surround sound in games as much as film. With AAA titles showcasing the surround sound experience, consumers are looking for that same experience in smaller titles. The market has gained a taste of surround sound and are now looking to get that same experience but to have it made more portable so that consumers do not have to lug around 7.1 speakers everywhere. The binaural solution is the future in terms of finding a medium to give the surround experience without the heavy investment into gear. Small titles can differentiate themselves from a whole slew of other indie titles by having good surround sound.
One of the highlights of your game if you will. Imagine having that at the “back-of-the-box” description of your game for your happy players to discover while browsing for games to play. Not to mention that it creates even more curiosity, your players thinking “What? A 2D game with surround sound? This must be special!”, and click the “buy” button.
(Kouvelis, 2019)
New Emerging Technologies
The Realiser A16 by Smyth Research is apparently the closest we are getting to reproducing the correct surround sound for your body size by placing earbud mics on the consumer and capturing the frequency response from each directions of standard 5.1 and 7.1 and calibrate your own PRIR data. This information is then used to recreate the individualized HRTF algorithm for your ear size.
The result is that you can get a binaural replication of a 5.1, 7.1 and even up to a full Dolby Atmos system with 24 speakers.
In Summary
The future looks bright with the development of surround sound for gaming. Currently many companies are still fighting to setup their own standard HRTF algorithms to overtake the market share. Dolby Atmos seems to be leading with their market penetration in streaming media and films and thus could have a knock-on effect onto games. There are other companies vying to get their own HRTF algorithms setup as standard and thus will be interesting to see how these technologies will develop in the future.
References:
Recommendations For Surround Sound Production. (2004). [Ebook] (p. 11). Retrieved from https://www2.grammy.com/PDFs/Recording_Academy/Producers_And_Engineers/SurroundRecommendations.pdf
Isaza, M. (2010). Dolby Surround 7.1, Toy Story 3 and The Future of Sound In 3D Films. Retrieved 6 May 2020, from http://designingsound.org/2010/05/24/dolby-surround-7-1-toy-story-3-and-the-future-of-sound-in-3d-films/
PC Gaming Surround Sound Round-up. (2011). Retrieved 7 May 2020, from https://satsun.org/audio/
Dolby Atmos for Home. Retrieved 7 May 2020, from https://www.dolby.com/us/en/technologies/home/dolby-atmos.html
Dolby Atmos for Home. Retrieved 7 May 2020, from https://www.dolby.com/us/en/technologies/home/dolby-atmos.html
Johansson, M. (2019). VR for Your Ears: Dynamic 3D Audio Is Coming Soon. Retrieved 7 May 2020, from https://spectrum.ieee.org/consumer-electronics/audiovideo/vr-for-your-ears-dynamic-3d-audio-is-coming-soon
Kouvelis, P. (2019). What 2D game developers fear about surround sound and why you should do it anyway - SoundFellas. Retrieved 7 May 2020, from https://soundfellas.com/blog/games/what-2d-game-developers-fear-about-surround-sound-and-why-you-should-do-it-anyway/