Acoustic Modelling in Games

The effects of a virtual environment on in-game audio are commonly mimicked by creating pre-set falloff distances for each sound source as well as occlusion zones and DSP based reverbs effects. There are however, ways in which the effect of the environment on a sound can be calculated by the computer rather than artificially set by a sound designer. Two of the methods of modelling an acoustic environment are Wave-based modelling and Raytraced modelling.

Wave based modelling calculates the propagation of a virtual sound as if it was an actual sound wave in a real environment using acoustic wave equations. This method is highly computationally expensive but it can accurately simulate all acoustic effects. (Miga B. and Ziolko B., 2014)

Ray traced modelling treats the sound in the same way as light rays are treated when lighting and reflectivity is computed for graphical purposes. The sound source emits virtual rays in random directions within the sources directional range and the paths and reflections of the rays are calculated. Should a ray come into contact with the player, the computer calculates the direction from which it approached as well as any filtering based on the distance the ray traveled and the surfaces from which it bounced during its path. One technique that may be used to make ray tracing more computationally inexpensive is to create virtual sound sources to simulate the reflections of the original sound during processing. This is known as Image Modelling.

"The image model ... has the advantage over ray tracing that the calculation deriving all the virtual voices for a sound source only have to be calculated once, and all sound reflections will continue to sound correctly, even if the listener rotates or moves around, as long as the sound source itself doesn’t move."

(Bengtsson J., 2009)

The inability to use this method for moving sound sources does limit its application in games. Ray tracing has been shown to be a viable method of real time acoustic modelling by others. Miga and Ziolko (2014) demonstrate that by affecting the sound within discrete frequency bands rather than across the full spectrum they can compute, with some compromise, reflection, transmission (sound passing through a material), air absorption, and attenuation over distance. The filtering of the sound was made efficient through the use of filters made using finite impulse response. In this way they managed to process two hundred audio streams in real time and used, “only part of the CPU power.”

(Miga B. and Ziolko B. 2014)

Scissler and Manocha (2016) also demonstrate ways in which ray tracing can be made computationally viable for real time use. A system was created so that sounds could be affected in 'clusters' based on their distance from the listener. They also used backwards ray tracing so that the tracing starts at the listener rather than the sound source. This way the beginning of the sounds path is the same for many of the rays. Hybrid convolution rendering was also used in that the system calculates doppler shifting using either interpolation or partitioned convolution rather than a doppler shift algorithm. Scissler and Manocha (2016) claim that acoustic environments modelled using wave based modelling methods are limited to static scenes and are not yet practical for scenes with many sources.

Mehra, Rungta, Golas, Lin and Manocha (2015) demonstrate the use of a wave based modelling system that can respond to dynamic objects and a dynamic listener in real-time in a VR (virtual reality) environment. The ability of players to locate a sound source was compared between this system and a ray traced audio modelling system. The results showed that players' performance improved by 27% in the wave based acoustically modelled environment. This suggests a significant application this kind of system in VR game development.

All of the above models treat the sound as if it was propagating in a linear medium. A system that can take into account temperature differences in the air, as well as wind and other factors that change the linear nature of sound propagation through air, has been tested by Mo, Yeh, Manocha (2016). This system allowed curved ray tracing rather than purely linear. While this may allow for increased realism in outdoor environments, the system is not yet viable for real time application in games which may run at 30 to 60 frames per second.

While it seems that artificially modelling acoustics using dsp reverbs and occlusion zones may soon be unnecessary, in practice a realistic environment designed to model the real world may not be the best way to design the sound in a game and may detract from gameplay compared to some currently used techniques. Miga and Ziolko (2014) used to subjective opinion of professional sound designers to judge the experience that their system and found that despite using ray tracing rather than sound wave based modelling, they had created a, “sufficiently good experience”. The use of more complex techniques such as wave based modelling may only be necessary in VR environments and only for sounds that are required to respond realistically.

References

Miga, B. and Ziolko, B. (2014). Real-Time Acoustic Phenomena Modelling for Computer Games Audio Engine. Archives of Acoustics, 40, pp. 205-211.

Bengtsson, J. (2009). Real-time acoustics modeling in games. [online]. Available at: http://nevyn.nu/realtime_acoustics/Version%207%20render.pdf [Accessed 24 Oct. 2016].

Scissler, C. and Manocha, D. (2016). Interactice Sound Propagation and Rendering for Large Multi-Source Scenes. ACM Transactions on Graphics, Vol 36, No.1, Article 2.

Mehra, R., Rungta, A., Golas, A., Lin, M., Manocha, D. (2015). WAVE: Interactive Wave-based Sound Propagation for Virtual Environments. IEEE Transactions on Visualization and Computer Graphics, Vol. 21, pp. 434- 442.

Mo, Q., Yeh, H., Manocha, D. (2016). Tracing Analytic Ray Curves for Light and Sound Propagation in Non-Linear Media. IEEE Transactions on Visualization and Computer Graphics, Vol. 22, pp. 2496-2506.