I heard that the main issue with decompressed assets was audio, not video (granted, video is image and audio so one is a strict superset of the other). One game - Titanfall or a CoD game, IIRC - had something like 20 GB of uncompressed audio in a 35GB installation footprint. Rationale was saving CPU cycles.
Meanwhile here I am with a sound card in my desktop PC for no real reason anymore :\
That doesn't really compute. Audio decompression is pretty light on CPUs these days.
It takes about half a second in a single core to decompress a 4 min MP3 to WAV in my 2012 MB Air, including writing to disk. In a gaming machine it will be way less. If anything, the audio could be decompressed and cached on demand, when loading levels, or even during installation.
Also sound cards do not participate on the decompression process. It's been CPU from the start, baring rare exceptions. Sound cards just receive raw PCM. There used to be hardware codecs but they're not really common, especially in consumer/gamer-grade soundcards.
Couple of things to note: MP3 is not appropriate for use in real time due to a variable amount of silence added to the start of the sample intrinsic to the compression. You can sometimes use it for music But if there’s any changes based on game events mp3 is unusable. A lot of work has been put into optimizing mp3 including at os and hardware levels but that’s not usable in games. Commonly it’s some custom spin on vorbis which is
Additionally, there can easily be 20-40 sounds playing at once, more if you haven’t optimized yet (which typically happens in the last few months before release). These also need to be preloaded slightly, and once playing stream from disk, so source starvation needs to be handled and the codec needs to not be glitchy if missing some packets.
It’s also happening in a system that needs to be real-time and keep each frame timer on the order of milliseconds, though Moore’s law moving to parallelization has helped a lot. You’d be surprised how under powered the consoles are in this regard (caveat: I haven’t developed on the upcoming gen, which is getting better)
As for loading and caching on demand, that’s limited by memory, given the sheer amount of samples used in games, it’s just not practical. For specific example in a very well known game, there are over 1600 samples for boxes hitting stuff (impact sounds). What I’m building right now is meant to make generative audio easier and reduce the number of samples needed, so more tools to process sound could make this naive caching approach practical
> For specific example in a very well known game, there are over 1600 samples for boxes hitting stuff
That almost sounds as if it could be worthwhile to synthesize on demand from a very small set of (offset/overlaid) base samples through a deep chain of parameterized FX. With 1600 FX parameter preset tuples as the MVP, bonus points for involving game state context.
That’s literally my startup, I won’t get deep into the reasons why good tools for this don’t exist yet, but if you imagine game development is like regular development but with orders of magnitude more chaos you can understand how difficult it is to build stuff for reuse. After 15 years in the industry, my approach is just the same as yours
> You’d be surprised how under powered the consoles are in this regard
As another commenter mentioned, these games shipped with compressed audio for consoles. Also that generation of consoles have pretty good hardware codecs for audio (320 channels in the Xbox).
And MP3 was just an example of what I had here at my disposal. But as an exercise I converted my 4 minute MP3 to Vorbis. Decoding it converting to WAV took the same amount of time as before: about half a second on a very old and underpowered MacBook Air. Most of this time is spent writing 50mb to disk.
Yeah that is curious if consoles shipped with compressed audio but not PC. The prevailing wisdom on PC is codecs are easier to deal with due to dedicated audio thread. Decisions like that are not made lightly so now I’m curious what the reason was
Edit: reasoning is here: https://www.rockpapershotgun.com/2014/03/12/respawn-actually...
Minspec is 2-core PC, probably to support large player base and as noted before there can be 20-40 audio files all streaming from disk and decoding, so sure one file might be fast but no way that’s happening on a 2-core PC. Sure one file might decode fast, but 40 of them, all also streaming from disk while keeping frame rate playable, just impossible
Good points. But there's still the possibility to decompress during installation, which shouldn't be too hard even for 2-core computers, and is probably faster than downloading.
Also, according to the article they're packing all the locales. To me this seems like a bigger issue.
If it's any similar to the hardware decoders on iPhones, they're probably limited to n streams. That might be good for playing some background music or ambient sounds, but it gets tricky really quickly when you have multiple sounds playing at once.
Not really: The Xbox has 320 independent decompression channels according to its marketing material, which is kinda powerful. The PS3 had similar tech in its Motion Decoder, but they didn't disclose how powerful it was.
And even if it had just a single decoder, there's always the possibility to pre-decode audio. Or just decode the music/film.
All the console versions shipped with compressed audio due to the anemic hard drive and disk speeds, as well as tiny RAM amount.
In general it doesn't make sense to use wavs. Games have been using compressed audio since the late 90s without any significant trouble, and mp3s are aurally transparant at 320kbps.
Rumour I heard was that it was because they realised 20% of their customers using much older PC's wouldn't be able to decode fast enough, so they distributed it with the uncompressed to everyone because no distribution platform allowed "optional" downloads.
Meanwhile here I am with a sound card in my desktop PC for no real reason anymore :\