Robert Bantin - OG Game Audio Programmer
Robert Bantin is a Game Audio Programming Veteran that worked on Guitar Hero Live and now with Codemasters. His is a unique perspective on the past, present and future of the industry. Roberts interests include a proper fried breakfast.
Discussion
101: Hello
ROBERT: Hi
101: Why is it that audio programmers are so hard to find these days?
ROBERT: In the games industry that's certainly true. I don't really know why that is but, yeah, it's a curious thing. I've also struggled in the past to sell my skills to studios who didn't know they needed an audio programmer. The two are probably related. There are some really good universities in Europe for audio programming. So there's one in Barcelona and there's a couple in the UK and a couple in Germany. Actually a lot of great audio programmers come from Greece and end up in the UK or Germany. It's a small world too. Invariably what happens to me is I meet someone for the first time and I'll talk to them for ten minutes and I realize that they know someone I know.
101: I see you worked on Guitar Hero
ROBERT: Yeah I worked on the last one "Guitar Hero Live". It was one of those three month contracts that mushroomed into twelve.
101: So do you freelance or are you with a certain company?
ROBERT: I was contracting at various places for about three years, but recently I took on a full time gig at Codemasters. My wife is still running our business on the side though. She is an audio engineer and edits dialogue and sound effects for films and video games. She's ridiculously qualified and speaks like five languages too. In conversation she'll reference some ancient Latin or a Greek philosopher and I won't recognise it. I didn't study philosophy at all!
101: Thats a rare combo! How did you get into game audio programming and middleware?
ROBERT: The first notion of game audio middleware was really nothing more than "Here's a sample - Let me know when you want me to play it." Early middleware would let you create a bank of sample slots and set a number of voices that could be played at once. It was nothing more than a sound effects board. The idea of actually knowing where sound images would be placed in a 3D world is something that was very new at the time and we still talk about it. And while we've come along way since then, it's funny how we come full circle with some of that simplicity. These days audio middleware is this collection of abstraction layers between the gameplay and the audio hardware that, if necessary, can be used just as sound effects board.
101: Can you explain what an abstraction layer is?
ROBERT: If we pick a time like the early-90s, a PC gamer would buy a sound card like the Soundblaster-16 or Gravis Ultrasound, and in either case there was a special software component (we would say a "driver") that interfaced that hardware. Game developers would support certain audio drivers explicitly, so the gamer would set a supported audio device in an in-game menu and supply some data values that helped the game know how to connect to that device on that ISA bus. The OS didn't get in the way. The first abstraction layer (hardware abstraction) came when the OS vendors decided that it would be better if game developers only needed to support a common software component (we would say a "driver model") that they defined and supplied in the OS. Audio device manufacturers then started to produce these thinner hardware-specific components that just hooked into that common component. A little later on, console manufacturers started to produce consoles that had a real OS and a similar abstraction. Faced with all these different platform combinations, there was demand for a software abstraction layer on top all of that to avoid all the extra code maintenance between differing platforms. Without audio middleware, it creates more work for the programmers. Supporting more than one platform becomes a maintenance issue for any studio, really, I mean particularly the big ones because they're normally cross-platform and end up with all these different people bogged down with specific maintenance of one platform or other. Using audio middleware can at least take that burden off them. It's a time saving thing.
101: Yeah, so it helps developers cross platforms more smoothly.
ROBERT: Exactly. Largely speaking I don't think about OS's until QA log an audio bug that's platform specific! Another useful part of audio middleware is the authoring tools they'll provide - tools that you don't have to maintain. At Codemasters we still have a number of in-house tools, but it's a lot less than it could be if we didn't use Wwise. And then of course there's the level of support you get with middleware, which in the case of Audiokinetic is superb. Problems will always crop up, like the other day when we discovered that a particular set of USB headphones where causing trouble. We logged the bug with Audiokinetic and they hooked us up with a developer at their end that took us through a bunch of scenarios over a week and an half until we got the resolution we needed. We have a source code license for Wwise, so we were digging through their laundry as it were, and really, they have nothing to hide. It turned out we just needed to upgrade to a later version of Wwise, but then that revealed a separate issue on our side that they then helped us fix immediately. And now I know a little more about how their middleware works, which is nice.
101: It sounds like great support from Wwise.
ROBERT: It's an incredible level of support. The level of support you'd expect for spending big money in-house. I mean I hear that Rockstar still have their own audio system (post-Renderware I suppose, which would have handled the audio in the past), but I think more and more often now people are used to licensing audio middleware because they can, in any case, customise it to work they way they want. For example, the ones I've used all have DSP plug-in support, so it's not like you're stuck with that company's vanilla serving suggestion. At Codemasters we have a collection of custom DSP plug-ins at our disposal, one of which I just added in recently. It's a simple enough thing to do if you're familiar with writing DSP plug-ins in general.
101: So you built a plug-in for Wwise, just for your own project?
ROBERT: Yeah I've done that. I once said to a client that if I had the chance I'd model all the analogue components of a Taito arcade cabinet just so you could do all the sound effects of Space Invaders procedurally. Then they got back to me and asked if I could actually do it, so in the space of two weeks I had something working with a version of Space Invaders that they used for their programming test. I'm not quite sure if they really remembered the sounds of Space Invaders though, because they didn't like that fact that it sounded really synthetic! But that's exactly how Space Invaders sounded! Haha oh well. Anyway, my point is, you're not stuck when you use decent audio middleware. If you're not happy with (say) a filter you can just make your own. That allows you to make the changes you need to make without the burden of the rest of it, and that means you can focus on making the game.
101: This is also something that people benefit from when using Unity.
ROBERT: Right, except that now the whole game engine is somebody else's, but you can still customise it at various levels. In the same way though, the maintenance of those custom extensions is quite small compared to maintaining an entire engine. There's a ton of stuff out there - it just accumulates as people work on their projects. For example, I've got a VU meter I once made in Unity, except that it's a fully textured 3D model of a VU meter with a needle that twitches according decibel updates it gets from its DSP processor. And because of the way I built it I can re-use whenever I'm using Unity.
101: So how did you learn how to program?
ROBERT: So what happened to me was that when I was eight years old the UK government had this mandate to get a computer into every home, with the BBC producing a primetime..yes, primetime... show on how to program in BASIC. I believe the original run lasted 12 weeks, although it spawned a more expositional follow-up show that ran for a few years called "Making the most of your Micro". There are some on YouTube you can dig up. The government found this start-up called Acorn out in Cambridge, and they built them the BBC's educational computer - the Acorn BBC Micro. The messaging to parents was "your children will need to know how to do this some day". So many of us in the UK got bought a BBC Micro by our parents so we could follow this show and "learn the skill of the future". It seemed a rather far-fetched in 1984 as there were no computing jobs outside of university, the government, or telecoms. None of those sounded very interesting to me. At that time I wanted to be a rock star or an astronaut. What I think was crucial at that time though was that no one saying "learn this and get a good job at the end of it".
Learning to program was a very nebulous time investment, personal improvement, all that. I think it's a shame that most people don't think that way anymore. And what people might not realise about computers back then, is that they didn't come with much software. They came with a book on how to program. "Congratulations on your new purchase, now here's how you make your new computer do something". And at that time the public were willing and able to accept that they had to put some work in. Everyone had a go. Taxi drivers had a go. Brick layers had a go. My dad had a go. That's how I got my start, because there wasn't any stigma to trying to figure out how a computer worked. I miss that. Companies started producing basic office software, but I what remember mainly was a lot of games coming out for these machines. At the start they were mainly arcade clones, but pretty soon there were original IPs too. My focus at that time was comic books and animation, so what drove me was making sprites move on-screen with dialog appearing below them. Laughable by today's standards, but there was definitely something "there". Then I moved up to secondary (i.e. high) school and was surrounded by boys that knew so much more than I did about programming. I'd loiter in the computer room with these other kids of varying ages, just in awe of how capable they were. There was this seventeen year old called Ian Crowther who would sit there and code away in every spare moment he had. He had finished games sat there on the network that other kids played. Epic stuff. I bumped into him last Christmas at the Kuju/Headstrong/ZoeMode christmas party. He said to me "you've grown!".
101: So you used to hang out with older kids in high-school?
ROBERT: Actually in my high-school you wouldn't interact with anyone outside your age, but there were two interests that I had that seemed to give me a free-pass on that hierarchy: Computers and Role-playing games. You know, Ian was seventeen when I first met him and had no reason to speak to me, but because I was interested in what he was interested in it gave me a leg up. So it was very good for me to be able to make friends who were older than me and to get to go to people's houses where they had cars etc. Do you remember the Amiga demo scene?
101: I sure do.
ROBERT: So I was involved in that with people who were older than me - me and my friend Jon were like fifteen and they were like twenty years old, and I remember getting on trains and going to these house parties in London and drinking nothing but Coca-Cola and eating crisps or whatever. We'd show up with our boxes of 3.5" diskettes and get busy making a demo. At the time I was the art and SoundTracker guy - "art" meaning that I would sit there working out the arrays of numbers that we needed and would punch them into the source. There was a lot of graph paper going on! Ha I remember working out vertices for a 3D logo once and other guy kept saying "are you sure that's right?" because the logo was glitching horribly as it turned. I now realise why that was - his triangulator can't have been handling concave polygons correctly. *Sigh.
101: So you were using trackers back then?
ROBERT: Yeah, basically there was this open-source assembler routine lurking around the bulletin boards that could parse the .MOD file and play/stop the pattern sequence. A lot of people put music into their demos and games that way during the Amiga/AtariST era. There were a whole bunch of tracker authoring tools out there that were for free (I hope), like ProTracker and StarTrekker ("To boldly go where no soundtrack has gone before"). I feel sorry of the guys that made OctaMED as it was intended as a commercial product but it was too easy to pirate. I just stuck to using ProTracker and StarTrekker anyway, but I remember years later hearing about the trouble they went through. There must have been a lot of that going around. Anyway, you'd have the brother or sister of some coder show up because they were the musician in the family and wanted to help out, and I would be asked to help them get to grips with a tracker tool, and it never went well haha. You're essentially asking an artistic person to pour their heart and soul into an event list.
Them "How much is eight pattern lines worth?"
Me "That depends on the BPM of the pattern. Try it."
Them "No that's too fast"
Me "OK try sixteen".
Them "How do I fade that in?"
Me "You'll need to put in a list of volume changes. Do you know hex?"
Them "No."
Me "It's a number system like decimal, but you spin on 16 instead of 10. And you use letters for the extra characters."
Them "Huh...?" 101: I remember working with trackers on my own and at first, it felt like hieroglyphics. After the steep learning curve, it became kind of fun. Of course, that seems like ancient history nowadays.
ROBERT: There was a lot of source code shuffling between companies back then too. The company that published Notator didn't own the source, it was owned by the programmers! This would never happen now! So then one day the programming team had enough and started their own company - eMagic - which released Logic soon after. eMagic grew in size, and then Apple bought them. Some people had to stay, but plenty of others went to Steinberg to work on Cubase and Nuendo. I get the impression that Hamburg is bunch of revolving doors for audio programmers, as there's Native Instruments, Ableton as well. Audio programmers seems to migrate from one to the other as the mood the takes them. The source isn't shuffling around anymore, but the knowledge certainly is.
101: When did the audio programming start for you?
ROBERT: At 11 probably, when I took the music general studies module. One of the classes was in the computer room, and was about making music with the computers. I thought, "What's this going to be about?". They taught us how to make music by writing code to build melodies, and from that command the built-in speaker to produce the right tones. At the time I was learning classical guitar, and all the parts I had to learn required the teacher to back me up with a secondary part, which was annoying because I couldn't practice to full effect unless the teacher was there. So I went home that night and I transcribed all his back-up parts so I has something to play against. You see, there was no persona to programming back then, there wasn't any social media profile I felt compelled to feed with "hey look what I'm doing today". I just needed something that didn't exist, so I built it. It was digital carpentry I suppose.
101: When you say you were coding, was that C "plus plus" OR C "minus minus"
ROBERT: In 1987, it was BBC Basic and 6502 assembler! That's all we had. I remember a friend invested in a commercial C compiler for his Commodore Amiga in 1990, and I specifically remember being aghast by how inefficient it was when compared to 68000 assembler. There were other languages out there we could have used, I remember AMOS Basic got some good use near the end of the Amiga era. It has a ruthless compiler and had lots of hardware accessors that mean you could work on the metal without having to write any assembler. I wrote a simple flight simulator for a school assignment that way. If you were ever into drawing fractals - there was a very detailed one that was written in AMOS Basic. For most part though, we were using unlicensed copies of ArgASM to build our demos (sorry Argonaut!)
101: There are many different programming disciplines in games. Given all these different flavors, what would you recommend to someone starting out now? Just learn C++?
ROBERT: C++ - despite what some people say on social media - is still a widely used language, especially in high performance software. That's because it's scalable and still runs fast, and we like that in games. I've seen some murmurs about Haskell being a viable alternative, but I've not been sold on it yet. My main advice though is to have a reason to learn how to program. No one ever learned to speak French by thinking "hey this'll be fun". What happens is, you're in Paris and realise you need a coffee, and the only way to get one is to figure out how to order it. With programming it's the same way.
101: I really appreciate your time Robert it's been great talking to you.
ROBERT: And you. Hope I didn't talk your ears off!
