Our Universe Revealed, Part 11: Crafting Music & Synthesizing Sounds with Geometry

Episode Topic: Crafting Music & Synthesizing Sounds with Geometry

Have you ever wanted to create your own unique sounds but didn’t know how to start? Using basic geometry, it’s easy to synthesize new and exciting sounds through software you can run on a laptop! In this talk, Indiana University South Bend professor Ryan Olivier will demonstrate a few synthesis techniques and explain how thinking in geometries has informed his compositional process.

Featured Speakers:

•  Ryan Oliver, Indiana University South Bend

Read this episode's recap over on the University of Notre Dame's open online learning community platform, ThinkND: https://go.nd.edu/b6876b.

This podcast is a part of the ThinkND Series titled Our Universe Revealed.

Thanks for listening! The ThinkND Podcast is brought to you by ThinkND, the University of Notre Dame's online learning community. We connect you with videos, podcasts, articles, courses, and other resources to inspire minds and spark conversations on topics that matter to you — everything from faith and politics, to science, technology, and your career.

• Learn more about ThinkND and register for upcoming live events at think.nd.edu.
• Join our LinkedIn community for updates, episode clips, and more.

Transcript

Welcome and Meeting Ryan Olivier

Speaker 10 0:08

Good evening. How are you? Um, my name is Deb Maher. I'm a professor of ecology at Indiana University South Bend, um, and I'm serving as moderator for the Our Universe Revealed lecture series. Um, this series includes talks that include all areas of science, and this year we have added the arts and music, uh, so STEAM for everyone. Um, and tonight I'm, I'm really pleased to, um, we'll be starting in on the, on the music part of this. So we feature current research and creative work that's being done in our region, and it's an opportunity to be curious about ourselves, our world, and our universe. Uh, this is a partnership between Indiana University South Bend, the University of Notre Dame, and the St. Joseph County Public Library. Tonight is my pleasure to introduce Ryan Olivier. His work spans music, science, and technology. He received a bachelor's degree in music composition from Loyola University in New Orleans, a doctorate of musical arts from Temple University in Philadelphia, and he's an assistant professor in the Department of Music at Indiana University South Bend. His current focus is creating visualized electronic music with live performers. His work has been performed by ensembles throughout the United States, from New Orleans to New York City. Um, he's also performed internationally in Taiwan, U- uh, United Kingdom, and Iceland. And, uh, Olivier's work, Moments from Movements, was commissioned and premiered by the South Bend Youth Symphony in May of twenty twenty-three. So it is a real pleasure to have him tonight and share with us some of the creative things that you can do with music

Speaker 2 2:07

Test, test. All right, I passed the first test. I've got the mic working. Um, thank you so much for having me. Uh, I've got a lot of technology, and I know how to use some of it, most of it. Uh, so I'll try to talk a little bit about what's here on stage, and then also talk a lot about what I do here with my computer. Um, I was trained and started as a composer and a musician, but nowadays I spend a lot of time making music and performing with others using my laptop and using synthesizers and visuals to complement what other performers are doing with me on stage, or to create works that have their own movement even if there are no performers present. So this talk, Our Universe Revealed, a lot of the previous talks that I watched online, which were wonderful, had all these really amazing, big ideas, and I thought, "Oh, no. This is a little bit about just like me. It's my universe." And so apologies. However, um, I do think that there's something that we can take more broadly from the arts. Even if it's one person's creative output, there's something to be said about people gathering together and experiencing art together, and what it says about our perception of the world, how we view it, and how we communicate that perception with one another in experiencing that art. So while we are talking about my universe in this talk, we're going to do a little bit more, I hope, than just my universe as we can hear Strauss. All right Strauss the fade out here So as I mentioned before, um, there's, uh, a lot of composers throughout history who have spent a lot of time working with art and music and thinking about how their music and their art can comment on the way we understand our-- the universe, but probably more importantly and more accurately, how we understand ourselves and one another. As a composer, I spend a lot of time in that world. I think a lot about what sounds I might create, what sounds interest me, and then learning as much as I can about those sounds, how they're put together, and also just general things that I can understand that I then want to share with other people. The way that I view a melody might be something different than someone else has experienced, or, and probably more accurately, it touches on something that you have experienced before and speaks to you in a particular way. In this way, music allows us to share experiences together and inform one another's understanding of our past experiences and hopefully tell us a little bit about future experiences that we can encounter. So by investigating sound, investigating music, I hope to try to tell you something about my experiences and hopefully in turn, go and experience some of the art that you create and learn a little bit more about your experiences.

Music as Motion: Visualizing Sound

Speaker 2 5:13

As I mentioned before, there's something kind of unusual about going to a music concert when you start to think about what that experience really is. It's a bunch of people sitting in a room quietly usually trying to listen to the faintest vibrations that are moving through the air caused by another person moving things. Like, I'm trying to imagine if someone said, "Okay, let's all gather and watch me just push this brick back and forth. I'm gonna push this brick back and forth, and we should all just watch me do it, and I'm gonna change the speed at which I'm pushing this brick back and forth." It'd be kind of a weird thing, but we have no problem with someone saying, "Come to this concert. I'm gonna push this string back and forth as I play with the bow," or, "I'm gonna press some keys and move things back and forth." I keep mentioning this back and forth thing because it's one of those things that when I started to really think about the way music is put together and what exactly is happening with music, I started to come to this realization that I think many others have had, that music is really about motion. When we go to experience a concert, we're going to experience someone else creating motion in order to convey a feeling. Sometimes we think of that feeling as a very abstract concept, right? Oh, the music, the melody was so melancholy or so sad, and now I'm experiencing that same emotion. But the physical nature of the concert truly is feeling that motion that the person created on stage as they disrupted something that was still or in stasis. and pushed some sound waves out into the space for your ears to encounter. I mention all of this because, uh, while I was trained as a composer and thought a lot about melody and harmony, these were the topics that really excited me when I started to think about what is sound actually doing. Because when I started to think in that way, I was able to kind of break down the concepts, the-- my basic understanding of sound, so I could reconstruct it in different ways, and that really intrigued me as a composer. So once I started doing that, I-- of course, I started with sounds that I knew. That's what we do. But then the question was, okay, if I break this down, can I come up with sounds that I haven't heard before and maybe share those experiences with other people? So what I'd like to do now is share a piece. Um, it's actually kind of old at this point. I think it was twenty eleven that I wrote this piece. It was one of the first pieces I created of a genre known as visual music. I wanted to start with this piece early on because it's short. It's one minute long. I'm gonna play the piece, and then I'll talk a little bit about what we experienced, and then we'll talk about how I created the piece Okay. So that piece was, uh, from a suite called Colorful Movements. The title of that particular movement is Metronomic Homage. Uh, it was right at the... I was in graduate school, and it was at a time when I needed to be studying, right about now, which is when I feel most creative, when I need to be doing other things. Uh, the same is true now that I spend more time teaching. So, uh, who knows what I'll be composing next week. But, uh, in this case, what I was really thinking about was there is this composer, um, named Ligeti, and he has... He's a very funny composer. He has some very serious music, but, uh, he also wrote some very silly things. And one of my favorite pieces is a piece where he takes, uh... It's called Symphony for Metronomes, and he sets up a bunch of metronomes, physical metronomes, the, the obelisk type, right, where the thing clicks back and forth. And he had a mechanical arm, and the arm would go through the space and start all of these metronomes. And they would all be set to all these random times and, you know, you'd wind up the thing. And the piece didn't end until the last metronome was just sitting there clicking, and it would run out of time. And it's f- it's a wonderful, fun piece. You should find it on YouTube. Uh, and I thought, "Okay, cool." But what if we start all the metronomes at exactly the right time? A mechanical arm, of course, there's a delay as you go through it. And then I thought, "Okay, not only should we start all the metronomes at the same time, but what if we wait for them to all sync back together?" So what I was really interested in is taking this concept of beats per minute and then dividing it by the metronome settings on those old obelisk metronomes. If you've ever played a piece of music and you see MM equals, when I was a student, I always thought it was metronome marking. But it actually refers to the Maelzel metronome. Uh, Maelzel was kind of an inventor, but he was more of a stealer, if you will. So the, that metronome was invented by someone else, but Maelzel was a great salesman. So the Maelzel metronome's the one that you'll find in Beethoven scores. And on the Maelzel, the lowest tempo is 40, and the highest tempo, I should remember this, I think it's 240 at the top. And they go by different increments. So the lower end, they're going by fours. Then you get to 60 beats per minute, and they start going by threes. And then they start moving by twos. So okay, I've got a limit. I'll choose only the metronome markings on Maelzel's metronome. There's 39 of them. So I wanted something to be visually present that we could see, and while 39 doesn't really fit well into a grid, so the bottom right one is just a bit faster right off the Mazel metronome marking at the bottom right. But what I was gonna do was start them all at the same time, and then let the rhythms, the natural polyrhythms that arise from these different ratios of the metronome kind of make the rhythm of the piece. So what you get is when you have beats per minute at 120 versus 60, you get your eighth notes, right? Mm, pa, mm, pa, mm. When you get your threes against two, something like 90 versus 60, you get a polyrhythm, so, uh, mm-mm-mm, mm-mm-mm, mm-mm-mm. So you get these nice rhythms And I thought, okay, great, I've got a piece, but you know, the metronome thing, it's kinda been done. So if I do the clicking, I feel like it's not gonna be that interesting. What more can I do to show these relationships and allow us to hear when things sync up together? So that led me thinking a little bit more about ratios, and I started to think about the components of sound. So I wanna take a moment to talk a little bit about what we experience when we listen to different kinds of sounds, in particular musical sounds, and different ways we can understand them. So, uh, I don't know if some of you have seen this before. This is the, uh, couple screenshots from the Chrome Music Lab. I remember when this came out, I was really excited, uh, 'cause it was a great teaching tool. It was also a nice performance tool. Uh, some people have used them to perform in different laptops ensembles. And I was shocked when I was at a conference and the presenter was the person who came up with this, uh, website, and he was from the Google Ad team, and I thought, "Oh no, it's corrupt. They're trying to sell something to me." Um, but he did explain that, you know, part of their advertisement is to kind of embark on new things that would in- invite people to explore and discover, and this was one of those things. So this grew out of a, um, Chrome Google Doodle from one day, and they decided to explore a bit more. So there's a bunch of different little, um, tools that you can learn to use-- l- you can use to learn about music. We're gonna look at some of them. The one I wanted to start with here is the spectrogram. So the spectrogram allows us to see different components of the vibrations that we're hearing. It breaks down regular and irregular movements into different constituent partials based on a Fourier analysis. So that all sounds, um, kinda complicated unless you know it, and in that case, I'm sorry if I got it wrong. Uh, and I'll hit Allow here. But what we can see is as I'm talking, we can see my voice being analyzed, and you might notice some particular patterns as I'm talking, right? So my consonants look kind of like a big sp ladder of things, shhs, but when I say a vow, you start to see rows of things appear. And what we're seeing when we see those rows is we're seeing particular sine tone frequencies that stay fairly regular. What's also interesting is that most of the musical pitches that we like to gather around and hear often have these patterns that are in whole number ratios to one another. So typically you'll have a fundamental that's fairly lowww, and then somewhere about an octave above that, higher, you might have something that is going to sound in a ratio of two to one. So our octaves actually typically get pretty close to a two-to-one ratio So, ah, two to one. Hmm, that kind of sounds like 120 beats by over 60 beats. And then, hmm, three to two, well, that's a perfect fifth. That's also a ratio that we find in this harmonic series, and that's kind of like 90 beats to 60 beats. So what I started to do was take this understanding of tones and map those ratios to frequencies for each one of my metronomes. So when we were watching the piece, we saw each of those boxes flashing. Each box was a metronome clicking, and the pitch of that metronome was just a simple sine tone. The sine tone was then mapped onto the rate at which it was flashing, so that when different boxes hit together, we would hear the combinations of those tones. So when you're listening, some of them sound like splat, right? Because we had a bunch of them all syncing up together, and they're not in that ordered harmonic series that makes something fairly pitchy. But others sounded like nice little tones coming out, and at that point, we would hear things like two to ones or fours to twos and threes to twos all lining up. So this is just a quick example of how I took something that's a fairly basic understanding of music and tried to break it down and rebuild it up to create something that sounds a little bit different, like a little bit off, right? So some new sounds from those concepts.

Synthesis Fundamentals

Speaker 2 16:53

So I wanted to talk a little bit today about how you could create those sounds because, uh, the stuff that I use is all free. A lot of it, um, some of it's not, right? So I brought in some hardware, but a lot of these things you can find and use, uh, yourself. Let me get back to this presentation here Phew. That was a close one. Okay, here we go. So, um, I wanna talk a little bit about, uh, four different techniques that are fairly simple for making your own sounds. So the first is wavetable synthesis, and wavetable synthesis uses basic shapes. Um, part of my presentation title was Geometries and Hearing Some Geometries. So we can see, here's Mr. Square. He's ready to perform, and this is also part of that Google Chrome, uh, website. Let me see if I can get over to perform Mr. Square. There he is. Okay. So you can see we have an oscillator at the bottom, so it's going to regularly play this shape, and if you think about a square, we've got a top, and we've got a bottom to the square. The top is our on, and the bottom is our off. So what we're gonna hear is we're gonna hear something turning on and turning off at a regular rate that's even, and here on my oscillator frequency, I can control the rate of that on and off. At a really low rate, we can hear on and off, right? So if you've got a watch, you could probably be checking your seconds here, 'cause we've got one hertz, which is one time per second, right? So we hear that on and off. As we go up a little faster... We can hear 10 times a second, and soon you can start to hear a lawnmower starting, right? All right. If that's what your lawnmower sounds like, you're in trouble. But we've got our square wave making sound, and it really is just this on/off at a particular frequency that's reoccurring that produces this sound. They cheated a little bit. There's a little bit of vibrato to this square wave, but it's close enough So let's pull up a different shape. We'll come back to the saw wave. Uh, how about the triangle wave? So let's take a listen. We've got the same concept here, frequency, and we're gonna follow a triangle wave, but instead of simply going on and off, we'll ramp evenly on and ramp evenly off. And what I say when I'm saying ramping evenly on and off is we're actually pushing that b- Remember pushing that brick? What we're actually watching, although we can't see it, is the cones inside these, uh, speaker grates are gonna be moving out and then moving back at an even rate on and off. So let's give this a try. Let's let the triangle perform a little bit Let's bring it down real low. Okay, can you hear it moving back and forth? We lost that pop as it would go out and back because now we've got this smooth transition, and just like you're not hearing my hand push air at you, we don't really hear the little triangle wave moving back and forth, right? So it's a bit different, which brings me to our circle here. So if you map a circle on an X, Y axis over time, you get a sine tone. So while we're looking at a circle, it does say sine because that's the sine function that we'll listen to. Sine has the same problem as triangle. We can't hear it at that low frequency, right? But as we bring it up, we start to kind of hear this fairly pure sound. Again, there's a lot of vibrato, but for the most part, it's fairly pure. And these sine tones are those things that we could see when I was speaking. It's the combination of these sine tones that produce more complicated sounds, even sounds like if I was on stage with that brick and then pushing it across and we heard scraping. Even if you combine a bunch of sine tones and those irregular ratios and at different amplitudes, you can reproduce the sound of that brick scraping across the ground. So there's a whole world of things you can do with each one of those, uh, waveforms Which brings me back to our models. So the second model I have here is additive synthesis, which is doing exactly that. It's taking each one of those sine tones, there's one graphed on the X, Y axis, and adding them to each other. So this is a website that allows us to add different sine tones to create our own waveform. It's usually pretty loud, so I'm gonna put my hand on the volume control here. All right, phew, that's nice and quiet. So right now we've got one sine tone, and here, this is at the two-to-one ratio, so if we add an octave above it, we can hear the sound change a bit. Here's the third And the fourth. And the fifth. And we can start to build new waves as we add them together. Bring the amplitude under control here So here's just a quick flashpoint of a violin and what kind of wave, uh, form we might be looking at if we add up those constituent parcels. Let's bring up the volume just a hair. And you can hear why I'm so thankful that I have great colleagues who play a real violin. So while you can try to recreate a lot of sounds from the real world using technology, what really excites me is not necessarily trying to rebuild a violin with my computer, I'm never gonna get there, but instead trying to understand a little bit about the sounds to do something different with the computer that the violin can't do. So by using these concepts, these simple concepts of adding up sine tones, I was able to map that onto the frequencies of the rhythms and create a new piece that sounded a little bit different, and I would challenge my two violin colleagues to play a bunch of metronome colors. I don't know if they can. Maybe that'll be the next library concert by the Euclid Quartet. So two more models to share with you. I've got subtractive synthesis. So we were talking just a moment ago about adding up those sine tones. We could also subtract them out. So back to our spectrogram. If we look here, I'll bring back my microphone and my wonderful singing. So we all have subtractive synthesis. Subtractive synthesis allows me to talk to you. Well, we can filter out particular partials and harmonics using our heads. So for example, if I start singing some vowel sounds, A, and I change to a different vowel, E. I'll try to stay on the same pitch this time. A, E, I, O, U, sometimes Y. Okay. So most of those kind of stay fairly the same, but what do we notice starts to happen at the tops of those vowel changes? Let's watch it one more time. A, E, I, O, U We start to filter out particular partials from that low tone. The fundamental stays the same, but we're able to change those partials at the top. We can do the same thing if we're singing, and I just simply move my mouth. Ah, wow, wow. And I'm doing that particular sound because we can actually recreate that sound using filters in synthesizers. By keeping the same fundamental, we can kind of filter out those sounds and make kind of speechy-like things. Doesn't really sound like the synthesizers start talk- talking to us, but it does allow us to do new musical things. So that brings me to the last part here, which is modular synthesis. And for this, I think I'll probably go to the table here

Speaker 4 25:22

All right.

Speaker 2 25:23

This is not prerecorded. I feel like a magician. This is me. Uh, so I brought this up here. Uh, at the end, we'll hopefully have some time. If you'd like to come out and try some of these synthesizers, you can do that. These are called Little Bits. They're very little, hence the camera. So ooh, and zoom out maybe there. You can kinda read. It says oscillator on it. It's got one knob, and all it does is it follows one pattern of a waveform. You can flip between two different ones. There's a square wave and a sawtooth wave. The sawtooth one goes up and then comes down, up, down. The square wave just turns on and off like we're used to having. So what we have here is a nine-volt battery, nothing special, and then we've got a little speaker with a headphone jack. So what I'll do, hopefully, if the battery cooperates, we'll set this up

Speaker 4 26:37

Okay, so woo-hoo, we've got a sound. I'm gonna bring it really low And now we can

Speaker 2 26:44

hear that on/off sound, right? So pretty basic But what we can do is combine that one oscillator with another oscillator. So this is another oscillator. It does the same thing. It's the exact same component. And this is where the term modular synthesis comes up. We take little modular components that do kind of the same thing, and we have them control one another. So what I'm gonna do is use this one to control some aspect of the other oscillator. So I'll put this in- inline Okay, so we can hear our on/off going at a fairly slow rate. I can change the pitch And I can change the rate So kind of exciting More exciting. So what are we listening to now? We've got one of these oscillators that's ramping down the frequency of the other oscillator. So we can hear it going really high, pew. And what does it do next? Pew, at a regular rate. So we've got one oscillator that's on a sawtooth wave moving down the frequency of the other oscillator that's moving at a rate that's causing pitch So this really basic concept is called frequency modulation. We're modulating the frequency of one oscillator with the rate of the other oscillator. Now, things get a little bit more exciting when we turn the rate that it's moving up and down really fast. I'll slow this down first. First off, now we're in some sort of sci-fi

Speaker 4 29:11

film

Speaker 2 29:25

And we're back to our cue cues. It's a very technical term. So modular synthesis allows us to control these things, uh, with just basic little components. And by the way, these kits are really fantastic, but they are unfortunately discontinued, so if you wanna play with them, you can come up at the end and try them out. But in this kit, there's a bunch of little modular circuits, and, uh, this was made by the company littleBits. They were a steam company. They built these little circuit bits, and you could do all kinds of things, make little robots, et cetera. Of course, my favorite was the one where they partnered with Korg, who makes synthesizers, and they took the components, I think it's the Korg MS-20 or something like this synthesizer. They took out all those circuit components, and then they put them in little boxes like a candy box that you can kinda take each little candy and connect them together to make different sounds. What's really cool about that is that that's the same thing that's going on in something like this. This is a semi-modular synthesizer, and it's the much smaller version of things that you used to see taking up, like, an entire room with big patch cables that were a quarter inch. We used to have one in the basement of IU South Bend that took up the entire space. And now we've got a little small box. So there's two things going on here. It's semi-modular because if I took this and I covered this up, all of these things are connected and patched together. Unfortunately, unlike the littleBits, I can't take them apart, right? So they're all wired together. What I can do, though, is use this patch bay on the side to change those connections. So I can take something that's normally routed from this oscillator to this, um, filter and send that filter to control something else on the, the modular. So I can move it and change that routing and create new and different sounds. What gets really, really fun is you can take something like this and connect it with something from a completely different company and start routing those sounds to one another and making all new tour-- types of movements and possibilities, and you can see how if you get a lot of these, there's endless things you can do with them. So, uh, let's take a listen to just this one for a moment. We'll see how the concepts on the... I'm sorry, my lav mic's kind of causing all sorts of issues here. Um, the components of the littleBits can be found on the, uh, Behringer here. So turn up my red one. There it is.

Speaker 4 32:11

All

Speaker 2 32:11

right,

Speaker 4 32:11

so we've got a big knob over here

Speaker 2 32:18

You can imagine this is our little shape from the, uh, Google page, right? So we've got our sounds, and what's nice about this one is I, I've actually got two here. So if I switch to this other side This is the other one. I can combine the two And then I've got a filter. So this is kind of like me going woo, woo, woo with my mouth. I can control that kind of filter movement Kind of sounds like me opening and closing my mouth just a, a little bit, right? Wow. Okay, that's a pretty bad joke. All right. So then the other thing here is that I can control that opening and closing without me needing to turn that knob back and forth. Can you see the blue light that's turning on and off? So that blue light is being controlled by a really slow or low-frequency oscillator. So it's turning that filter open and closed, changing the frequency cutoff by following the rate of another oscillator, and we're seeing the blue light here. So I can bring this up. Wabruda Right? And all it's doing is doing that mouth open and close, but, uh, my vocal colleagues, I think they could probably do the wow part, but I'm not sure they can do that part, right? And that's where things get kind of exciting and fun. And you can also change the shape. So right now, here we've a bunch of different shapes. This is a sine shape. More of a sawtooth. On and off. And I can get another oscillator to do this, so I don't have to move this knob. But I would need to then grab one of these nice patch cables and patch it in, right? So I'm gonna turn this down

Tools: Ableton and Max

Speaker 2 34:29

So these four models are, um, really f- fun and really dynamic ways of kind of creating all these possible sounds by turning knobs, but what if you don't have a bunch of knobs and analog synthesizers at home? Ableton is a company that produces a digital audio workstation called Ableton Live, and one of the things that they've done is put out a website for you to learn synthesis. It's called Learning Synth. And at the very end, they created this playground. Now, there's no, uh, connections, whether magnetic or by a cable, but what we do have are different modular components. So I'm gonna try to isolate just a few

Speaker 4 35:13

Forgot that I mapped this here. All right There's the sound of our square wave

Speaker 2 35:31

The sound of our sawtooth wave And I'm gonna go down here and turn my LFO off so that we, uh, do not have all of our back-and-forth sounds. There we go. Ah, much more static. Okay, so here's our square tone. Here's our sawtooth. And what we were hearing just a moment ago was that the pitch of our square wave and the pitch of our saw wave was being controlled by this moving sine tone over here. So down here, I'm able to say, all right, I want to adjust the, uh, semitones of that particular oscillator using this particular shape and speed. So I can slow it down, and we should hear something a bit like this as it's moving up and down at speed But if we bring it up a little bit faster, we start to hear new and different sounds So we can follow our sine tone shape our sawtooth, our square wave, and my favorite, which is noise. A little bit of randomness Now, it's pretty difficult to perform these things with my mouse, uh, so one nice feature that they added is this little performance pad, which allows you to map movements left and right onto the things that you would find here. So let's say that we want to change the rate at which we're moving through this LFO by going left and right, and we'll change the amount that it's adjusting the pitch by going up and down Okay, so we've got a way of doing this. You can also, uh... What I really like about this tool is you can record. Only get a minute, so you have to be really good in that one minute, and then you can record again. And you can put these together and develop your own sounds and create your own compositions. You can also take these components and export them and run them in Ableton Live, which has the exact same synthesizer background to it. So if you've got a good understanding of how this works, you can export that and run that synthesizer in a more pro-level software. So as you learn, you can then take that and apply it to other creative tools So as I mentioned before, this kind of thinking has informed the way that I put music together, but it's also kind of made me start to realize, well, when I see a musician performing an acoustic instrument, I get to see their movement. When I hear electronic music coming from one of these speakers on stage, I don't get to see the speaker moving back and forth. Even if we took the grate off, I'm not gonna s- be able to see how far it's moving and how quickly it's moving. But I do wanna experience that motion, that agency that we have when performers are moving on stage. So that's led me to kind of think, how do I provide that same experience of seeing someone move something or do something on stage, and then create a particular sound? Well, a lot of software exists to allow you to see various cool-looking shapes based on sound. I don't know if any of you have experienced the Windows Media player from back in the early 2000s, but you could play your music, and it would do all sorts of fancy sounds. And what it's doing is it's taking the signal, usually the amplitude, and then kind of generating different graphics. Now, that, that's pretty cool, but what I'm really interested is kind of breaking it down to my understanding of how music works and how I've put it together, and sharing that understanding with my audience. And what that required was for me to go back and think about how I put the sounds together and take those components and map them onto the visuals. So it really takes me thinking musically about how I would perform visual work So I use a software called Max. Uh, it has other names too, Max MSP, Max Jitter. They're currently on the eighth version, so they want me to call it Max 8. I'm just always gonna call it Max. And what it does is it gives you a blank, uh, screen. There's a window with literally nothing on it, and then you can put in a bunch of little components and connect them together with virtual patch cables. So the same idea of patching these things together, you can do with little virtual things. And the beauty of this is I've only got two oscillators in one of these little bits, but on here, I can copy and paste until my computer crashes, which I've done. Uh, so I maxed out at about 700 sine tones on a piece that I did a couple years ago, and so, uh... But I haven't done it on this laptop, so maybe I can push, like, 1,000. I don't know. And the simple copy and paste allows you to be very creative very quickly, and it also allows you to paste and copy, map those things into the visual world. Um, I did a presentation for a class at IU Indianapolis, formerly IUPUI, for those of you not in the know. Uh, and uh, I was showing some examples. They had just done a section on FM synthesis, which we listened to a moment ago, and I wanted to show how I might map some FM synthesis work onto a, something visual. So I built this little Max patch, and this is a video that I prepped for that class

Speaker 4 41:46

So I'm on the right side changing some of the speed of the oscillators. You can see some numbers moving over here, speeding things up

Speaker 2 42:02

That's plenty of that. So this is usually where I find myself working creatively these days. And, um, I wanted to give you just a quick little kind of sendoff here, which is if you're interested in this kind of technology, making your own sounds, there is a great resource so close, right down the hall in Studio 304. Studio 304, if you haven't been, is a great media workshop space. They have recording studio spaces. They've got computers, lots of graphic tools. And one of the programs that they have that's, uh, fairly expensive is Logic Pro X. It's about a $300 program, but you can use it here at your South St. Joseph County Public Library for free. And one of the things in Logic Pro X is that there's a bunch of different synthesizers that you can use, and you can apply these tools, a basic understanding of oscillators, how to combine them, in Studio 304 and make your own sounds. And I highly recommend you, uh, head over there and check that out and do that. If you, uh, want to do this on your own computer and not sure how to get started, you can also download different virtual emulations of these programs. Some are free. Uh, some students showed me one called Helm that they really liked that works for free, and you can build out these different s- sounds. If you really like name brand things, the Moog synthesizer has the Model D and, uh, a few others that you can find. And finally, there's a free version called VCV Rack that you can download and patch to your heart's content for free in a virtual environment. Copy, paste, copy, paste. Make something really exciting. Also, uh, the last thing I'll say is if you're really interested in kind of going all in on this, um, there's plenty of non-commercial, uh, products out there as well. I guess they're not products if they're not commercial. But there's code and visual programming that you can do the same kind of work. So if you're curious, I put, uh, all this up on this link here, and there's a bunch of downloads and web links that you can find. And please, if you make something interesting, send it to me too so I can hear it. Um, I think that's, uh, probably more than I had time allotted, so I'll turn it over to some questions if we have some, and then afterwards, maybe you can come up and try to make some sounds with us. Thank you. Yeah?

Speaker 6 44:26

Okay.

History, Frontiers, and Wrap-Up

Speaker 6 44:27

I do have a question. The Moog synthesizer, which was real big in music and apparently still is, so is that like the earliest that was like a, a commercial form to, uh- Kind of educate the people. I, I don't exactly know how to ask the question, but instead just being regular music, they added the synth- synthesizer and how'd it get out?

Speaker 2 44:53

Yeah. I, I'll... Let me try to answer it. If I don't answer the question- Okay bug me. So, um, when the early synthesizers, these analog synthesizers were kind of coming out, they had some really monstrous ones that were very expensive, not made for the public, like the RCA Mark II, which, like, took up an entire room. We had one at IU South Bend. There were two people that came up with smaller... And when I say smaller, they were still, like, you know, the size of an arcade game. But the two inventors was, one was Robert Moog and the other was Buchla. And the people that started to use the Buchla synthesizers were really excited about them because they started to move in a direction that was away from things like a musical keyboard, like patching different controllers and modules together. They even had a really cool interactive kind of, like, touch thing that you could play. Moog, on the other hand, put a keyboard with his synthesizers, and immediately it made it more accessible to musicians who were trained to use a keyboard. So they can make different sounds, but they could also play things on the keyboard. Probably most famously, Wendy Carlos put out an album called Switched-on Bach, which was a huge commercial success, and she used a early Moog synthesizer to perform all of that music. She was an organist, so it was immediately accessible for her to play all of that Bach music just like she would on a pipe organ, but was then able to use all these new and exciting timbres. So I think the Moog became so popular because it appealed to a lot of musical training that a lot of people had. But of course, then people were able to veer off in different directions. And yes, they're still widely popular. So we've... have got a small one up here. Did I, did I answer it? Okay, great. Are there, uh, any other questions? Yeah.

Speaker 7 46:39

That Max piece that you had up there, is that analogous to a score? Oh. I mean, could somebody look at that and say, "Oh, I see. This is what's gonna happen here," and...

Speaker 2 46:51

Yeah, uh, kind of. So a score, if we really, really step far, far back on a score, it's a set of instructions for physical movement, right? So it's a weird way to kind of think about it, but e- essentially that's what we're doing when I give a score to another musician. So in that particular patch, I didn't put any good comments about what to do, 'cause I knew I was gonna do it. But if I put a bunch of comments, "Number one, turn the audio on. Number two, move the sine tone up and down. Number three," then I've got a set of instructions for someone to s- to perform, and I, I would, yes, call that a score. So I think any time you give someone a set of instructions, you've got a musical score on your hands.

Speaker 10 47:36

I have a question. So I'm curious with, uh-- So when I think about electronic music, sort of initially, it was more the sound, producing sound, and now it seems it's more mapping visuals and maybe more complex sound. And I'm wondering, what's the next frontier?

Speaker 2 47:52

Oh. Um, okay, great question. So I'll just repeat the last part. So what is the next frontier? So I, I appreciate that you think, uh, from my presentation that, you know, the visuals are more exciting or com- uh, complex. I think it's just me. You're hearing a biased opinion. But, uh, what a lot of people are really interested in these days, I think like in many different disciplines, is AI. Um, but probably more accurately, machine learning. So taking sounds and training a computer to listen for particular components, you know, we talked about listening to partials, the things that make up that sound, training to listen for particular timbres. And some of the more exciting pieces I've seen that do that listen to sounds as they're being performed and then react based on what they're hearing. Uh, and so what that allows us to do is to have more improvisational opportunities on stage with electronic music. Uh, and then I think the only other one that, uh, comes to mind right away is kind of the new commercial development of 3D immersive sound, whether that's with, uh, VR or AR capabilities, or if you're in a concert hall setting like this one, and they have a Dolby Atmos system, which is a new commercial product to allow us to hear sound from left and right. You can hear sound from behind you, but kind of the new exciting thing is hearing sound above you. We don't do that so well as humans, but, uh, we can get scared by things above us, right? So, uh, that's kind of been the recent developments that a lot of composers have been working with. So there's probably more that I'm missing.

Speaker 8 49:32

So this might be a little bit off topic from what you do, but it's thinking about making-- turning music into a visual, it made me wonder is, is there an effort or have you thought about making music more ac- accessible, like, uh, to hear someone who is hearing impaired, for example?

Speaker 2 49:53

Yeah. So that's moving a little bit out of my field of knowledge, so, um, please take that as a huge caveat. But, uh, there are a lot of people who work in that area that are constantly trying to think about ways that technology can make things more accessible to a general public. So I'll probably stop there before I say something incorrect. But music therapists are, uh, probably a huge field of knowledge in that way, not only from a listening perception-based, but also an experiential, uh, base side of things. How can we get-- allow people to perform music in ways that might be less ableist from our, our previous understanding of how one performs and moves through music? I don't know if that answered your question. Maybe leads to future possibilities. So yeah. Anything else?

910 50:45

So when you're mapping out some visual representation from a piece that you've composed, do you find that you have to work to make it feel analogous to the sound? Or because you're basing it on the parameters you're adjusting, does it kind of come out fully formed like a beat?

Speaker 2 51:03

Oh, that would be great. So the question was, uh, when I'm working on mapping, does it, you know, always kind of work out, right? Maybe quick summation. The answer is no. Uh, I... There's a lot of tweaking, um, because I have a lot of ideas about what it should look like when I'm hearing a particular thing, or especially if I'm composing it. That example that I played just a moment ago, you know, we're hearing particular things, and sometimes the mapping doesn't really kind of cohere to what maybe we're expecting it to be doing. Some of this is a restriction based on some of our visual technology. For example, I'm dealing with frame rates when I'm looking at something that's visual. So I've got thirty frames or sixty frames per second, and then I'm moving that speaker at a particular rate per second. And if I'm moving at different rates, I might be seeing things at different times than I'm hearing them. The wagon wheel is like that typical experience where it's spinning forward, and then if the wheel starts going fast enough and we're watching film, it starts to go backwards because our frame rate speed is too slow for the speed of the wheel moving forward. And that same thing happens when you start mapping music or acoustic things to, to visual parameters. So yeah, I'm constantly coming up with stuff that looks terrible, and then that's the art side of it, right? So then I get to do things differently 'cause we get to do that in art. So Yeah.

Speaker 11 52:28

Uh, so you're-- you generate visuals based on music or sound. Do you ever do anything in the reverse direction, start with visuals and then try to extract music from?

Speaker 2 52:39

Yeah. Um, well, uh, f- so the quick answer is yes. There's a great program that you can use in Studio three oh four. Uh, Logic Pro has a synthesizer called the Alchemy Synth. It's very complicated, but there is one particular parameter that uses that same spectral imaging that we looked at in the spectrogram, but you can import a picture. So, um, uh, I, I did a presentation for Studio three oh four, and the, the James Webb visual had just come out, like, the day before. So I immediately threw that into the Alchemy synthesizer and just hit play on one note, and it ran through, and with each star or galaxy... So you clearly know I don't know what I'm talking about. Uh, it would start making sound, and it was great. But, uh, for me, that, that-- it really depends on your interests. Um, there's others. A really fun kind of YouTube rabbit hole is a genre called dark MIDI, which sounds really dark, but it's actually typically quite playful, and it's people who make pictures, like video game images, with notes. So what you'll see is, like, usually it starts off with, like, a nice little param- like, pattern going and then all of a sudden you'll see, like, Mario, and it will play, like, the sound of Mario as a bunch of notes. So it's a bit silly, but there's a lot of people out there that kind of think in this way and try stuff like that. For me, as an artist, most of the time it's not, um, music than visuals or visuals than music. I'm usually trying to think of the whole experience for the people that are listening and what, what is the full experience? What do I wanna tell with the visuals? What do I wanna play with the music and kind of heading in that direction?

Speaker 4 54:25

Okay. Thank you so much