Speaker 1: Get in touch with technology with tech Stuff from how
Speaker 1: stuff works dot com. Hey everybody, and welcome to tech Stuff.
Speaker 1: I'm Jonathan Strickland. I'm the host of the show, and
Speaker 1: this is a Saturday morning rerun episode where we take
Speaker 1: a classic episode of tech Stuff and we present it
Speaker 1: to you guys who may have missed it. I've been
Speaker 1: talking a lot about tech and music recently. If you've
Speaker 1: been listening to the recent episodes, you know all about that,
Speaker 1: and there have been some great discussions. But it also
Speaker 1: requires a little bit of uh knowledge of previous episodes
Speaker 1: at times, and I know it can be tricky to
Speaker 1: dig through the archives. So in this classic episode, I
Speaker 1: talk about how the MP three compression format works, so
Speaker 1: that you can actually understand how MP three works as
Speaker 1: opposed to something like middy, and you can get an
Speaker 1: appreciation for the differences between the two formats. This episode
Speaker 1: originally published on January two thousand and seventeen. This is
Speaker 1: a whole year ago more than that. Now we're in
Speaker 1: April two eighteen as I record this. I hope you
Speaker 1: enjoyed this classic episode. I hope it gives you a
Speaker 1: deeper appreciation of the technical aspect of creating digital music
Speaker 1: and I'll see you guys on the other side. So
Speaker 1: let's remember that the heart of digital information is the
Speaker 1: bit that's either a zero or a one. The basic
Speaker 1: unit of information for digital formats zeros and ones. Now
Speaker 1: we can use those zeros and ones to describe all
Speaker 1: sorts of information, from text to audio, to video and
Speaker 1: really pretty much anything you can think of that's represented digitally. Ultimately,
Speaker 1: when you get down to it, it's a bunch of
Speaker 1: zeros and ones. So let's say you start off with
Speaker 1: your uncompressed audio file. You've got this enormous audio file
Speaker 1: in front of you. It's made up of zeros and ones.
Speaker 1: How do you make that file smaller? So in the world,
Speaker 1: we can compress stuff, right, we can apply physical pressure
Speaker 1: to things. Think about packing a suitcase. You can make
Speaker 1: sure you get that extra outfit and if you just
Speaker 1: press it down hard enough and get that zipper zipped
Speaker 1: before it can burst open. But once you get to
Speaker 1: a certain level of compression, you cannot make things smaller,
Speaker 1: at least not without hurting yourself or whatever it is
Speaker 1: you're trying to compress. Digital files are a little different
Speaker 1: because you cannot physically cram the zeros and ones closer together.
Speaker 1: It doesn't work like that. These are abstract things. You
Speaker 1: can't make them smaller, right, You can't decrease the font.
Speaker 1: It doesn't work that way. The numbers represent two different states.
Speaker 1: So if you want to create a smaller audio file
Speaker 1: containing the recording that was in a larger audio file,
Speaker 1: you have to start getting creative now. In the last
Speaker 1: part of this series, I talked about how the MP
Speaker 1: three compression algorithm was born from an applied research institution
Speaker 1: in Germany and the team behind the m B three
Speaker 1: wanted to find a way to compress audio, specifically music
Speaker 1: for transmission over phone lines. Eventually this evolved into the
Speaker 1: Motion Pictures Expert Group audio Layer three compression methodology, better
Speaker 1: known as the MP three, and there's also Impact two
Speaker 1: and IMPEG four standards. Impact two, by the way, is
Speaker 1: the basis of compression on DVDs, although the actual DVD
Speaker 1: format is really a modification of Impact two. An Impact
Speaker 1: four is a compression strategy for audio and video that's
Speaker 1: frequently used in lots of different up capacities, including streaming
Speaker 1: media services. So by the late nineteen seventies, researchers began
Speaker 1: to explore the possibility of leveraging psychoacoustics to figure out
Speaker 1: how to compress audio. And psychoacoustics refers to the way
Speaker 1: we perceive sound, it's uh and also the physiological effects
Speaker 1: of sound on us. So this involves not just our
Speaker 1: our physical sense of hearing, but also our brains and
Speaker 1: the way our brains interpret sound. Owned So, for example,
Speaker 1: there's a psychoacoustic phenomenon that's called the Hawse effect h
Speaker 1: A A S. And I think it's pretty interesting. So
Speaker 1: here's how the Hawse effect works. If you hear the
Speaker 1: exact same sound coming from different directions, but the two
Speaker 1: sounds arrive within thirty to forty milliseconds of each other,
Speaker 1: your brain will be convinced that you really only heard
Speaker 1: one sound and it came from the direction that hit
Speaker 1: you first. So let's say a sounds coming from directly
Speaker 1: in front of you and to your left, and you
Speaker 1: get both of them within that thirty forty millisecond range,
Speaker 1: and you hear the one coming from ahead of you
Speaker 1: first to you. You're convinced that you only heard that
Speaker 1: sound once and it came from dead on straight ahead
Speaker 1: of you. Your brain kind of discounts the one that
Speaker 1: came off from the left, although it can reinforce it,
Speaker 1: which ends up being really useful if you're planning out
Speaker 1: p A systems for stage shows. I'm not joking. That
Speaker 1: really is the way that uh people plan those things out.
Speaker 1: It's pretty neat. Humans perceived sounds in a way that's
Speaker 1: not necessarily representational of all the sounds surrounding us. You
Speaker 1: can think of your brain as the filter between your
Speaker 1: understanding and what reality actually is. A lot of stuff
Speaker 1: goes on that it ends up getting rid of information
Speaker 1: that your brain just says, you know what, he or
Speaker 1: she doesn't need that, it's just gonna confuse things. We're
Speaker 1: gonna dump it. And that's kind of how it works.
Speaker 1: It's all on an unconscious level. It's not like you're
Speaker 1: actively working to do this. So let's say you're in
Speaker 1: a relatively busy hallway and there could be a lot
Speaker 1: of sounds in that hallway. Stuff that's going on constantly
Speaker 1: around you. Maybe they are doors opening and closing, Maybe
Speaker 1: their footsteps going up and down the hallway. Maybe someone
Speaker 1: shoes are squeaking against the linoleum floor. People are chattering
Speaker 1: away in there. But you are having a conversation with someone,
Speaker 1: so you turn your focus on that person and other
Speaker 1: sounds seemingly fade away. They're still doesn't but they're not important.
Speaker 1: So in this example, you would actually call those other
Speaker 1: sounds of distraction and you would really focus on the conversation. Uh.
Speaker 1: That also shows how we're able to consciously direct our
Speaker 1: since our perception of hearing. So both of these factors
Speaker 1: come into play. Now. One thing that MP three encoding
Speaker 1: takes advantage of is something called masking, and there are
Speaker 1: a couple of different variations of the masking effect. One
Speaker 1: of them is called frequency masking. So let's say you've
Speaker 1: got to sound frequencies that are similar, perhaps there's just
Speaker 1: a few hurts apart. Remember, UH, frequencies are measured in hurts,
Speaker 1: which is really the number of oscillations per second. So
Speaker 1: let's say you've got a sound that's at I don't know, uh,
Speaker 1: one thousand killer hurts, and another one that's at one
Speaker 1: thousand and ten killer hurts. Now, the human ear is
Speaker 1: precise enough to be able to tell the difference of
Speaker 1: two sounds that are at least two hurts apart from
Speaker 1: each other. That's how precise our resolution of hearing, it's
Speaker 1: it's at that level. But if you get two sounds
Speaker 1: played at the same time and they are that close
Speaker 1: together in frequency, and one of those frequencies is played
Speaker 1: at a greater volume than the other, our brains will
Speaker 1: pick up on the louder sound and ignore the quieter sound,
Speaker 1: even though both of them are present. What becomes important
Speaker 1: at that point is the amplitude. Now, the further apart
Speaker 1: in frequencies you get, the less that has an effect.
Speaker 1: So if you get far enough apart where there are
Speaker 1: two pitches, one of them noticeably louder than the other,
Speaker 1: but they're far enough apart, you will hear both of them.
Speaker 1: It only works if the two pitches are relatively close together,
Speaker 1: and there's not a universal formula for frequency masking. As
Speaker 1: you get closer to the boundaries of human hearing, frequency
Speaker 1: masking becomes easier, So if it's a really low pitch
Speaker 1: or a really high pitch, it's easier to get away
Speaker 1: with it. Once you started getting into what is the
Speaker 1: out of as the sweet spot for human hearing, which
Speaker 1: is generally considered to be between two and five killer hurts,
Speaker 1: you need a greater difference in volume or a smaller
Speaker 1: difference in frequency in order for masking to work. Frequency
Speaker 1: masking at any rate. But then there's also temporal masking,
Speaker 1: and you might say, okay, I got it. Temporal that
Speaker 1: means time. Indeed it does, my friend. This describes the
Speaker 1: effect of a short but loud sound masking a softer
Speaker 1: sound for a short time. Weird thing is the loud
Speaker 1: sound can actually mask sounds that precede it slightly, not
Speaker 1: by a whole lot, but a little bit. MP three
Speaker 1: compression takes advantage of both frequency and temporal masking when
Speaker 1: it's trying to determine which data needs to be included
Speaker 1: and which data can be dumped, because it won't affect
Speaker 1: your perception of whatever the the audio file is in
Speaker 1: the first place. So you also probably remember I talked
Speaker 1: about the physical limitation to what we humans can hear,
Speaker 1: no matter what our brains might be up to, so
Speaker 1: that this doesn't have to do with our brains, you know,
Speaker 1: filtering through the information that's coming in. This has to
Speaker 1: do with the physical limitations of the human ear. In
Speaker 1: the last episode of the series, I said typical human hearing.
Speaker 1: Keep in mind typical there are exceptions. UH covers the
Speaker 1: range of frequencies between about twenty hurts and twenty killer
Speaker 1: hurts or twenty thousand hurts, So twenty to twenty thou
Speaker 1: higher frequencies represent higher pitches and sound lower frequencies lower pitches, right,
Speaker 1: And as you get older, your ability to perceive those
Speaker 1: higher frequencies starts to diminish. So most adults actually have
Speaker 1: an upper range closer to sixteen killer hurts, not twenty. Uh. Kids,
Speaker 1: they can hear those higher pitches. You may have heard
Speaker 1: the story about how some convenience stores experimented with getting
Speaker 1: rid of teenage loiterers by by uh projecting out these
Speaker 1: super high pitches that that adults could not here but
Speaker 1: kids could, and it discouraged kids from hanging out at
Speaker 1: the convenience store and loitering. Um. I love that idea
Speaker 1: so much. Anyway, that's because I'm old and my hearing
Speaker 1: is terrible. Well, remember I also mentioned you can detect
Speaker 1: changes in pitch at two hurts increments if you get
Speaker 1: below two hurts and change, like, if it's just a
Speaker 1: one hurts difference between two frequencies, it's too low a
Speaker 1: resolution for us to detect. To us, it will sound
Speaker 1: exactly the same. So if you were to hear a
Speaker 1: frequency at one thousand one hurts or one point zero
Speaker 1: zero one killer hurts and one point zero zero to
Speaker 1: kill hurts, you wouldn't notice the difference. They would sound
Speaker 1: exactly the same to you. So if you're gonna take
Speaker 1: audio and compress it, one step you could consider is
Speaker 1: eliminating anything that's outside the actual range of frequencies that
Speaker 1: we can hear, or simplifying any changes in frequency that
Speaker 1: are smaller than two hurts. If you get take all
Speaker 1: that data and you say it is physically impossible for
Speaker 1: a human to perceive this, get rid of that information,
Speaker 1: then in theory it wouldn't have any effect on the
Speaker 1: rest of the recording. But how you go further than that, right,
Speaker 1: how do you create a method so that you can
Speaker 1: really compress this file? You want a method that will
Speaker 1: preserve the important sounds while potentially ignoring all the unimportant
Speaker 1: or incidel sounds. And you wanted to be automatic because
Speaker 1: if you have it manually, then that's going to take
Speaker 1: countless hours just to edit a single sound file. So
Speaker 1: that was the challenge that the MP three research team
Speaker 1: faced as a group. Now, their solution, which ultimately created
Speaker 1: even more challenges was to come up with what was
Speaker 1: essentially a simulated human ear and brain. They needed to
Speaker 1: replicate the experience of perceiving music so that an algorithm
Speaker 1: could evaluate every sound in an audio file and judge
Speaker 1: if in fact was relevant enough to include in the
Speaker 1: final compressed version. If a sound were imperceptible, then it
Speaker 1: wouldn't make sense to include it in the MP three file.
Speaker 1: So by leaving out all the irrelevant data, they can
Speaker 1: make the audio information take up less bandwidth. The file
Speaker 1: itself would be smaller because you just dumped everything that
Speaker 1: wasn't important. So the team used an algorithm called the
Speaker 1: low complexity Adaptive Transform Coding or lc DASH a TC
Speaker 1: as the foundation for their research. This was kind of
Speaker 1: their starting point, and this is an approach that that
Speaker 1: tries to do away with redundancy as much as possible,
Speaker 1: and it also incorporates adaptation to perceptual requirements. Also, MP
Speaker 1: three's oh a lot to the IMPEG Layer two standard,
Speaker 1: So the Layer two obviously came out before Layer three,
Speaker 1: and so a lot of the features of layer three
Speaker 1: are really um their legacy features from Layer two. Uh.
Speaker 1: In other words, MP three group kind of got stuck
Speaker 1: with them because otherwise they would have had a problem
Speaker 1: with backwards compatibility. So the result is kind of a
Speaker 1: clunky arrangement under the hood, and some of the features
Speaker 1: may make very little sense when I go through them,
Speaker 1: but some of that is because it's a holdover from
Speaker 1: an earlier compression strategy, which isn't terribly satisfying as an answer.
Speaker 1: But the reason many parts of the MP three compression
Speaker 1: algorithm are the way they are is because that's the
Speaker 1: way we've always done it. So next I'm gonna dive
Speaker 1: into the phases of compression. But before I do that,
Speaker 1: let's all take a deep breath and take a moment
Speaker 1: to thank our sponsor, and we're back. So there are
Speaker 1: two big phases we'll need to talk about with MP
Speaker 1: three compression. The first phase is analysis and the second
Speaker 1: phase is the actual compression itself. And after that there's
Speaker 1: the process of decoding and MP three for playback. But
Speaker 1: that's way simpler once we get an understanding of how
Speaker 1: the encoding process actually happens. So let's begin with analysis. Now.
Speaker 1: This is the part where the standard has to figure
Speaker 1: out which frequencies within an audio range are recording rather
Speaker 1: are important or perceptible. So how does a program and
Speaker 1: encoder figure out what we can hear and what we
Speaker 1: cannot hear? Alright, time to get technical. So you start
Speaker 1: off with your pulse code modulation audio file or PCM file.
Speaker 1: And you might remember I talked about PCM audio in
Speaker 1: the first episode of this series, but just in case
Speaker 1: you don't, it's a lossless digital audio file. The actual
Speaker 1: format could be a wave or ai f F or
Speaker 1: something along those lines, but the important thing to keep
Speaker 1: in mind is that it is uncompressed. Now, that means
Speaker 1: those files tend to be pretty big. This is our
Speaker 1: raw material that we want to take and squish down
Speaker 1: to a more manageable transferable size. And in our our
Speaker 1: last episode in this series, I also mentioned that the
Speaker 1: standard for c D audio is a sample rate of
Speaker 1: forty four point one killer hurts. And we learned that
Speaker 1: you need a sample rate twice the frequency of the
Speaker 1: highest frequency in your recording, and since human hearing tops
Speaker 1: out at around twenty kill hurts, the standard for c
Speaker 1: ds is forty four point one killer hurts. The MP
Speaker 1: three standard can support lots of different sample rates, but
Speaker 1: forty four point one killer hurts is pretty much the
Speaker 1: common standard. So you've got a number of samples with
Speaker 1: your audio file, and that number will depend upon how
Speaker 1: long the audio file is. You've got forty four samples
Speaker 1: per second, actually twice that for stereo. But for the
Speaker 1: purposes of this discussion, let's kind of stick with mono
Speaker 1: sound so that I don't start having math coming out
Speaker 1: of my ears. And we're still in the very easy,
Speaker 1: simple part as far as math goes. We haven't gotten
Speaker 1: to the complicated stuff yet. All right, So you've got
Speaker 1: forty four thousand, one hundred samples per second. To compress
Speaker 1: it into an MP three format, the algorithm first groups
Speaker 1: all of these samples into collections called frames. So take
Speaker 1: those four thousand one per second, and then you start saying, okay,
Speaker 1: we're gonna group you in batches. Each batch is called
Speaker 1: a frame, and each frame contains one thousand, one fifty
Speaker 1: two samples. Now that's specifically to maintain backwards compatibility to
Speaker 1: IMPEG Layer two, which established that one thousand, one fifty
Speaker 1: two number. But we're not talking about IMPEG layer two.
Speaker 1: We're talking about IMPEG Layer three, and though that means
Speaker 1: we have to get a little more complicated. So each
Speaker 1: frame consists of two subgroups called granules. So each granule
Speaker 1: has five hundred seventy six samples six times two one two,
Speaker 1: so five seventy six samples per granule. Now, technically MP
Speaker 1: three encoders only work on one granule at a time,
Speaker 1: but they may reference the granules immediately before and immediately
Speaker 1: after the current one in order to see how the
Speaker 1: audio within the file changes over time. All right, So
Speaker 1: now you've got your granules of five hundred seventy six
Speaker 1: samples each. Then the MP three encoder runs the samples
Speaker 1: through a filter bank, which sorts the sound into thirty
Speaker 1: two frequency ranges. Are you? Are you crazy about the
Speaker 1: numbers yet, Dylan? Are you? Dylan's Dan's nodding. Dylan gets
Speaker 1: worse from here. So you have thirty two frequency ranges,
Speaker 1: which is another nod to the layer two method, which
Speaker 1: use those thirty two ranges for encoding purposes. But we're
Speaker 1: not talking about layer two, are we. No, we're talking
Speaker 1: MP three. Gosh darn it. That means we take those
Speaker 1: thirty two ranges and we subdivide them by a factor
Speaker 1: of eighteen. That means we have five hundred seventies six
Speaker 1: bands of frequencies each band containing one seventy six of
Speaker 1: the frequency range of the original sample. So what that
Speaker 1: actually means and this this is actually pretty easy. The
Speaker 1: bands are not limited to a specific number for their
Speaker 1: frequency range, right. The bands don't mean that on the
Speaker 1: on band number one it goes from twenty hurts up
Speaker 1: to a certain range, and on band five D seventy
Speaker 1: six it ends at twenty killer hurts. That's not what
Speaker 1: it means. They're dependent upon the original audio. So if
Speaker 1: the original audio contains sounds within a narrow range of frequencies,
Speaker 1: the five seventy bands will be more precise. But if
Speaker 1: the original recording has a vast range of frequencies, the
Speaker 1: bands are less precise. So another way to think about
Speaker 1: this is with a pizza. So let's say you get
Speaker 1: extra large pizza and you cut it into eight equal slices,
Speaker 1: and then you get a small pizza and you cut
Speaker 1: that into eight equal slices. Well, in both cases you
Speaker 1: have with each slice one eighth of a pizza. But
Speaker 1: the extra large pizza pizza slice is bigger than the
Speaker 1: small pizza pizza slice. It all depends on the size
Speaker 1: of the pizza. So in this case, it depends upon
Speaker 1: the range of frequencies. And and Dylan, do you think
Speaker 1: we could go for some pizza, you know, just just
Speaker 1: put the episode on hold and go get pizza. Dylan's nodding.
Speaker 1: It's great for audio. Yeah, so, uh, pizza, We'll be
Speaker 1: right back. Okay, I was good pizza. Now um oh, man,
Speaker 1: I got a whole bunch more notes. Okay, well, let's
Speaker 1: let's go ahead and and do the rest of this.
Speaker 1: All right, So you've got your sound divided up into
Speaker 1: those five seventy six sub brands of frequencies, you know,
Speaker 1: the thing I compared to pizza slices earlier. Now you
Speaker 1: get two different mathematical processes applied to this data. One
Speaker 1: is the fast Furrier trans form or f T, and
Speaker 1: the other is the modified discrete Cosine transform or m
Speaker 1: d c T. Now, I am not going to dive
Speaker 1: deeply into how these transforms work, because frankly, they are
Speaker 1: beyond my mathematical understanding. But I know what they do.
Speaker 1: I just cannot explain the process like how they do
Speaker 1: what they do. So I'm going to give you the
Speaker 1: explanation of what they do. What the outcome of each
Speaker 1: of these transformed processes happens to be, but I'm not
Speaker 1: going to be able to tell you the actual mathematical
Speaker 1: steps involved in each because I don't math. So good guys,
Speaker 1: But let's start with a fast for your transform. So
Speaker 1: transform is kind of what it sounds like. It's all
Speaker 1: about transforming information in some way. So in this particular case,
Speaker 1: the f f T transforms the frequency bands we just
Speaker 1: talked about into data that can be further analyzed by
Speaker 1: a psychoacoustic model that's in the encoder. So this is
Speaker 1: that simulated human ear and brain we were talking about earlier.
Speaker 1: So what the encoder does is it analyzes each bit
Speaker 1: of data and looks for signs that it represents audio
Speaker 1: that wouldn't be perceived by a human. So it's look
Speaker 1: looking for any potential for masking possibilities. So are there
Speaker 1: collections of frequencies that are grouped close together, and is
Speaker 1: one of those frequencies louder than the others. You might
Speaker 1: be able to do away with those softerw frequencies because
Speaker 1: of frequency masking. The encoder will also look at whether
Speaker 1: or not the audio has a lot of complexity to it,
Speaker 1: if it has a lot of changes, or if it's
Speaker 1: just relatively steady or simple audio. Any transient sounds that
Speaker 1: are present in the audio might end up being temporal masking,
Speaker 1: so it'll analyze those as well and see if that's
Speaker 1: a possibility. So really what they're looking is for, you know,
Speaker 1: just any really loud sounds that stand out above the
Speaker 1: rest of the recording. That's what the f f T
Speaker 1: is doing. So what about the modified discrete cosine transform. Well,
Speaker 1: this is happening in parallel with the f f T,
Speaker 1: and the samples get sorted into different patterns called windows. Uh.
Speaker 1: And the criterion for sorting all has to do with
Speaker 1: whether the sample represents a steady sound or varied sound.
Speaker 1: So if you have a simple steady sound that goes
Speaker 1: into a long window. If there's a lot of variation
Speaker 1: in the sound, like there are a lot of consonants
Speaker 1: in a vocal line, or it's like a drum solo
Speaker 1: or something like that, it would get sorted into a
Speaker 1: series of three short windows. And each short window contains
Speaker 1: one two samples. That amounts to four whole milliseconds, so
Speaker 1: four thousands of a second in three patterned windows. So
Speaker 1: you've got these windows now, either long windows for simple
Speaker 1: sounds or short windows for the more complex sounds, and
Speaker 1: then the modified discrete cosine transformed kicks into gear. It
Speaker 1: looks at each long window or set of three sort
Speaker 1: windows and converts them into a set of spectral values.
Speaker 1: To some of you, that probably sounds meaningless. So let's
Speaker 1: talk about spectral analysis for a second. First, I was
Speaker 1: very disappointed to learn that spectral analysis doesn't involve a
Speaker 1: psychologist talking to a ghost about its emotional state. So bummer.
Speaker 1: But spectral analysis is when you look at a spectrum
Speaker 1: of information, like a spectrum of frequencies or related information
Speaker 1: like energy states. That's what this transform does. It takes
Speaker 1: data that originally represented a slice of time in a
Speaker 1: sound waveform. That's what sample is. A sample is an
Speaker 1: instance of time in a wave form and converts it
Speaker 1: into information representing sound as energy across a range of frequencies. Now,
Speaker 1: you can plot out spectral information in a lot of
Speaker 1: different ways, but one common method is to use brightness
Speaker 1: to indicate energy levels. Higher energy levels are brighter patches
Speaker 1: in your vision. Dual representation of spectral data. High frequencies
Speaker 1: would appear at the top of a spectral view like
Speaker 1: imagine a box, and at the top of the box
Speaker 1: that's where you would find high frequencies. At the bottom
Speaker 1: of the boxes where you find low frequencies, and it's
Speaker 1: just lots of patches of color. The really bright patches
Speaker 1: of color represent very high energy frequencies, so they could
Speaker 1: be high or low in in actual frequency, but we're
Speaker 1: talking about energy levels, not whether it's a higher low pitch.
Speaker 1: Looking left or right represents the passing of time, and
Speaker 1: looking along any vertical points shows you the actual frequency
Speaker 1: or pitch, and then the respective energy level is the brightness.
Speaker 1: So it's kind of like looking at sound as a wave,
Speaker 1: but instead of being a wave, you're looking at information
Speaker 1: that indicates frequency range and energy level. That representation is
Speaker 1: actually kind of analogous to how we hear audio, So
Speaker 1: and encoder can analyze the spectral view and start to
Speaker 1: filter out the data we would and perceived due to psychoacoustics. Now,
Speaker 1: after all that processing, the encoder looks at the frequency
Speaker 1: sub brands and the levels of spectral intensity for each
Speaker 1: and that information can then be used for the next phase,
Speaker 1: which is compression. But right now I think we could
Speaker 1: all stand a little decompression, So let's take another quick
Speaker 1: break to thank our sponsor all right, So now you're
Speaker 1: ready to compress your analyzed audio. Good for you, and
Speaker 1: by you I mean encoders. This has to be simpler
Speaker 1: than that analysis segment, right, I mean that got a
Speaker 1: little crazy with all the different bands and sub bands
Speaker 1: and windows and frames and granules. Sadly it gets more complicated.
Speaker 1: All right. So there are two layers of compression going
Speaker 1: on with IMPEG Layer three. One of those layers depends
Speaker 1: upon the psychoacoustic analysis and the other doesn't. So why
Speaker 1: would you use two layers with different strategies like that? Well,
Speaker 1: the reason is that one strategy is great for complex
Speaker 1: audio with lots of components, but not so great with
Speaker 1: simpler sounds, and the other strategy is kind of the opposite.
Speaker 1: So the psychoacoustic approach is the one that's really good
Speaker 1: for complicated sounds. If if you've got a lot of
Speaker 1: volume changes, lots of different frequencies, it's just complicated and
Speaker 1: rich sound. You've got a lot of opportunities to look
Speaker 1: for masking and other acoustic elements that limit the actual
Speaker 1: sounds that people perceive. So it means there are a
Speaker 1: lot of chances for you to uh fudge by dropping
Speaker 1: all the stuff that people probably wouldn't notice anyway. And Uh,
Speaker 1: if you take a piece that's got a lot of
Speaker 1: elements at varying volumes, there are likely several opportunities to
Speaker 1: to do this. But if you're talking about relatively straightforward
Speaker 1: audio with few components, few changes in volume, there's really
Speaker 1: not a whole lot of data you can ditch without
Speaker 1: it actually affecting the quality of the audio in a
Speaker 1: perceptible way. And this is part of what Brandenburg, that
Speaker 1: guy I was talking about in our first episode in
Speaker 1: this series. Uh, that's when he discovered when he was
Speaker 1: working with the MP three standard and he was listening
Speaker 1: back to that Suzanne Vega acapella track Tom's Diner. Uh,
Speaker 1: he was listening to a compressed version of it, and
Speaker 1: he said it was terrible. He said it ruined the
Speaker 1: quality of the audio. And part of that is because
Speaker 1: that particular song is fairly simple, there's just not a
Speaker 1: lot of opportunity to take advantage of masking and other
Speaker 1: tricks without potentially compromising the quality. So they decided to
Speaker 1: also incorporate some traditional compression strategies which which worked better
Speaker 1: with those types of recordings. So the MP three format
Speaker 1: takes advantage of both the traditional approach and the psychoacoustic approach,
Speaker 1: and that allows the encoder to compressed files into smaller
Speaker 1: size without just following a single strategy, like it doesn't
Speaker 1: have to do a one size fits all for all
Speaker 1: elements of audio. Now, combining those two strategies requires a
Speaker 1: little more mathematical gymnastics. So let's go back to those
Speaker 1: five seventy six frequency bins. You know, those sub bands
Speaker 1: we talked about earlier. You gotta quantize those suckers. What
Speaker 1: does that mean. It means assigning a quantity to each
Speaker 1: to each frequency bin, you have to give it a
Speaker 1: quantity of some sorts so that you can end up
Speaker 1: judging how much you can get away with dropping data.
Speaker 1: So to do this, the encoder sorts those five six
Speaker 1: bins into twenty two scale factor bands. How you doing
Speaker 1: over there, Dylan? Just checking in on you? Okay, Dylan's
Speaker 1: got Dylan's got a thousand yards stare going. I hope
Speaker 1: you guys are doing okay over there? All right, So
Speaker 1: before smoke starts coming out of your ears, let me
Speaker 1: explain what the scale factor bands are all about. The
Speaker 1: whole purpose of the scale factor bands is to determine
Speaker 1: how the information will be stored within the compressed state.
Speaker 1: So you want to get away with as little data
Speaker 1: as possible before affecting sound quality. So if you can
Speaker 1: say the same thing in a shorter space without affecting
Speaker 1: the quality of what it is you're saying, you go
Speaker 1: with it. Brevity is the soul of compression. So if
Speaker 1: we were talking about language, I would say it's more
Speaker 1: efficient to say it's raining outside, or even just it's raining,
Speaker 1: because you would assume that it would be outside where
Speaker 1: the rain is happening, and it would be inefficient for
Speaker 1: me to say it's coming down like cats and dogs
Speaker 1: out there. It's not as efficient as saying it's raining.
Speaker 1: So if you can get away with shorter statements without
Speaker 1: affecting the actual quality, and you could argue that by
Speaker 1: switching from it's coming down like cats and dog out
Speaker 1: there and it's raining changes the quality, and that could
Speaker 1: be a valid argument. But if you can get away
Speaker 1: with shorter without affecting quality, you do it. So each
Speaker 1: scale factor band is represented by a quantity, Then the
Speaker 1: encoder divides that quantity by a given number called the quantizer,
Speaker 1: which is the same across the entire frequency spectrum for
Speaker 1: that recording. The resulting number is then rounded up or
Speaker 1: down to a whole digit. And here's an important point.
Speaker 1: Individual scale factor bands can be scaled up or down
Speaker 1: for more or less precision to represent the actual value
Speaker 1: of those bands. So what the heck does all that mean? Well,
Speaker 1: the purpose of dividing and rounding is just to simplify
Speaker 1: the data to reduce the amount you need in order
Speaker 1: to store the information. So let's go with a totally
Speaker 1: hypothetical example. Let's say you've got a scale factor band
Speaker 1: and you've decided your rep is sending that scale factor
Speaker 1: band with the quantity seven eight four zero seven thousand,
Speaker 1: eight hundred forty, and you've chosen the number one hundred
Speaker 1: to quantize your data, meaning that you will divide each
Speaker 1: uh scale factor bands quantity by one hundred. So this
Speaker 1: is seven thousand, eight hundred forty. You divide it by
Speaker 1: one hundred UH, and the scale factor for this particular
Speaker 1: band you have determined is one point zero. That means
Speaker 1: that once you get that result where you've divided the
Speaker 1: quantity by the quantizer, you multiply by one. That means
Speaker 1: there's no change. You multiply by one you get the
Speaker 1: same number. More on that end a bit. Okay, So
Speaker 1: you take that seven thousand, eight hundred forty you divided
Speaker 1: by one hundred. That gives you seventy eight point four. Well,
Speaker 1: now you have to round that number, so you round
Speaker 1: it down to seventy eight. Now, when you have a
Speaker 1: decoder and you're ready to play back the information, it
Speaker 1: comes across this quantity the sight and it knows what
Speaker 1: the quantizer number was, so it multiplies by one hundred
Speaker 1: to get back to seven thousand, eight hundred. So the
Speaker 1: replicated number is actually forty off from the original number.
Speaker 1: The original number again was seven thousand, eight hundred forty.
Speaker 1: The replicated number is seven thousand, eight hundred. Now those
Speaker 1: inconsistencies manifest as noise in the actual playback. So if
Speaker 1: you wanted to increase the precision of any given scale
Speaker 1: factor band, you could do so by changing the scale
Speaker 1: factor number. So in that example, just now, I said
Speaker 1: the number was one point zero, meaning there's no change
Speaker 1: to that result. But I could have said it was ten,
Speaker 1: which means we would multiply the quanties number by ten.
Speaker 1: So we would take that seven thousand, eight hundred forty
Speaker 1: divided by one hundred, you get seventy eight point four,
Speaker 1: then multiplied by ten to get seven eight four. So
Speaker 1: when the decoder decompresses the file, it would reverse this
Speaker 1: this whole thing. It would just multiply by a hundred um.
Speaker 1: You would end up getting seven thousand, hundred forty again,
Speaker 1: which means that you wouldn't introduce any noise to the file.
Speaker 1: You would have a perfect representation. But in some cases
Speaker 1: the encoder may determine that any noise that you generate
Speaker 1: wouldn't be noticed or it wouldn't impact the quality of
Speaker 1: the audio enough for it to be a problem because
Speaker 1: of other factors for that particular scale factor band, like
Speaker 1: maybe it's really quiet, or maybe it's really complex. So
Speaker 1: in those cases, you could reduce the scale factor number
Speaker 1: by making it something else, like point one instead of
Speaker 1: one point oh. So that means you would multiply the
Speaker 1: quantized number by point one, So the seventy eight point
Speaker 1: four would become seven point eight four, and then you
Speaker 1: have to round it to get a whole integer, so
Speaker 1: you get eight seven point eight four rounds up to eight. Now,
Speaker 1: when a decode or decompresses the audio and multiplies eight
Speaker 1: by one hundred, that quantizer that we've talked about so much.
Speaker 1: Uh and uh. Actually at this point it would have
Speaker 1: to be eight thousand because it's also taking into account
Speaker 1: the scale factor, so it's multiplying it by a thousand,
Speaker 1: not just a hundred. So you would get a number
Speaker 1: that would pop up to eight thousand. And remember the
Speaker 1: original with seven thousand, eight hundred forty. So you look
Speaker 1: at the difference between these two, the original seven thousand forty,
Speaker 1: the new fact number is eight thousand. There's a pretty
Speaker 1: big difference there. That change might introduce enough noise for
Speaker 1: it to be a problem. So how does the encoder
Speaker 1: determine if a scale factor band is meeting the proper criteria?
Speaker 1: How can it tell if there is uh too much
Speaker 1: noise or if the noise falls below the threshold. Well,
Speaker 1: it goes through what it's called a Huffman coding process.
Speaker 1: At this point, Dylan is currently just staring at the
Speaker 1: wall and drool is coming out. Huffman coding process. It's
Speaker 1: converts scale factor bands into binary strings, and the process
Speaker 1: goes through a series of tables to determine if the
Speaker 1: data within the scale factor band requires more or less
Speaker 1: precision to describe the sound without affecting the audio quality.
Speaker 1: So Huffman coding is a process. And when you start
Speaker 1: with a large number of possibilities and you begin to
Speaker 1: narrow it down. Uh. Some people describe it as the
Speaker 1: coding equivalent of twenty questions. So you ask your first
Speaker 1: question like animal, vegetable, or mineral. You get an answer
Speaker 1: so animal. While that first answer eliminates a ton of
Speaker 1: other possibilities and narrows the focus, like anything that doesn't
Speaker 1: pertain to animal, you can automatically discount because you already
Speaker 1: know it can apply to that answer. With MP three compression,
Speaker 1: this means making certain the number of bits representing a
Speaker 1: granule because remember I mentioned that an MP three formats
Speaker 1: you have frames, and each frame, each frame has a thousand,
Speaker 1: one or fifty two samples and consists of two granules
Speaker 1: with five s each. So when you answer the first question,
Speaker 1: it eliminates a lot of other possibilities and narrows the focus.
Speaker 1: So like with animal, vegetable, mineral, if I say animal,
Speaker 1: you're gonna not ask any questions that have to do
Speaker 1: with minerals or vegetables only because it wouldn't make sense.
Speaker 1: You know, those aren't gonna apply. Same thing with m
Speaker 1: P three's, except this time it means making certain the
Speaker 1: number of bits representing a granule. Remember their two granules
Speaker 1: per frame with the MP three layer, Uh, you want
Speaker 1: to make sure that the number of bits representing that
Speaker 1: granule match the chosen bit rate for a compression. So
Speaker 1: if after going through this process, the encoder says, hey,
Speaker 1: this granule has more bits than what's allowed. It's too
Speaker 1: many bits. The we gotta get rid of some of these,
Speaker 1: the encoder can adjust the scale factor band so that
Speaker 1: there's less precision meaning that multiplier in other words, that
Speaker 1: but I talked about earlier, and thus reduce the amount
Speaker 1: of data needed to represent that particular granule. If a
Speaker 1: granule comes in under the bit rate, the encoder can
Speaker 1: increase the precision to reduce noise and fill that granule
Speaker 1: out properly so that matches the actual threshold. After all this,
Speaker 1: the pairs of granules become frames within the MP three files,
Speaker 1: and the only other component then MP three file apart
Speaker 1: from these frames is the I D three metadata. And
Speaker 1: this is pretty simple. This is like a header and
Speaker 1: it comes before all the frames in the audio file
Speaker 1: and contains information about about the file itself, which can
Speaker 1: include stuff like the title of a song, an artist name,
Speaker 1: an album title, other stuff like that. It can also
Speaker 1: include copyright information as well as information about the file itself,
Speaker 1: such as whether or not it's stereo recording or a
Speaker 1: mono recording. So when you use a decoder like an
Speaker 1: MP three player, it takes this compressed information, these these
Speaker 1: these representations that the music has been reduced to, and
Speaker 1: it converts that Huffman data back into the quantized format,
Speaker 1: scales the data back up to its original size or
Speaker 1: close approximation. Remember the the uncompressed version may actually be
Speaker 1: off by a significant amount depending upon each individual granule.
Speaker 1: And all of that data gets combined into a new
Speaker 1: PCM sample that can be played back to you. And
Speaker 1: that's all there is to it. Nothing could be easier,
Speaker 1: all right. That took a lot out of me, So
Speaker 1: I got really technical, and I apologize if I lost
Speaker 1: any of you out there, or for those of you
Speaker 1: who have a lot of experience working on compression algorithms,
Speaker 1: for oversimplifying in several cases. But now we've got a
Speaker 1: full episode about this, and I hope you have a
Speaker 1: better understanding of how a big sound file can be
Speaker 1: reduced to a smaller sound file. Next time, I'll just
Speaker 1: say magic. It will make everyone happier. If you guys
Speaker 1: have any questions for me, or comments or suggestions, anything
Speaker 1: like that, send me a message. My email is tech
Speaker 1: Stuff at how stuff works dot com, or you can
Speaker 1: drop me a line on Facebook or Twitter to handle it.
Speaker 1: Both of those is tech Stuff H. S W. And
Speaker 1: I'll talk to you guys again really soon. For more
Speaker 1: on this and thousands of other topics, is it how
Speaker 1: stuff works dot com, wh