Speaker 1: Get in tech with technology with tech Stuff from how
Speaker 1: stuff works dot com. Hey there, and welcome to tech Stuff.
Speaker 1: I'm your host, Jonathan Strickland. And in a recent episode
Speaker 1: I explored how digital audio works and gave kind of
Speaker 1: a brief history on the MP three file format. I
Speaker 1: warned you back then that that was part one of
Speaker 1: a three part series, and today we're gonna explore part two.
Speaker 1: So I hadn't forgotten about it. We're back to it, uh,
Speaker 1: And today we're gonna do a deeper dive with m
Speaker 1: P three's and how do they compress audio? And how
Speaker 1: can you take a file filled with information and make
Speaker 1: it a smaller size? What do you have to give
Speaker 1: up in order to make files smaller? And today we're
Speaker 1: gonna try and unravel the technical mystery behind the MP
Speaker 1: three And I am not going to lie to you people.
Speaker 1: This is gonna get a bit you know, man athy
Speaker 1: And that was an English major, So you mathematicians out there,
Speaker 1: get ready with your corrections because I'm probably gonna make
Speaker 1: some over generalizations for the purposes of my own sanity.
Speaker 1: There does get to a point where to really get
Speaker 1: into the technical details, it would likely be uh impossible
Speaker 1: for me to describe it in a way that would
Speaker 1: make sense and be accurate. Um, and I have given
Speaker 1: my producer Dylan the mandate that, should I get to
Speaker 1: cryptic and incomprehensible with my explanation, that he is to
Speaker 1: intervene in a way that he sees fit. Just not
Speaker 1: in the face, Dylan. It's not in the face. It's moneymaker, man.
Speaker 1: I gotta gotta take care of it. So let's remember
Speaker 1: that the heart of digital information is the bit that's
Speaker 1: either a zero or a one. The basic unit of
Speaker 1: information for digital formats zeros and ones. Now we can
Speaker 1: use those zeros and ones to describe all sorts of information,
Speaker 1: from text to audio, to video and really pretty much
Speaker 1: anything you can think of that's represented digitally. Ultimately, when
Speaker 1: you get down to it, it's a bunch of zeros
Speaker 1: and ones. So let's say you start off with your
Speaker 1: uncompressed audio file. You've got this enormous audio file in
Speaker 1: front of you. It's made up of zeros and ones.
Speaker 1: How do you make that file smaller? So in the
Speaker 1: real world, we can compress stuff, right, we can apply
Speaker 1: physical pressure to things. Think about packing a suitcase. You
Speaker 1: can make sure you get that extra outfit in if
Speaker 1: you just press it down hard enough and get that
Speaker 1: zipper zipped before it can burst open. But once you
Speaker 1: get to a certain level of compression, you cannot make
Speaker 1: things smaller, at least not without hurting yourself or whatever
Speaker 1: it is you're trying to compress. Digital files are a
Speaker 1: little different because you cannot physically cram the zeros and
Speaker 1: ones closer together. It doesn't work like that. These are
Speaker 1: abstract things. You can't make them smaller, right. You can't
Speaker 1: decrease the font. It doesn't work that way. The numbers
Speaker 1: represent two different states. So if you want to create
Speaker 1: a smaller audio file containing the recording that was in
Speaker 1: a larger audio file, you have to start getting creative now.
Speaker 1: In the last part of this series, I talked about
Speaker 1: how the MP three compression algorithm was born from an
Speaker 1: applied research institution in Germany and the team behind the
Speaker 1: MP three wanted to find a way to compress audio,
Speaker 1: specifically music for transmission over phone lines. Eventually, this evolved
Speaker 1: into the Motion Pictures Expert Group Audio Layer three compression methodology,
Speaker 1: better known as the MP three, and there's also IMPACT
Speaker 1: two and IMPEG four standards. Impact two, by the way,
Speaker 1: is the basis of compression on DVDs, although the actual
Speaker 1: DVD format is really a modification of Impact two and
Speaker 1: Impact four is a compression strategy for audio and video
Speaker 1: that's frequently used in lots of different up pacities, including
Speaker 1: streaming media services. So by the late nineteen seventies, researchers
Speaker 1: began to explore the possibility of leveraging psycho acoustics to
Speaker 1: figure out how to compress audio. And psychoacoustics refers to
Speaker 1: the way we perceive sound it's uh and also the
Speaker 1: physiological effects of sound on us. So this involves not
Speaker 1: just our our physical sense of hearing, but also our
Speaker 1: brains and the way our brains interpret sound. So, for example,
Speaker 1: there's a psychoacoustic phenomenon that's called the Hawse effect h
Speaker 1: A A S. And I think it's pretty interesting. So
Speaker 1: here's how the Hawse effect works. If you hear the
Speaker 1: exact same sound coming from different directions, but the two
Speaker 1: sounds arrive within thirty to forty milliseconds of each other,
Speaker 1: your brain will be convinced that you really only heard
Speaker 1: one sound and it came from the direction that hit
Speaker 1: you first. So let's say a sounds coming from directly
Speaker 1: in front of you and to your left, and you
Speaker 1: get both of them within that thirty to forty millisecond range,
Speaker 1: and you hear the one coming from ahead of you
Speaker 1: first to you, you're convinced that you only heard that
Speaker 1: sound once and it came from dead on straight ahead
Speaker 1: of you. Your brain kind of discounts the one that
Speaker 1: came off from the left, although it can reinforce it,
Speaker 1: which ends up being really useful if you're planning out
Speaker 1: p A systems for stage shows. I'm not joking. That
Speaker 1: really is the way that people plan those things out.
Speaker 1: It's pretty neat. Humans perceive sounds in a way that's
Speaker 1: not necessarily representational of all the sounds surrounding us. You
Speaker 1: can think of your brain as the filter between your
Speaker 1: understanding and what reality actually is. A lot of stuff
Speaker 1: goes on that it ends up getting rid of information
Speaker 1: that your brain just says, you know what, he or
Speaker 1: she doesn't need that, it's just gonna confuse things. We're
Speaker 1: gonna dump it. And that's kind of how it works.
Speaker 1: It's all on an unconscious level. It's not like you're
Speaker 1: actively working to do this. So let's say you're in
Speaker 1: a relatively busy hallway, and there could be a lot
Speaker 1: of sounds in that hallway, stuff that's going on constantly
Speaker 1: around you. Maybe they are doors opening and closing, Maybe
Speaker 1: their footsteps going up and down the hallway. Maybe someone
Speaker 1: shoes are squeaking against the linoleum floor. People are chattering
Speaker 1: away in there. But you are having a conversation with someone,
Speaker 1: so you turn your focus on that person and other
Speaker 1: sounds seemingly fade away. They're still present, but they're not important.
Speaker 1: So in this example, you would actually call those other
Speaker 1: sounds of distraction and you would really focus on the conversation. Uh.
Speaker 1: That also shows how we're able to consciously direct our
Speaker 1: sense our perception of hearing. So both of these factors
Speaker 1: come into play. Now. One thing that MP three encoding
Speaker 1: takes advantage of is something called masking, and there are
Speaker 1: a couple of different variations of the masking effect. One
Speaker 1: of them is called frequency masking. So let's say you've
Speaker 1: got to sound frequencies that are similar ahaps, there're just
Speaker 1: a few hurts apart. Remember, frequencies are measured in hurts,
Speaker 1: which is really the number of oscillations per second. So
Speaker 1: let's say you've got a sound that's at I don't know, uh,
Speaker 1: one thousand killer hurts, and another one that's at one
Speaker 1: thousand and ten killer hurts. Now, the human ear is
Speaker 1: precise enough to be able to tell the difference of
Speaker 1: two sounds that are at least two hurts apart from
Speaker 1: each other. That's how precise our resolution of hearing it's
Speaker 1: it's at that level. But if you get two sounds
Speaker 1: played at the same time and they are that close
Speaker 1: together in frequency, and one of those frequencies is played
Speaker 1: at a greater volume than the other, our brains will
Speaker 1: pick up on the louder sound and ignore the quieter sound,
Speaker 1: even though both of them are present. What becomes important
Speaker 1: at that point is the amplitude. Now, the further apart
Speaker 1: in frequencies you get, the less that hasn't a effect.
Speaker 1: So if you get far enough apart where they are
Speaker 1: two pitches, one of them noticeably louder than the other,
Speaker 1: but they're far enough apart, you will hear both of them.
Speaker 1: It only works if the two pitches are relatively close together,
Speaker 1: and there's not a universal formula for frequency masking. As
Speaker 1: you get closer to the boundaries of human hearing, frequency
Speaker 1: masking becomes easier. So if it's a really low pitch
Speaker 1: or a really high pitch, it's easier to get away
Speaker 1: with it. Once you start getting into what is the
Speaker 1: ought of as the sweet spot for human hearing, which
Speaker 1: is generally considered to be between two and five killer hurts,
Speaker 1: you need a greater difference in volume or a smaller
Speaker 1: difference in frequency in order for masking to work. Frequency
Speaker 1: masking at any rate. But then there's also temporal masking,
Speaker 1: and you might say, okay, I got it. Temporal that
Speaker 1: means time. Indeed it does, my friend. This describes the
Speaker 1: effect of a short but loud sound masking a softer
Speaker 1: sound for a short time. Weird thing is the loud
Speaker 1: sound can actually mask sounds that precede it slightly, not
Speaker 1: by a whole lot, but a little bit. MP three
Speaker 1: compression takes advantage of both frequency and temporal masking when
Speaker 1: it's trying to determine which data needs to be included
Speaker 1: and which data can be dumped, because it won't affect
Speaker 1: your perception of whatever the the audio file is in
Speaker 1: the first place. So you also probably remember I talked
Speaker 1: about the physical limitation to what we humans can hear,
Speaker 1: no matter what our brains might be up to, so
Speaker 1: that this doesn't have to do with our brains, you know,
Speaker 1: filtering through the information that's coming in. This has to
Speaker 1: do with the physical limitations of the human ear. In
Speaker 1: the last episode of the series, I said typical human hearing.
Speaker 1: Keep in mind typical there are exceptions. UH covers the
Speaker 1: range of frequencies between about twenty hurts and twenty killer
Speaker 1: hurts or twenty thousand hurts. So twenty to twenty thousand
Speaker 1: higher frequencies represent higher pitches and sound lower frequencies lower pitches, right,
Speaker 1: And as you get older, your ability to perceive those
Speaker 1: higher frequencies starts to diminish. So most adults actually have
Speaker 1: an upper range closer to sixteen killer hurts, not twenty. UH.
Speaker 1: Kids they can hear those higher pitches. You may have
Speaker 1: heard the story about how some convenience stores experimented with
Speaker 1: getting rid of teenage loiterers by by UH projecting out
Speaker 1: the super high pitches that that adults could not hear
Speaker 1: but kids could, and it discouraged kids from hanging out
Speaker 1: at the convenience store and loitering. UM. I love that
Speaker 1: idea so much. Anyway, that's because I'm old and my
Speaker 1: hearing is terrible. Well, remember I also mentioned you can
Speaker 1: detect changes in pitch at two hurts increments if you
Speaker 1: get below two hurts and change, Like, if it's just
Speaker 1: a one hurts difference between two frequencies, it's too low
Speaker 1: a resolution for us to detect. To us, it will
Speaker 1: sound exactly the same. So if you were to hear
Speaker 1: a frequency at one thousand one hurts or one point
Speaker 1: zero zero one killer hurts and one point zero zero
Speaker 1: to killer hurts, you wouldn't notice the difference. They would
Speaker 1: sound exactly the same to you. So if you're gonna
Speaker 1: take audio and compress it, one step you could consider
Speaker 1: is eliminating anything that's outside the actual range of frequencies
Speaker 1: that we can hear, or simplifying any changes in frequency
Speaker 1: that are smaller than two hurts. If you get take
Speaker 1: all that data and you say it is physically impossible
Speaker 1: for a human to perceive this, get rid of that information,
Speaker 1: then in theory it wouldn't have any effect on the
Speaker 1: rest of the recording. But how you go further than that? Right,
Speaker 1: how do you create a method so that you can
Speaker 1: really compress this file? You want a method that will
Speaker 1: preserve the important sounds while potentially ignoring all the unimportant
Speaker 1: or incidel sounds. And you want to be automatic because
Speaker 1: if you have a man you really then that's going
Speaker 1: to take countless hours just to edit a single sound file.
Speaker 1: So that was the challenge that the MP three research
Speaker 1: team faced as a group. Now, their solution, which ultimately
Speaker 1: created even more challenges, was to come up with what
Speaker 1: was essentially a simulated human ear and brain. They needed
Speaker 1: to replicate the experience of perceiving music so that an
Speaker 1: algorithm could evaluate every sound in an audio file and
Speaker 1: judge if an in fact was relevant enough to include
Speaker 1: in the final compressed version. If a sound were imperceptible,
Speaker 1: then it wouldn't make sense to include it in the
Speaker 1: MP three file. So by leaving out all the irrelevant data,
Speaker 1: they can make the audio information take up less bandwidth.
Speaker 1: The file itself would be smaller because you just dumped
Speaker 1: everything that wasn't important. So the team used an algorithm
Speaker 1: called the low complexity adaptive transform coding or lc DASH
Speaker 1: a t C as the foundation for their research. This
Speaker 1: was kind of their starting point, and this is an
Speaker 1: approach that tries to do away with redundancy as much
Speaker 1: as possible. And it also incorporates adaptation to perceptual requirements. Also,
Speaker 1: MP three's oh a lot to the IMPEG Layer two standard,
Speaker 1: So the layer two obviously came out before Layer three,
Speaker 1: and so a lot of the features of layer three
Speaker 1: are really um their legacy features from layer two. Uh.
Speaker 1: In other words, MP three group kind of got stuck
Speaker 1: with them because otherwise they would have had a problem
Speaker 1: with backwards compatibility. So the result is kind of a
Speaker 1: clunky arrangement under the hood, and some of the features
Speaker 1: may make very little sense when I go through them,
Speaker 1: but some of that is because it's a hold over
Speaker 1: from an earlier compression strategy, which isn't terribly satisfying as
Speaker 1: an answer. But the reason many parts of the MP
Speaker 1: three compression algorithm are the way they are is because
Speaker 1: that's the way we've always done it. So next I'm
Speaker 1: gonna dive into the phases of compression. But before I
Speaker 1: do that, let's all take a deep breath and take
Speaker 1: a moment to thank our sponsor, and we're back. So
Speaker 1: there are two big phases we'll need to talk about
Speaker 1: with MP three compression. The first phase is analysis and
Speaker 1: the second phase is the actual compression itself. And after
Speaker 1: that there's the process of decoding and MP three for playback.
Speaker 1: But that's way simpler once we get an understanding of
Speaker 1: how the encoding process actually happens. So let's begin with analysis. Now.
Speaker 1: This is the part where the standard has to figure
Speaker 1: out which frequencies within an audio range are recording rather
Speaker 1: are important or perceptible. So how does a program and
Speaker 1: in coder figure out what we can hear and what
Speaker 1: we cannot hear? All? Right, time to get technical. So
Speaker 1: you start off with your pulse code modulation audio file
Speaker 1: or PCM file. And you might remember I talked about
Speaker 1: PCM audio in the first episode of this series, but
Speaker 1: just in case you don't, it's a lossless digital audio file.
Speaker 1: The actual format could be a wave or ai f
Speaker 1: F or something along those lines, but the important thing
Speaker 1: to keep in mind is that it is uncompressed. Now,
Speaker 1: that means those files tend to be pretty big. This
Speaker 1: is our raw material that we want to take and
Speaker 1: squish down to a more manageable, transferable size. And in
Speaker 1: our our last episode in this series, I also mentioned
Speaker 1: that the standard for c D audio is a sample
Speaker 1: rate of forty four point one. Killer hurts and we
Speaker 1: learned that you need a sample rate twice the frequency
Speaker 1: of the highest frequency in your recording, and since human
Speaker 1: hearing tops out at around twenty kill hurts, the standard
Speaker 1: for CDs is forty four point one killer hurts. The
Speaker 1: MP three standard can support lots of different sample rates,
Speaker 1: but forty four point one killer Hurts is pretty much
Speaker 1: the common standard. So you've got a number of samples
Speaker 1: with your audio file, and that number will depend upon
Speaker 1: how long the audio file is. You've got forty four
Speaker 1: thousand one samples per second, actually twice that for stereo,
Speaker 1: but for the purposes of this discussion, let's kind of
Speaker 1: stick with mono sounds so that I don't start having
Speaker 1: math coming out of my ears. And we're still in
Speaker 1: the very easy, simple part as far as math goes.
Speaker 1: We haven't gotten to the complicated stuff yet, all right,
Speaker 1: So you've got forty four thousand, one hundred samples per second.
Speaker 1: To compress it into an MP three format, the algorithm
Speaker 1: first groups all of these samples into collections called frames.
Speaker 1: So take those forty four thousand one per second, and
Speaker 1: then you start saying, okay, we're gonna group you in batches.
Speaker 1: Each batch is called a frame and each frame contains
Speaker 1: one thousand, one fifty two samples. Now that's specifically to
Speaker 1: maintain backwards compatibility to IMPEG Layer two, which established that
Speaker 1: one thousand, one or fifty two number. But we're not
Speaker 1: talking about IMPEG layer two. We're talking about IMPEG Layer three,
Speaker 1: and though that means we have to get a little
Speaker 1: more complicated. So each frame consists of two subgroups called granules.
Speaker 1: So each granule has five undred seventy six samples seventy
Speaker 1: six times two one thousand fifty two, so five seventy
Speaker 1: six samples per granule. Now, technically MP three encoders only
Speaker 1: work on one granule at a time, but they may
Speaker 1: reference the granules immediately before and immediately after the current
Speaker 1: one in order to see how the audio within the
Speaker 1: file changes over time. All right, so now you've got
Speaker 1: your granules of five hundred seventy six samples each. Then
Speaker 1: the MP three encoder runs the samples through a filter bank,
Speaker 1: which sorts the sound into thirty two frequency ranges. Are
Speaker 1: you are you crazy about the numbers yet, Dylan? Are you?
Speaker 1: Dylan's Dylan's nodding. Dylan gets worse from here. So you
Speaker 1: have thirty two frequency ranges, which is another nod to
Speaker 1: the layer two method which use those thirty two ranges
Speaker 1: for encoding purposes. But we're not talking about layer two early, No,
Speaker 1: we're talking MP three. Gosh darn it. That means we
Speaker 1: take those thirty two ranges and we subdivide them by
Speaker 1: a factor of eighteen. That means we have five hundred
Speaker 1: seventies six bands of frequencies, each band containing one six
Speaker 1: of the frequency range of the original sample. So what
Speaker 1: that actually means, and this this is actually pretty easy.
Speaker 1: The bands are not limited to a specific number for
Speaker 1: their frequency range. Right. The bands don't mean that on
Speaker 1: the on band number one it goes from twenty hurts
Speaker 1: up to a certain range and on band five D
Speaker 1: seventy six in that twenty killer hurts. That's not what
Speaker 1: it means. They're dependent upon the original audio. So if
Speaker 1: the original audio contains sounds within a narrow range of frequencies,
Speaker 1: the five bands will be more precise. But if the
Speaker 1: original recording has a vast range of frequencies, the bands
Speaker 1: are less precise. So another way to think about this
Speaker 1: is with a pizza. So let's say you get extra
Speaker 1: large pizza and you cut it into eight equal slices.
Speaker 1: And then you get a small pizza and you cut
Speaker 1: that into eight equal slices. Well, in both cases you
Speaker 1: have with each slice one eighth of a pizza. But
Speaker 1: the extra large pizza pizza slice is bigger than the
Speaker 1: small pizza pizza slice. It all depends on the size
Speaker 1: of the pizza. So in this case, it depends upon
Speaker 1: the range of frequencies. And and Dylan, do you think
Speaker 1: we could go for some pizza, you know, just just
Speaker 1: put the episode on hole and go get pizza. Dylan's nodding.
Speaker 1: It's great for audio. Yeah, so, uh, pizza, We'll be
Speaker 1: right back. Okay, that was good pizza. Now um oh man,
Speaker 1: I got a whole bunch more notes. Okay, well, let's
Speaker 1: let's go ahead and and do the rest of this.
Speaker 1: All right, So you've got your sound divided up into
Speaker 1: those five seventy six sub brands of frequencies, you know,
Speaker 1: the thing I compared to pizza slices earlier. Now you
Speaker 1: get two different mathematical processes applied to this data. One
Speaker 1: is the fast Furrier transform or f f T, and
Speaker 1: the other is the modified discrete cosine transform or m
Speaker 1: d c T. Now I am not going to dive
Speaker 1: deeply into how these transforms work because frankly, they are
Speaker 1: beyond my mathematical understanding. But I know what they do.
Speaker 1: I just cannot explain the process like how they do
Speaker 1: what they do. So I'm going to give you the
Speaker 1: explanation of what they do what the outcome of each
Speaker 1: of these transformed processes happens to be. But I'm not
Speaker 1: going to be able to tell you the actual mathematical
Speaker 1: steps involved in each because I don't math. So good guys,
Speaker 1: But let's start with a fast for your transform. So
Speaker 1: transform is kind of what it sounds like. It's all
Speaker 1: about transforming information in some way. So in this particular case,
Speaker 1: the f f T transforms the frequency bands we just
Speaker 1: talked about into data that can be further analyzed by
Speaker 1: a psychoacoustic model that's in the encoder. So this is
Speaker 1: that simulated human ear and brain we were talking about earlier.
Speaker 1: So what the encoder does is it analyzes each bed
Speaker 1: of data and looks for signs that it represents audio
Speaker 1: that wouldn't be perceived by a human. So it's looks
Speaker 1: looking for any potential for masking possibilities. So are there
Speaker 1: collections of frequencies that are grouped close together, and is
Speaker 1: one of those frequencies louder than the others, you might
Speaker 1: be able to do away with those softer frequencies because
Speaker 1: of frequency masking. The encoder will also look at whether
Speaker 1: or not the audio has a lot of complexity to it,
Speaker 1: if it has a lot of changes, or if it's
Speaker 1: just relatively steady or simple audio. Any transient sounds that
Speaker 1: are present in the audio might end up being temporal masking,
Speaker 1: so it'll analyze those as well and see if that's
Speaker 1: a possibility. So really what they're looking is for, you know,
Speaker 1: just any really loud sounds that stand out above the
Speaker 1: rest of the recording. That's what the f f T
Speaker 1: is doing. So what about the modified discrete cosign transform. Well,
Speaker 1: this is happening in parallel with the f f T
Speaker 1: and the samples get sorted into different patterns called windows
Speaker 1: uh and the criterion for sorting all has to do
Speaker 1: with whether the sample represents a steady sound or varied sound.
Speaker 1: So if you have a simple steady sound that goes
Speaker 1: into a long window, if there's a lot of variation
Speaker 1: in the sound, like there are a lot of consonants
Speaker 1: in a vocal line or it's like a drum solo
Speaker 1: or something like that. It would get sorted into it
Speaker 1: series ease of three short windows, and each short window
Speaker 1: contains one two samples. That amounts to four whole milliseconds,
Speaker 1: so four thousands of a second in three patterned windows.
Speaker 1: So you've got these windows now, either long windows for
Speaker 1: simple sounds or short windows for the more complex sounds.
Speaker 1: And then the modified discrete cosine transform kicks into gear.
Speaker 1: It looks at each long window or set of three
Speaker 1: short windows and converts them into a set of spectral values.
Speaker 1: To some of you, that probably sounds meaningless. So let's
Speaker 1: talk about spectral analysis for a second. First, I was
Speaker 1: very disappointed to learn that spectral analysis doesn't involve a
Speaker 1: psychologist talking to a ghost about its emotional state, so bummer.
Speaker 1: But spectral analysis is when you look at a spectrum
Speaker 1: of information, like a spectrum of frequencies or related information
Speaker 1: like energy states. That's what this transform does. It takes
Speaker 1: data that originally represents a slice of time in a
Speaker 1: sound waveform. That's what sample is. A sample is an
Speaker 1: instance of time in a wave form and converts it
Speaker 1: into information representing sound as energy across a range of frequencies. Now,
Speaker 1: you can plot out spectral information in a lot of
Speaker 1: different ways, but one common method is to use brightness
Speaker 1: to indicate energy levels. Higher energy levels are brighter patches
Speaker 1: in your visual representation of spectral data. High frequencies would
Speaker 1: appear at the top of a spectral view, like imagine
Speaker 1: a box, and at the top of the box that's
Speaker 1: where you would find high frequencies, at the bottom of
Speaker 1: the box that's where you find low frequencies, and it's
Speaker 1: just lots of patches of color. The really bright patches
Speaker 1: of color represent very high energy frequencies, so they could
Speaker 1: be high or low in in actual frequency, but we're
Speaker 1: talking about energy levels, not whether it's a higher low pitch.
Speaker 1: Looking left to write represents the passing of time, and
Speaker 1: looking along any vertical points shows you the actual frequency
Speaker 1: or pitch, and then the respective energy level is the brightness.
Speaker 1: So it's kind of like looking at sound as a wave,
Speaker 1: but instead of being a wave, you're looking at information
Speaker 1: that indicates frequency range and energy level. That representation is
Speaker 1: actually kind of analogous to how we hear audio. So
Speaker 1: an encoder can analyze the spectral view and start to
Speaker 1: filter out the data we wouldn't perceive due to psychoacoustics. Now,
Speaker 1: after all that processing, the encoder looks at the frequency
Speaker 1: sub brands and the levels of spectral intensity for each
Speaker 1: and that information can then be used for the next phase,
Speaker 1: which is compression. But right now I think we could
Speaker 1: all stand a little decompression, So let's take another quick
Speaker 1: break to thank our sponsor. All right, so now you're
Speaker 1: ready to compress your analyzed audio. Good for you, and
Speaker 1: by you I mean encoders. This has to be simpler
Speaker 1: than that analysis segment, right, I mean that got a
Speaker 1: little crazy with all the different bands and sub bands
Speaker 1: and windows and frames and granules. Sadly it gets more complicated,
Speaker 1: all right. So there are two layers of compression going
Speaker 1: on with MPEG Layer three. One of those layers depends
Speaker 1: upon the psychoacoustic analysis and the other doesn't. So why
Speaker 1: would you use two layers with different strategies like that? Well,
Speaker 1: the reason is that one strategy is great for complex
Speaker 1: audio with lots of components, but not so great with
Speaker 1: simpler sounds, and the other strategy is kind of the opposite.
Speaker 1: So the psychoacoustic approach is the one that's really good
Speaker 1: for complicated sounds. If if you've got a lot of
Speaker 1: volume changes, lots of different frequencies, it's just complicated and
Speaker 1: rich sound, you've got a lot of opportunity to look
Speaker 1: for masking and other acoustic elements that limit the actual
Speaker 1: sounds that people perceive. So it means there are a
Speaker 1: lot of chances for you to uh fudge by dropping
Speaker 1: all the stuff that people probably wouldn't notice anyway. And uh,
Speaker 1: if you take a piece that's got a lot of
Speaker 1: elements at varying volumes, there are likely several opportunities to
Speaker 1: to do this. But if you're talking about relatively straightforward
Speaker 1: audio with few components, few changes in volume, there's really
Speaker 1: not a whole lot of data you can ditch without
Speaker 1: it actually affecting the quality of the audio in a
Speaker 1: perceptible way. And this is part of what Brandenburg, that
Speaker 1: guy I was talking about in our first episode in
Speaker 1: this series. Uh, that's what he discovered when he was
Speaker 1: working with the MP three standard and he was listening
Speaker 1: back to that Suzanne Vega acapella track Tom's Diner. He
Speaker 1: was listening to a compressed version of it, and he
Speaker 1: said it was terrible. He said it ruined the quality
Speaker 1: of the audio. And part of that is because that
Speaker 1: particular song is fairly simple. There's just not a lot
Speaker 1: of opportunity to take advantage of masking and other tricks
Speaker 1: without potentially compromising the quality. So they decided to also
Speaker 1: incorporate some traditional compression strategies, which which work better with
Speaker 1: those types of recordings. So the MP three format takes
Speaker 1: advantage of both the traditional approach and the psychoacoustic approach,
Speaker 1: and that allows the encoder to compressed files into smaller
Speaker 1: size without just following a single strategy, like it doesn't
Speaker 1: have to do a one size fits all for all
Speaker 1: elements of audio. Now, combining those two strategies requires a
Speaker 1: little more mathematical gymnastics. So let's go back to those
Speaker 1: five seventy six frequency bins. You know, those sub bands
Speaker 1: we talked about earlier. You've got to quantize those suckers.
Speaker 1: What does that mean. It means assigning a quantity to
Speaker 1: each to each frequency bin, you have to give it
Speaker 1: a quantity of some sorts so that you can end
Speaker 1: up judging how much you can get away with dropping data.
Speaker 1: So to do this, the encoder sorts those five six
Speaker 1: bins into twenty two scale factor bands. How you doing
Speaker 1: over there? Dylan just checking in on you? Okay, Dylan's
Speaker 1: got Dylan's got a thousand yards stare going. I hope
Speaker 1: you guys are doing okay over there? All right, So
Speaker 1: before smoke starts coming out of your ears, let me
Speaker 1: explain what the scale factor bands are all about. The
Speaker 1: whole purpose of the scale factor bands is to determine
Speaker 1: how the information will be stored within the compressed state.
Speaker 1: So you want to get away with as little data
Speaker 1: as possible before affecting sound quality. So if you can
Speaker 1: say the same thing in a shorter space without affecting
Speaker 1: the quality of what it is you're saying, you go
Speaker 1: with it. Brevity is the soul of compression. So if
Speaker 1: we were talking about language, I would say it's more
Speaker 1: efficient to say it's raining outside, or even just it's raining,
Speaker 1: because you would assume that it would be outside where
Speaker 1: the rain is happening, and it would be inefficient for
Speaker 1: me to say it's coming down like cats and dogs
Speaker 1: out there. It's not as efficient as saying it's raining.
Speaker 1: So if you can get away with shorter statements without
Speaker 1: affecting the actual quality, and you could argue that by
Speaker 1: switching from it's coming down like cats and dogs out
Speaker 1: there and it's raining changes the quality, And that could
Speaker 1: be a valid argument. But if you can get away
Speaker 1: with shorter without affecting quality, you do it. So each
Speaker 1: scale factor band is represented by a quantity, Then the
Speaker 1: encoder divides that quantity by a given number called the quantizer,
Speaker 1: which is the same across the entire frequency spectrum for
Speaker 1: that recording. The resulting number is then rounded up or
Speaker 1: down to a whole digit. And here's an important point.
Speaker 1: Individual scale factor bands can be scaled up or down
Speaker 1: for more or less precision to represent the actual value
Speaker 1: of those bands. So what the heck does all that mean? Well,
Speaker 1: the purpose of dividing and rounding is just to simplify
Speaker 1: the data to reduce the amount you need in order
Speaker 1: to store the information. So let's go with a totally
Speaker 1: hypothetical example. Let's say you've got a scale factor band
Speaker 1: and you've decided you're representing that scale factor band with
Speaker 1: the quantity seven eight four zero seven thousand, eight hundred forty,
Speaker 1: and you've chosen the number one hundred to quantize your data,
Speaker 1: meaning that you will divide each uh scale factor bands
Speaker 1: quantity by one hundred. So this is seven thousand, eight
Speaker 1: hundred forty. You divide it by one hundred. Uh and
Speaker 1: the scale factor for this particular band you have determined
Speaker 1: is one point zero. That means that once you get
Speaker 1: that result where you've divided the quantity by the quantizer,
Speaker 1: you multiply by one. That means there's no change. Multiply
Speaker 1: by one you get the same number. More on that
Speaker 1: end a bit. Okay, So you take that seven thousand,
Speaker 1: eight hundred forty you divided by one hundred. That gives
Speaker 1: you seventy eight point four. Well, now you have to
Speaker 1: round that number, so you round it down to seventy eight. Now,
Speaker 1: when you have a decoder and you're ready to play
Speaker 1: back the information, it comes across this quantity the seventy eight,
Speaker 1: and it knows what the quantizer number was, so it
Speaker 1: multiplies by one hundred to get back to seven thousand,
Speaker 1: eight hundred. So the replicated number is actually forty off
Speaker 1: from the original number. The original number again with seven thousand,
Speaker 1: eight hundred forty, the replicated number is seven thousand, eight hundred. Now,
Speaker 1: those inconsistencies manifest as noise in the actual playback. So
Speaker 1: if you wanted to increase the precision of any given
Speaker 1: scale factor band, you could do so by changing the
Speaker 1: scale factor number. So in that example, just now, I
Speaker 1: said the number was one point zero, meaning there's no
Speaker 1: change to that result. But I could have said it
Speaker 1: was ten, which means we would multiply the quantized number
Speaker 1: by ten. So we would take that seven thousand, eight
Speaker 1: hundred forty divided by one hundred you get seventy eight
Speaker 1: point four, then multiplied by ten to get seven four.
Speaker 1: So when the decoder decompresses the file, it would reverse
Speaker 1: this this whole thing. It would just multiply by a
Speaker 1: hundred um. You would end up getting seven thousand, hundred
Speaker 1: forty again, which means that you wouldn't introduce any noise
Speaker 1: to the file. You would have a perfect representation. But
Speaker 1: in some cases, the encoder may determine that any noise
Speaker 1: that you generate wouldn't be noticed or it wouldn't impact
Speaker 1: the quality of the audio enough for it to be
Speaker 1: a problem because of other factors for that particular scale
Speaker 1: factor band, like maybe it's really quiet, or maybe it's
Speaker 1: really complex. So in those cases, you could reduce the
Speaker 1: scale factor number by making it something else like point
Speaker 1: one instead of one point oh. So that means you
Speaker 1: would multiply the quantized number by point one, So the
Speaker 1: seventy eight point four would become seven point eight four,
Speaker 1: and then you have to round it to get a
Speaker 1: whole integer, so you get eight seven point eight four
Speaker 1: rounds up to eight. Now, when a decode or decompresses
Speaker 1: the audio, it multiplies eight by one hundred. That quantizer
Speaker 1: that we've talked about so much, uh and uh, actually
Speaker 1: at this point would have to be eight thousand because
Speaker 1: it's also taking into account the scale factor, so it's
Speaker 1: multiplying it by a thousand, not just a hundred. So
Speaker 1: you would get a number that would pop up to
Speaker 1: eight thousand. And remember the original with seven thousand, eight
Speaker 1: hundred forty. So you look at the difference between these two,
Speaker 1: the original seven thousand forty, the new fact number is
Speaker 1: eight thousand. There's a pretty big difference there. That change
Speaker 1: might introduce enough noise for it to be a problem.
Speaker 1: So how does the encoder determine if a scale factor
Speaker 1: band is meeting the proper criteria? How can it tell
Speaker 1: if there is ah too much noise or if the
Speaker 1: noise falls below the threshold? Well, it goes through what
Speaker 1: it's called a Huffman coding process. At this point, Dylan
Speaker 1: is currently just staring at the wall and drool is
Speaker 1: coming out. Huffman coding process. It's converts scale factor bands
Speaker 1: into binary strings, and the process goes through a series
Speaker 1: of tables to determine if the data within the scale
Speaker 1: factor band requires more or less precision to describe the
Speaker 1: sound without affecting the audio quality. So, Huffman coding is
Speaker 1: a process. And when you start with a large number
Speaker 1: of possibilities and you begin to narrow it down, uh.
Speaker 1: Some people describe it as the coding equivalent of twenty questions.
Speaker 1: So you ask your first question like animal, vegetable or mineral.
Speaker 1: You get an answer so animal. While that first answer
Speaker 1: eliminates a ton of other possibilities and narrows the focus
Speaker 1: like anything that doesn't pertain to animal, you can automatically
Speaker 1: discount because you already know it can apply to that answer.
Speaker 1: With MP three compression, this means making certain the number
Speaker 1: of bits representing a granule because remember I mentioned that
Speaker 1: in MP three formats you have frames, and each frame.
Speaker 1: Each frame has a thousand, one or fifty two samples
Speaker 1: and consists of two granules with five s each. So
Speaker 1: when you answer the first question, it eliminates a lot
Speaker 1: of other possibilities and narrows the focus. So like with animal, vegetable, mineral,
Speaker 1: if I say animal, you're gonna not ask any questions
Speaker 1: that have to do with minerals or vegetables only because
Speaker 1: it wouldn't make sense. You know, those aren't gonna apply.
Speaker 1: Same thing with m P three's except this time it
Speaker 1: means making certain the number of bits representing a granule.
Speaker 1: Remember their two granules per frame with the MP three layer, Uh,
Speaker 1: you want to make sure that the number of bits
Speaker 1: representing that granule match the chosen bit rate for a compression.
Speaker 1: So if after going through this process, the encoder says, hey,
Speaker 1: this granule has more bits than what's allowed. It's too
Speaker 1: many bits. The we gotta get rid of some of these,
Speaker 1: the encoder can adjust the scale factor band so that
Speaker 1: there's less precision meaning that multiplier in other words, that
Speaker 1: but I talked about earlier, and thus reduce the amount
Speaker 1: of data needed to represent that particular granule. If a
Speaker 1: granule comes in under the bit rate, the encoder can
Speaker 1: increase the precision to reduce noise and fill that granule
Speaker 1: out properly so it matches the actual threshold. After all this,
Speaker 1: the pairs of granules become frames within the MP three files.
Speaker 1: And the only other component in an MP three file
Speaker 1: apart from these frames is the I D three metadata.
Speaker 1: This is pretty simple. This is like a header, and
Speaker 1: it comes before all the frames in the audio file
Speaker 1: and contains information about about the file itself, which can
Speaker 1: include stuff like the title of a song, an artist name,
Speaker 1: an album title, other stuff like that. It can also
Speaker 1: include copyright information as well as information about the file itself,
Speaker 1: such as whether or not it's a stereo recording or
Speaker 1: a mono recording. So when you use a decoder like
Speaker 1: an MP three player, it takes this compressed information. These
Speaker 1: these these representations that the music has been reduced to,
Speaker 1: and it converts that Huffman data back into the quantized format,
Speaker 1: scales the data back up to its original size or
Speaker 1: close approximation. Remember the the uncompressed version may actually be
Speaker 1: off by a significant amount depending upon each individual granule.
Speaker 1: And all of that data gets recombined into a new
Speaker 1: pc M sample that can be played back to you.
Speaker 1: And that's all there is to it. Nothing could be easier.
Speaker 1: All right, that took a lot out of me, so
Speaker 1: I got really technical, and I apologize if I lost
Speaker 1: any of you out there, or for those of you
Speaker 1: who have a lot of experience working on compression algorithms,
Speaker 1: for oversimplifying in several cases. But now we've got a
Speaker 1: full episode about this, and I hope you have a
Speaker 1: better understanding of how a big sound file can be
Speaker 1: reduced to a smaller sound file. Next time, I'll just
Speaker 1: say magic. It will make everyone happier. But I hope
Speaker 1: you guys appreciated this. In the next episode in this
Speaker 1: series it will be far less technical. I'm going to
Speaker 1: be more historical. I'm going to talk about the progression
Speaker 1: of the MP three player, how it came, about, how
Speaker 1: it evolved, and how the iPod ended up becoming the
Speaker 1: dominant brand in a c of MP three players, and
Speaker 1: then maybe kind of explore where MP three players are today,
Speaker 1: like how many are there, how how big is the market?
Speaker 1: Are are people still buying them? That kind of question.
Speaker 1: If you guys have any questions for me, or comments
Speaker 1: or suggestions anything like that, send me a message. My
Speaker 1: email is tech Stuff at how stuff works dot com,
Speaker 1: or you can drop me a line on Facebook or Twitter,
Speaker 1: the handle of both of those those tech stuff h
Speaker 1: s W and I'll talk to you guys again really
Speaker 1: soon for more on this and sense of other topics.
Speaker 1: Is it how stuff works? Dot com m