How Well Can You Hear Audio Quality? : The Record : NPR
My results with headphones in the computer: I could not hear a single difference with the Suzanne first track, so I switched on the independent dac and did the test seriously.
I clicked 4/6 of the uncompressed samples clicked the 128kbp for Jzze and Suzanne, this is not my type of music so I was relieved ,
btw, with a good headphone setup I found the differences were not obvious at all, I was stressed when I clicked the button lol, especially after choosing 128kbp with Suzzane, I was like, wow, this is not going well...
My results with headphones in the computer: I could not hear a single difference with the Suzanne first track, so I switched on the independent dac and did the test seriously.
I clicked 4/6 of the uncompressed samples clicked the 128kbp for Jzze and Suzanne, this is not my type of music so I was relieved ,
btw, with a good headphone setup I found the differences were not obvious at all, I was stressed when I clicked the button lol, especially after choosing 128kbp with Suzzane, I was like, wow, this is not going well...
I gave a second try for Zazee and Suzanne and got them right! 6/6 but can I repeat the test and get them all rigth? probably!
For Zazee the bass is just better with 256k and uncompressed, the reason I could pick the uncompressed over the 256k was the cow bells sounding like junk in the 256k, they were clear with the uncompressed, so I choose it.
For Suzanne I wasn't clear, the 256k and uncompressed sounded more natural, but nothing was missing in the mp3, So I was just lucky?
For Zazee the bass is just better with 256k and uncompressed, the reason I could pick the uncompressed over the 256k was the cow bells sounding like junk in the 256k, they were clear with the uncompressed, so I choose it.
For Suzanne I wasn't clear, the 256k and uncompressed sounded more natural, but nothing was missing in the mp3, So I was just lucky?
Last edited:
If I did the test in an A/B situation I could never tell between the uncompressed and 256k with headphones. I would probably go 50/50 , I had to focus...
I can do the test this morning, I don't remember the answers, on my DIY speakers with DIY amp, DIY dac, I like how they sound, but can they discern as well as the headphones? We will see.
I can do the test this morning, I don't remember the answers, on my DIY speakers with DIY amp, DIY dac, I like how they sound, but can they discern as well as the headphones? We will see.
I got 3/6 but the ones I got correct had real instruments. The three I got wrong were synthesized music which is very difficult to discern. For me it is easy with real instruments as I listen for the realistic sound of the cymbals. The one that baffled me was the cappella Suzanne Vega where I was torn between 2. The one I chose was 128k! the one I was on the fence with was the uncompressed. Ironically I found the 320k compressed the sharp sibilance of the moisture in her mouth moreso than the 128k.
Completely disagree. If it was the same source all 6 times then I would agree.
Awhile back, when I was first getting into streaming audio around my home from a central "audio computer", I did some experiments on bit rate and sound quality for some compressed music formats. I could choose MP3, AAC, FLAC, or whatever versus straight up PCM audio. The source was ripped CD tracks. I could not really A/B the program material, so I had to rely on memory and listening as my only tools, but this was enough to identify problems in the sound.
I found that under MP3 (I tested this format the most) I could, on some program material, detect that the audio had some defects up to about 256k. At 320k I could not tell the difference between PCM and MP3 compressed audio. The type of music that I found to be most revealing (and it just so happens that I often listen to) is jazz that includes cymbal hits (from the drum kit), solo trumpet and solo piano passages. I played piano as was always around pianos when I grew up so I am very familiar with their sound. The particular "warble" and resonances of a piano (a true acoustic piano, not an electric synthesized one) would often be rendered poorly or just wrong, and this was a give-away for me. But by 320k I could no longer detect them. For classical music there were fewer obvious issues and I could get away with listening to as low as 128k, perhaps because with the larger ensemble size I was not listening to one instrument and the sound would simply get "muddy". It's not as obvious but you can definitely tell if you listen carefully.
After this early period I no longer needed or wanted to stream compressed music, and so now I stream only PCM audio at e.g. 16bits and 48kHz so the point is moot. Now it's only the source material that I need to worry about, but this is often streaming audio sources from the internet (today's "FM radio") and I keep my previous investigations on bit rate in mind when choosing stations.
Speaking of FM, when I lived near Sacramento the local NPR/Jazz station KXJZ had an online "live stream" of the same program that was going out over FM. I often listened to it, since the host often spun some really nice material and they had a wide variety of program material (classic jazz, acid jazz, the eclectic "Blue Dog Jam", etc.). But then all of sudden one day the stream sounded horrible. Next day, the same. After a week I was curious/concerned, so I looked at the stream info and found that they had reduced the stream rate to 96k! That's fine for talk radio, but for audio it's terrible. I sent several "feedback" emails to the station pointing out the problem but this never resulted in any changes. They said they wanted to make the stream rate low so that iPhone users could also listen. WTF? When I pointed out that the music quality from the stream was terrible (really, I often had to run and turn it off because it sounded THAT bad) they didn't seem to care. So I stopped listening, and also stopped donating to the station!
Edit: I decided to check KXJZ again and, sure enough, they are still streaming MP3 at 96k. Listen for yourself and decide:
Listening Options - capradio.org
I found that under MP3 (I tested this format the most) I could, on some program material, detect that the audio had some defects up to about 256k. At 320k I could not tell the difference between PCM and MP3 compressed audio. The type of music that I found to be most revealing (and it just so happens that I often listen to) is jazz that includes cymbal hits (from the drum kit), solo trumpet and solo piano passages. I played piano as was always around pianos when I grew up so I am very familiar with their sound. The particular "warble" and resonances of a piano (a true acoustic piano, not an electric synthesized one) would often be rendered poorly or just wrong, and this was a give-away for me. But by 320k I could no longer detect them. For classical music there were fewer obvious issues and I could get away with listening to as low as 128k, perhaps because with the larger ensemble size I was not listening to one instrument and the sound would simply get "muddy". It's not as obvious but you can definitely tell if you listen carefully.
After this early period I no longer needed or wanted to stream compressed music, and so now I stream only PCM audio at e.g. 16bits and 48kHz so the point is moot. Now it's only the source material that I need to worry about, but this is often streaming audio sources from the internet (today's "FM radio") and I keep my previous investigations on bit rate in mind when choosing stations.
Speaking of FM, when I lived near Sacramento the local NPR/Jazz station KXJZ had an online "live stream" of the same program that was going out over FM. I often listened to it, since the host often spun some really nice material and they had a wide variety of program material (classic jazz, acid jazz, the eclectic "Blue Dog Jam", etc.). But then all of sudden one day the stream sounded horrible. Next day, the same. After a week I was curious/concerned, so I looked at the stream info and found that they had reduced the stream rate to 96k! That's fine for talk radio, but for audio it's terrible. I sent several "feedback" emails to the station pointing out the problem but this never resulted in any changes. They said they wanted to make the stream rate low so that iPhone users could also listen. WTF? When I pointed out that the music quality from the stream was terrible (really, I often had to run and turn it off because it sounded THAT bad) they didn't seem to care. So I stopped listening, and also stopped donating to the station!
Edit: I decided to check KXJZ again and, sure enough, they are still streaming MP3 at 96k. Listen for yourself and decide:
Listening Options - capradio.org
Last edited:
I just got 4/6 and I wasn't even listening to the files. Just random guessing.
Statistics is counter intuitive.
P. S. These kind of tests are very very easy to commit fraud on.
I decided to try this "test". Who the F*%k chose the tracks? This is supposed to discriminate anything when the source material is Coldplay or Katy Perry???
Most of these had a lot of static tones, or was processed pop music. With this source material I really could not be all that confident in which was best and there were sometimes only very subtle differences.
The piano in the classical piece sounded awful on all tracks. Did someone leave a pencil on top of the strings or something???
Anyway, I chose the 128k stream for the Katy Perry track. I only chose the uncompressed stream once, and for the remaining tracks I chose the 320k stream. I was listening through my headphones and using a low quality DAC built into the CODEC on the motherboard.
Most of these had a lot of static tones, or was processed pop music. With this source material I really could not be all that confident in which was best and there were sometimes only very subtle differences.
The piano in the classical piece sounded awful on all tracks. Did someone leave a pencil on top of the strings or something???
Anyway, I chose the 128k stream for the Katy Perry track. I only chose the uncompressed stream once, and for the remaining tracks I chose the 320k stream. I was listening through my headphones and using a low quality DAC built into the CODEC on the motherboard.
Agreed. I am getting repeatedly correct result on Suzanne Vega and Neil Young. However, if Coldplay and Kate Perry was in 32kbps, I would not be able to recognize it. Simply terrible sound job.
results with 4 min test with speakers:
I tried to identify the 3 types: WAV, 320, 128
I played each song 1 time or 1.5 time, except Jay Z , only 5 first seconds
coldplay Wav OK, 320 mixed up 128.
neil young OK OK OK
Katy, was very easy to guess the 128, but I changed my mind for the WAV and 320 at the last second.
Perahia, 128k easy to guess, but I mixed up the WAV and 320, it sounds bad.
Jay Z, this is very shitty music, mixed up 128k with wav LOL
Suzanne WAV OK, mixed up 320/128
So results with speakers :
WAV 3/6 2/6 320k 3/6 128k
I could hear a difference, I only select 128K for Jay E
my personal sound quality grading of the sample from worst to best:
JayE - Perahia - Neil Young, Cold Play, Suzanne, Katy
I tried to identify the 3 types: WAV, 320, 128
I played each song 1 time or 1.5 time, except Jay Z , only 5 first seconds
coldplay Wav OK, 320 mixed up 128.
neil young OK OK OK
Katy, was very easy to guess the 128, but I changed my mind for the WAV and 320 at the last second.
Perahia, 128k easy to guess, but I mixed up the WAV and 320, it sounds bad.
Jay Z, this is very shitty music, mixed up 128k with wav LOL
Suzanne WAV OK, mixed up 320/128
So results with speakers :
WAV 3/6 2/6 320k 3/6 128k
I could hear a difference, I only select 128K for Jay E
my personal sound quality grading of the sample from worst to best:
JayE - Perahia - Neil Young, Cold Play, Suzanne, Katy
Last edited:
It makes a big difference what kind of music is used for the test. In another thread I showed an effect of cross-over like distortion to narrow band noise (1000 - 1300 Hz). Though the distortion used seems quite low on the usual 1kHz high level sine, it is quite devastating for the low level signals. The files are here
http://pmacura.cz/noise_test.zip
and the distortion used is shown in the plot attached. If this distortion is applied to a rock music (I tried well recorded Godwhacker by Steely Dan in 96/24), it is almost impossible to tell that the distortion was added. If I added the same distortion to Beethoven's No. 9 symphony Movement No. 1, the distortion is immediately audible in the low level passage just after the beginning. So the test files should be chosen very carefully and they should contain high dynamic range and both loud and silent passages, the music that is always close to full scale level is not good for testing.
http://pmacura.cz/noise_test.zip
and the distortion used is shown in the plot attached. If this distortion is applied to a rock music (I tried well recorded Godwhacker by Steely Dan in 96/24), it is almost impossible to tell that the distortion was added. If I added the same distortion to Beethoven's No. 9 symphony Movement No. 1, the distortion is immediately audible in the low level passage just after the beginning. So the test files should be chosen very carefully and they should contain high dynamic range and both loud and silent passages, the music that is always close to full scale level is not good for testing.
Attachments
No, to conclude that you must be able to read his mind.
If you exercise an exact binomial test - like you did - all you get is the probability to reach that specific result (or even a more extreme one) by random guessing.
I'm sorry, but that is one of the points in this kind of test (or analysis) that is counterintuitive.
The analysis takes place under the _assumption_ that the nill-hypothesis is true (means random guessing; independent trials) and we are doing an exact binomial test on the data (more precisely on the number of correct responses).
The binomial test gives as a result a probability to reach this number of correct responses (or even a higher number, as it is a cummulative probability) by random guessing.
If the resulting probability is low, we conclude that the result is less compatible with our nill-hypothesis and reject it.
Otoh if the resulting probability is high, we conclude that the result is compatible with our nill-hypothesis and do not reject it.
But as we do not examine if the listener _was_ _really_ just randomly guessing, we can´t say that he was.
There might be a lot of other hypothesises that could explain (maybe even better) why the listener got this number of correct responses.
Edit: As a general reminder, doing such a low number of trials (like 6) is associated with a high risk for false negatives, if the detection ability under the specific test conditions isn't at least at 90%.
In any case it is better to do several runs and to collect all the results.
Some training and accomodation time is strongly recommended. (getting used to the specific test conditions)
Last edited:
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.