ABX, MP3 and other stories

[Evan Robinson]

This might be a poor choice for my first post on these boards.

John Hope: Have you ever done an ABX comparison of a CD to a mp3 encoded with a high quality codec such as LAME? If you were to rip a CD and encode it at a setting that produced mp3s with a VBR of around 200kbps and run an ABX test, I doubt you could identify the mp3 even on a high end system. A guy I know ran this test on a $100,000+ rig with two guys that do studio work and four others that own a hi-fi store and I don't believe any of them passed. Well, I should say none passed at this quality setting or higher.

[John Hope]

Hi erobinson

Welcome to the forum and to this thread.

I would observe that the compression used in the current systems intended for consumer audio such as DAB and the audio of DTV uses technology that is outdated. I stand corrected, but I think it's MPEG1. The highest bit rate used on DAB is 192kbps (BBC3 classical) and most channels use 128kpbs.

Virtually everybody I've come across can discern the quality impairment on these compression systems, even on relatively modest hifi equipment, especially when playing 'busy' musical pieces with a lot of percussive high frequency content. The coders simply run out of available bits to code. Something noisy from the 70's, like Bob Seger's Katmandu is an excellent test.

The thrust of this thread is on the disappointing quality of recording of some CDs. So we're already developing a good idea of the hearing abilities of some recording engineers! These guys are certainly not Ear Gods. There's a school of thought that believes that persons with long-term visual impairment often make the most critical audio listeners, because their sense of hearing is fine tuned to compensate for their visual deficiencies.

I'm not familiar with this LAME codec you mention and so couldn't comment on how subjectively transparent it might be, except to say it has an unfortunate name! Perhaps you have some links to sites offering details of this system ?

Best Wishes

John Hope

[Evan Robinson]

The LAME codec does have an amusing name. It is an open-source project, and the acronym stands for "Lame Ain't an Mp3 Encoder." Of course, the first line on their website states that LAME (uppercase) is an LGPL mp3 encoder. It is about six years old now and what I find most people ripping their CD's at high quality use. Here is a link to their webpage: http://lame.sourceforge.net/

You're definitely right about 128kpbs recordings. Of course, this depends on the original source as you mentioned. I can't stand when someone has obviously ripped their CD's to 128. They will put a burned CD into my car stereo system and I often have to ask them "what bitrate is this recording?"

Thanks for the welcome by the way, and sorry for steering this thread a little off track. I'm a young guy (20) and don't have a lot of CDs (around 70), so I don't have a lot of material to compare recording qualities against. I do have an embarrasing number of mp3's, but how can you be sure it is the mp3 or original recording that causes the problem?

As I get closer to the topic of this thread, I have begun to notice this "loudness" battle going on in the recording industry. Mostly on rock and hip-hop (probably not very popular in this forum). Some CDs it is just so obvious they have overly used compressors, making the CD lose all of its dynamics. And without dynamics, you are losing one of the most important things to quality music! I may notice this more than the average CD buyer because I've played in a symphonic band for 10 years. I can't count the number of times the director would ask for more dynamics, especially from the horn section! Of course, the average buyer, listening on his stock radio will think a piece that sounds "loud" also sounds better. But, it will leave the ones that care with a look of distaste on their faces.

One last thing. How can such great artists like the Red Hot Chili Peppers (subjective of course) produce such poorly recorded CDs? I guess they just don't care enough about the fans that have quality equipment to make that extra effort.

Evan

[Bas Horneman]

'The Essential Billy Joel" Columbia

Bought Billy Joel's greatest hits for 1 euro (Vinyl) sounds allright!

Bought Lois Armstrong's greatest hits for 1 euro..sounded brilliant...lifelike....probably recorded with tube preamps!

I somehow stopped buying cd's..on a subconscious level...(only realized this recently since I own a turntable (good one)) I have not been re-discovering vinyl...I am discovering it..since I grew up in the CD era.

John Hope: Have you ever done an ABX comparison of a CD to a mp3 encoded with a high quality codec such as LAME? If you were to rip a CD and encode it at a setting that produced mp3s with a VBR of around 200kbps and run an ABX test, I doubt you could identify the mp3 even on a high end system. A guy I know ran this test on a $100,000+ rig with two guys that do studio work and four others that own a hi-fi store and I don't believe any of them passed. Well, I should say none passed at this quality setting or higher.

interesting but what can we conlude from it? ....plus .....price says nothing about the sound quality...nor does your occupation.

But I must say that music recorded in this Millenium have been noticeably better.

. Which recordings are you refering to?

[SimontY]

Originally posted by erobinson
If you were to rip a CD and encode it at a setting that produced mp3s with a VBR of around 200kbps and run an ABX test, I doubt you could identify the mp3 even on a high end system. A guy I know ran this test on a $100,000+ rig with two guys that do studio work and four others that own a hi-fi store and I don't believe any of them passed. Well, I should say none passed at this quality setting or higher.

That's very interesting to hear.

Once my house mates (when I was still a student ) played me a track taken from CD and put into 320kb MP3 and the same thing encoded as full quality OGG. I could tell the difference, and I preferred the OGG.

Maybe when these pros were listening on the ultra-expensive system their expectation was too great and ruined the test. I was under no pressure and expected to hear no difference.

I have tested using throwaway quality PC speakers for differences between 128kb and top quality VBR MP3s and scored well enough to conclude there was a real difference. I'd imagine higher bit rates could be differentiated using a hi-fi, in a relaxed environment.

I think MP3 sound is not off-topic on this thread, as some of the common failings of MP3-sound seem to afflict CDs - notably some that feature copy-protection.

Listening to a disapointing CD and then looking on the back to find a 'copy-protected' warning is sort of the opposite to going "wow, this Joni Mitchell sounds amazing" later to find the tiny HDCD logo there... hehe. If only they were all HDCDs!!!!

[Evan Robinson]

Originally posted by Bas Horneman
interesting but what can we conlude from it? ....plus .....price says nothing about the sound quality...nor does your occupation.

What can be concluded? Well, if many sets of trained ears could not identify the mp3 at the quality setting it was encoded at then it can be concluded that it is unlikely other people could tell the difference either, and thus a mp3 of that bitrate is satisfactory to encode since you will never know if you are listening to the original recording or not.

The insult at my "occupation" as an engineering/premed student was uncalled for. I don't need a degree in an audio field to understand an ABX test and how it is the only truly scientific method.

Originally posted by SimontY
Once my house mates (when I was still a student ) played me a track taken from CD and put into 320kb MP3 and the same thing encoded as full quality OGG. I could tell the difference, and I preferred the OGG.

But this also was not an ABX test, and how can we not say psychology played a role?

Maybe when these pros were listening on the ultra-expensive system their expectation was too great and ruined the test. I was under no pressure and expected to hear no difference.

They were doing it for fun, not an article. They are all friends and wanted to see if they could tell a difference. I'm not sure what pressure they would have been under. Even if they were, they were under no more pressure than they would be in a non-ABX test, and ABX like I said before is the only valid method.

I hope I have not come to a forum where ABX testing is not considered valid. If so, what other way do you guys suggest?

[Bas Horneman]

The insult at my "occupation" as an engineering/premed student was uncalled for. I don't need a degree in an audio field to understand an ABX test and how it is the only truly scientific method.

I did not mean YOUR occupation..but ONE'S occupation. Working in a Studio or hifi shop does not automatically mean you have good hearing. The proof is an incredible amount of rubbish studio's and hifi shops sell and produce.

I therefore don't find it a valid test WHATSOEVER...a dutch computer magazine did the same type of test...with a similar crowd..and got the same results...TO ME that does not mean that one cannot hear the difference...I am convinced that in a GOOD system...almost everyone could hear it. But I'm not attacking you.....I just find the 100 000$ system and people's job (hifi store/studio people) completely irrelevant. However for the people involved in that test..it was completely true.


If you knew me...you would know that I NEVER intentionally insult people...EVER. And for the brief period you felt insulted by my less than perfect English I profusely apologize.

[SimontY]

Originally posted by erobinson
I hope I have not come to a forum where ABX testing is not considered valid. If so, what other way do you guys suggest?

Please see the many, many threads on ABX/blind testing.

The way I test is to gauge enjoyment over a long period with all music. This is what tends to work for me. But it aint scientific, and it's slooow progress sometimes. I would imagine a combination of methods should produce the most insight when auditioning equipment.

Anyway, this has nothing to do with bad CDs. Music listening is all about enjoyment, or lack thereof.

[SY]

And as a quick note from someone who does sensory testing for a living, ABX is NOT the only valid method. It's just one tool (a damn good tool!), but unfortunately, it's become stuffing for many straw men.

[SY]

BTW, to answer your question about other methods, there are many, many, many of them.

For example, A/B preferences. A subject is given a pair of stimuli and chooses a preference. This is repeated with the stimuli being randomly shuffled in order. If the subject consistently picks one as preferred, that's a significant result.

Here's another: paired stimuli with "same" or "different" randomly determined. If the subject consistently can detect same or different, the result is significant.