I do not discredit any effort readily, but a scientific approach has to withstand an analysis based on scientific rules. A lack of rigor is a lack of rigor and it doesn´t get better if there does not exist a study with a contradicting outcome.
You might reread my posts because i did not make any counter claim just noted some issues with the study referenced.
Do you really think it is a stretch if an experimenter of a scientific study expresses in front what the result of a listening test has to be (namely no audible difference)?
There is a whole bunch of literature in experimental psychology dealing with all possible bias mechanism and this one is something like the mother of experimenter bias.
The experimenters detected some day that the player of the first system was broken- but they did not report what number of tests was done with the broken player. Would you really call that a strech of an argument if one suspects that a broken player might not be the best one to reveal an audible difference?
With all respect, the issues posted should have only work as hints to reread the Meyer/Moran with these points in mind; but of course you are right, up to now there is a lack of cited papers in my post i´ll correct that tomorrow.
BTW, the next paper you´ve linked somewhat contradicts the results of Meyer/Moran, interestingly in a point that i´ve mentioned before- that Meyer/Moran (if their results were correct) might have found accidentally the most transparent downsample machine in the world.
As stated before, studies in audio lack often in scientific rigor especially if the experimental part comes into play.
I'll look forward to your cited papers as they pertain to the AES papers in question. I'm currently working (slowly) on a blog article that includes this very topic. So it would be useful for me to further research opposition to the studies--but it has to be credible.
So far, I've not found anything I consider to be significant valid criticism of the basic premise they tried to evaluate or their conclusion. But if it's out there, I'd love to see it. As an AES member, it would at least make for some lively conversation in NYC this fall at the convention I'll be attending.
I keep saying it, but if these studies are wrong, and readily audible differences do exist among the audio formats the studies evaluate, why is it apparently so difficult to prove that? Shouldn't it be just a matter of designing a study that's relatively free of the significant flaws that you claim cause the "inconclusive outcome" of say Meyer/Moran?
Let's face it, there's a huge amount of money behind high-end audio, and even formerly behind SACD. In fact, it's a far greater amount of money than is behind the arguments put forth by Meyer/Moran etc. Those authors have relatively little to gain.
So with all that money out there, why hasn't even a tiny little fraction of it been spent on some "scientific rigor" that demonstrates why end users (not recording studios) should keep buying gear that supports bitrates beyond 16/44? As I said, what I have seen that supports the higher bitrates is even easier to debunk.
Apple is now entering into the fray with 24 bit hardware and 24 bit iTunes content. They're an 800 pound consumer electronics gorilla that could easily afford to fund such a study with their Friday afternoon employee beer budget. But what do you want to bet they entirely fuel the demand with carefully crafted marketing hype and few of the big media voices will dare oppose them?
I'm all for "scientific rigor". I'm just waiting for someone to do a better job of applying it, for a change, in way that proves differences DO exist where previous studies have attempted to prove said differences do NOT exist.
So far, I've not found anything I consider to be significant valid criticism of the basic premise they tried to evaluate or their conclusion. But if it's out there, I'd love to see it. As an AES member, it would at least make for some lively conversation in NYC this fall at the convention I'll be attending.
I keep saying it, but if these studies are wrong, and readily audible differences do exist among the audio formats the studies evaluate, why is it apparently so difficult to prove that? Shouldn't it be just a matter of designing a study that's relatively free of the significant flaws that you claim cause the "inconclusive outcome" of say Meyer/Moran?
Let's face it, there's a huge amount of money behind high-end audio, and even formerly behind SACD. In fact, it's a far greater amount of money than is behind the arguments put forth by Meyer/Moran etc. Those authors have relatively little to gain.
So with all that money out there, why hasn't even a tiny little fraction of it been spent on some "scientific rigor" that demonstrates why end users (not recording studios) should keep buying gear that supports bitrates beyond 16/44? As I said, what I have seen that supports the higher bitrates is even easier to debunk.
Apple is now entering into the fray with 24 bit hardware and 24 bit iTunes content. They're an 800 pound consumer electronics gorilla that could easily afford to fund such a study with their Friday afternoon employee beer budget. But what do you want to bet they entirely fuel the demand with carefully crafted marketing hype and few of the big media voices will dare oppose them?
I'm all for "scientific rigor". I'm just waiting for someone to do a better job of applying it, for a change, in way that proves differences DO exist where previous studies have attempted to prove said differences do NOT exist.
Last edited:
RS, nail, head. After over thirty years of objections to careful and controlled work on issues like component audibility and transparency of 16/44, no-one seems to have been able to do any properly controlled tests showing contrary results. Some claim they have, but refuse to provide details, which is neither (IMO) ethical nor credible. Some present results where the controls are laughable to non-existent. Ditto.
It's easier to wave hands and dismiss controlled listening results you don't like than to get off one's lazy butt and try to gather contrary data. I will admit that when I finally did that, it was an epiphany.
It's easier to wave hands and dismiss controlled listening results you don't like than to get off one's lazy butt and try to gather contrary data. I will admit that when I finally did that, it was an epiphany.
No, you are just looking for "studies" to validate your personal opinion, without a personal scientific method or analysis and without an understanding of human hearing psychology.
Sorry, that is enough for me here.
While I don't disagree with the above (I can't- it's a factual test), I must respectfully disagree if the conclusion is that SACD sounds no better than CD. I own an SACD player and without fail, every disk that I own in common (both the CD and the SACD) the SACD sounds better (better meaning more detail, less noise, just overall "clearer" sounding). It's not my imagination - I can consistently hear the difference, but it is subtle and I can only hear the difference in direct A/B comparisons. I can't identify if it's a CD or SACD without an immediate direct comparison, the difference is so slight, but it's there. I thought maybe that different converters are used in the SACD player for CD vs SACD (I'm sure they are) and the converter for the CDs is poorer quality. But I get the same results with any CD player put up against my $130 Sony SACD.
However, I would bet that fewer than 1 out of 100 people would hear any difference. My wife can't and my family can't. The thing I like most about my SACD are the surround mixes (Dark Side of the Moon - AWESOME!!!!). Otherwise, SACD is a ripoff. The mp3 (as much as I hate them) derailed the greedy plans by the record companies to switch formats to SACD to try to force everyone to repurchase the music they already owned on CD for only an incremental increase in fidelity IMHO.
Not those who already own systems that can clearly expose the shortcomings of inferior formats! However, I think your statement applies to over 99% of the people out there. An SACD player will not help if you don't have a very high quality sound system. Spend your money elsewhere. Take it from someone who bought $25 and $30 SACDs of albums he already owns for a barley perceptible increase in sound quality. It honestly wasn't worth the money, but I wanted to hear the "best" that is offered to satisfy my own curiosity.
It seems to me that all that's needed to prove there is a difference in the sound of CD vs SACD would be a single individual who can consistently identify the difference.
The differences in the ability of different people to hear flaws in audio is as big as the difference between the math ability of a first grader vs. a math major IMHO. It takes exposure to quality sound to be able to discriminate between subtle differences in high quality audio. If all you listen to is FM radio in your car you would probably think a cassette tape using Dolby B sounds as good as a CD.
Some of my friends will crank up their stereos to the point of extreme distortion and they think it sounds fine. Their participation in ANY high end audio test would be worthless.
That's exactly what I wanted to say too... Some people cannot identify the differences because their limitations or adapation of their hearing.
The scientific method needs just ONE repetitive fact to prove/disprove a theory. I can tell the difference and I was amazed that my wife can tell it too (she really was a blind tester, doesn't have a clue what SACD is).
Science is not a "probability". Because that "probability" and "average value" is hiding just our incapacity to analize all the posible causes (causality).
Relaying blindly on probability is like... when you have a hand on a stove and another in ice cold water, we can tell that in "average" we feel good.
Sample testing a population that is half deaf to establish if the CD is better than SACD, and saying that 50% didn't hear differences, is meningless from scientific method point of view. ONE person that can proove constantly the fact that can identify the difference is enough.
Last edited:
One possibility is that there is a vast number of parameters in any playback system & in the listeners. A rigorous scientific approach has to deal with this variability first & foremost.
JosephK did some interesting measurements on SPDIF jitter & repeated the Hawksford & Dunne experiment on bandwith limited jitter analysis described in the paper "Is the AES/EBU-S/PDIF Digital Audio Interface Flawed?"
Here's JosephK's conclusions after doing a series of measurements taken directly at the SPDIF output of various digital sources.:
What's left to do is argue (& I'm sure many here will) over whether these sort of changes are audible after having passed through the many variables of SPDIF receiver, DAC, DAC output stage, interconnects, (preamp), amp, speaker cables, speakers.
JosephK did some interesting measurements on SPDIF jitter & repeated the Hawksford & Dunne experiment on bandwith limited jitter analysis described in the paper "Is the AES/EBU-S/PDIF Digital Audio Interface Flawed?"
Here's JosephK's conclusions after doing a series of measurements taken directly at the SPDIF output of various digital sources.:
What's left to do is argue (& I'm sure many here will
) over whether these sort of changes are audible after having passed through the many variables of SPDIF receiver, DAC, DAC output stage, interconnects, (preamp), amp, speaker cables, speakers.
I'd say you've got more work to do if you really want to claim thats a format advantage. One SACD I have (the only one I've tried this with) obviously has a different mix-down on the SACD layer compared to the CD. The difference is by no means subtle - the CD layer is massively compressed - this is from just eyeballing in Audacity.
Dear sample accurate, this test if done as suggested would then be an incorrect sampling since it wouldn't be representative of the average population which is what was suggested. If one person hear the difference but most don't, then for most people there is no audible difference. Logically the answer to the subjective question if sound from one format sounds better or different or not for most people, then only asking the individual who can hear the difference is statistically correct (but with a standard error at 100% completely irrelevant).
It would be the same as ask an political "expert" which political party they favor and use that as a proxy for outcomes across populations -that is statistically invalid and a systemic bias (ie. Non scientific).
I find the perfectly random distribution to be statistically significant, it suggests a lack of bias in the sample and also suggests that there really is no meaningful difference in what the sampled population heard across the tests, and especially so since it is true in repeated tests.
If there was systemic bias due external influences as suggested, then this would be visible across repeated samplings. The repeated studies provided suggests that you scientifically can refute the objection of systemic bias.
it would have been far more easier to argue systemic bias if the sat a did not end up randomly distributed.
Ah, no. I just asked for some sort of reference that disproves the references I quoted. I don't need to understand psychology to interpret the author's results--they're spelled out in plain English. There's a huge amount of money behind high-end audio and formerly SACD. If it really sounds better, why hasn't any of that money come up with some real proof that's at least as valid and rigorous as the studies that show there's no difference?
So a referenced, peer reviewed, study, published in the most respected audio journal in the world, with 500 trials over the course of a year, further validated with other similar tests, isn't "enough for you"? But some completely un-referenced, un-documented, "armchair analysis" is enough? Hmmmm. That sort of thinking had people believing the earth was flat.
As I said, the classic response to blind tests is usually some form of "the test isn't valid because (insert armchair quarterback reasons here) and what really matters is that I can hear the difference". People often claim they have much better hearing/listening ability and/or a much higher resolution system so that makes their situation different than all those "poor saps" who participated in the blind trial.
But there are examples where such individuals that claim to hear things other's can't in blind tests get tested themselves. Even using their own "higher resolution" system, with their own music, under whatever conditions they request they can't hear the difference either. I'll have examples in my upcoming blog article. In fact, some of these published "high profile" trials are the reason almost nobody with a public face in high-end audio will even agree to participate now. Deep down they're apparently afraid of failing despite all their claims of "obvious" differences that are easily heard.
Another thing that commonly happens is people hear differences that were not heard in proper blind tests because the differences are related to something else. So they really DO hear a difference, but it's not caused by what they claim.
For example, from what I understand, there are few completely identical CD vs SACD recordings. Even when both are from the same original tapes, they are often mastered at least slightly differently. That's one reason the AES study was done the way it was.
What's most common, of course, is it's just pure psychological bias during sighted listening. And that's not just my opinion, there are plenty of studies and articles on that. Here's one I've already provided:
Dishonesty of Listening Tests
And, finally, if differences are so small only 1 person in 10,000 (or whatever) can hear them, how much do they really matter? They become theoretical rather than practical at some point.
As for multi-channel SACD, no argument there. That's a perfectly valid reason to prefer and enjoy SACD. I should have been more clear about people who prefer it for 2 channel listening.
Jkeny, it's easy to prove the superiority of higher bitrates on paper. I don't think anybody is contesting that. Bringing jitter into that argument, as in the JosephK quotes you've posted, doesn't really change anything.
The AES listening tests I've referenced account for all sources of distortion--jitter included. You can try to make a case that with certain configurations of A/D's, D/A's, types of connections, etc. it might make a difference. But I come back to my earlier point that if it were just a matter of say using a certain S/PDIF interface that would make 16/44 sound worse, why hasn't anyone done that--especially when there's so much money to be made if they do?
There are also AES papers that discuss reducing THD from say 0.005% to 0.0005%. But nobody (that I know of) is claiming anyone can hear the difference as 0.005% is already considered below the threshold of audibility. They're simply academic/theoretical papers on distortion reduction.
Jitter is a bit controversial as it's not as easily quantified as say THD. So I'm not surprised some want to drag it into the bitrate argument as it's a more "squishy" topic. But, ultimately, it's accounted for in the listening tests I've presented. And if it didn't surface as an audible problem, it very likely wasn't.
The AES listening tests I've referenced account for all sources of distortion--jitter included. You can try to make a case that with certain configurations of A/D's, D/A's, types of connections, etc. it might make a difference. But I come back to my earlier point that if it were just a matter of say using a certain S/PDIF interface that would make 16/44 sound worse, why hasn't anyone done that--especially when there's so much money to be made if they do?
There are also AES papers that discuss reducing THD from say 0.005% to 0.0005%. But nobody (that I know of) is claiming anyone can hear the difference as 0.005% is already considered below the threshold of audibility. They're simply academic/theoretical papers on distortion reduction.
Jitter is a bit controversial as it's not as easily quantified as say THD. So I'm not surprised some want to drag it into the bitrate argument as it's a more "squishy" topic. But, ultimately, it's accounted for in the listening tests I've presented. And if it didn't surface as an audible problem, it very likely wasn't.
What you are neglecting to take into account is that the reason that the average listener can't hear the difference in many cases is simply due to the fact that their ears have not been accustomed to quality sound reproduction. Like I said, if all you listen to is FM radio you won't be able to tell CD from SACD.
People's ears have been "dumbed down" by mp3s, radio, and cheap sound systems, and most don't care, especially in this age of disposable music.
I have a good analogy: CRT computer monitors. My first monitor had a 60 Hz refresh rate. It looked fine. Then I got a 100hz monitor. After getting used to it, one day I hooked up the 60Hz monitor (I forget why). Immediately it was clearly apparent that the screen was flickering badly. I never noticed it before.
Once my eyes got accustomed to 100Hz there was no going back to 60Hz. I simply can't even look at it it flickers so bad. I repeated the same experiment with different monitors with exactly the same results.
Now that I'm used to LCD I can't look at a CRT with less than about 100Hz refresh without noticing significant flicker.
The same holds true for your other senses including touch and hearing. If your ears are used to poor sound, you will not be able to discern subtle differences in CD vs. SACD. If you're used to CDs on a good system, there's a very good chance that with careful listening that you will be able to hear the difference between CD and SACD.
Sure, if you want to cater to the lowest common denominator, encode everything in 128kbps mp3 and listen to a boom box. Since most listening is probably done in vehicles let's just abandon the quest for better sound because "most people can't tell the difference".
Most people in the general population (I don't include those who post here - this is an atypical group) aren't very bright IMHO (flame me - I don't care). But I don't think that's a reason to teach only what "average" students can comprehend and deprive the smarter pupils the more in depth education they deserve, in the same way I don't believe that audio distributors should encode audio and "dumb it down" to a format that caters to the average listener and deprive those of us lucky enough to have good sound systems (and subsequently trained ears) of quality audio sources.
"Average" is defined by what people are exposed to. If everyone had a quality sound system and an SACD player the "average" listener would demand better quality sources, and mp3s (and even CDs to some extent) would be considered below average sources.
As far as the "different mixes", that's certainly a possibility (and outright true for some albums). It's hard to imagine however that every single SACD that I have in common with the CD is a remix.
There is one SURE FIRE way to tell SACD from CD, and almost anyone can hear it. Find a very low level musical passage (perhaps the end fade out of a song) and crank up the volume so you can clearly hear it. Compare the CD with the SACD. In every case, the quantization distortion of the CD once it reaches only a few bits of resolution is highly noticable. The SACD is still clean. TRY IT!!! (If you have an SACD, which again I don't recommend buying because the discs are overpriced and the selection is extremely limited. If you like classical music the selection is better.
If you wonder why this is so long again I'm sitting in an airport waiting for a delayed flight.
Again, these are all "armchair arguments". Where's the actual proof? If it's so easy, why can't anyone present some? As I, and others like Abraxalito, have pointed out, it's hard to find an identically mastered CDs and SACDs. So it's often apples and oranges being compared.
The "few bits" argument has some truth behind it, but the problem is the "crank up the volume" part of the above statement. That's not a realistic test as few listen with an extra 20 dB (or whatever) of gain as it would cause extreme clipping during the loud passages. Listening at any realistic volume it's been shown to be inaudible. The "loss of bits" as a track fades out end up lost in the noise and/or inaudible for other reasons.
Like I said, there's no question there are theoretical advantages to higher quality formats. And you can easily prove these advantages exist. But I don't know of anyone who's proven they're actually audible in real world use. Or even anyone who's presented evidence that's at least as credible as those who have run extensive trials and tried to prove the opposite.
The above is right up there with the "earth is flat because it seems to be when I look at the horizon". Two things are really important here:
1 - These blind tests are not looking for absolute judgments (as in which wine tastes better), only any perceived differences (as in does this California merlot taste exactly the same as this Spanish cabernet). People routinely confuse the two. Even if I eat some really strongly spiced food, I can still likely tell that two wines are different. But which one I prefer might change. The same is true in listening tests. Picking a favorite might be influenced by all sorts of things including mood, time of day, what you were listening to a half hour ago, etc. But being able to hear any difference at all is much less influenced by those things. Sure if someone was standing next to a jack hammer for an hour immediately prior to a listening test it might mask their ability to hear differences, but how realistic is that?
2 - If the number of trials is high enough you can generally assume that such effects are accounted for--i.e. the listeners cover a typical range of people who would care about such things. Certainly they weren't all around a jack hammer earlier. Hopefully the group represents a typical sample of the sort of people who would care about whatever is being evaluated--like recording engineers listening to different bitrates in the AES study.
When you look at the raw data in some of these trials it's not uncommon to find that none of the participants could reliably hear any difference. In that case it's even more difficult to argue all of them suffered from some pre-conditioning effect, or were "dumbed down" by MP3 listening, or otherwise had hearing that was vastly worse than all those claiming to hear such differences.
We're not talking about if someone prefers the sound of say MP3 over WAV. We're talking about even being able to tell them apart. And that's an entirely different thing.
And you're conveniently ignoring the fact the study included recording engineers who make their living listening to small differences in sound quality. They have also been done with serious hardcore esoteric audiophiles who claim to easily hear differences in cables, etc. The results are nearly always the same. The differences people think they hear disappear when you remove the psychological bias of sighted listening.
Last edited:
Yes, the first part of my post was opinion, backed up by "my" logic. But I'm hardly calling the earth flat. And I am talking simply about being able to hear a difference.
BUT, then I gave you a sure fire test which no recording engineer with any experience with digial systems would argue with.
If you can't hear the quantization noise and distortion on CDs when the music reaches very low levels where only 4 bits perhaps are being used then you are severely hearing impaired. At this point I now simply have to agree to disagree. No argument will persuade you if you're going to outright reject a specific test I suggest to prove my point. If all you can say to that is it sounds like "the earth is flat" I give up on you.
And yes, I do believe it's possible for a large group of people to all have a poor ability to tell CD from SACD. And I also believe many recording engineers are half deaf and don't know what they are doing. Just listen to some of the CDs that are coming out.
I didn't "outright reject" your test, I said it's not realistic to "crank up the volume" (your words) to where you can hear such differences and hold that up as realistic proof. I can get out a microscope and prove that plate that you claim is clean because it just came out of the dishwasher really isn't perfectly clean. But does that matter? No. It's clean enough. Just like 16 bits is enough for the way people really listen to music.
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.