World's Best DAC's

No, because you are using different parts of the brain and are in a different state of mind … a really good test might involve subjects unaware that they were being tested and direct readout of their brain activity.
What do you mean "no"? You've got this backwards. When you say "concentrating on hearing a difference", it means more focus on certain aspect. You should talk to Jakob2 regarding that "gorilla" video. In that video, when you focus on the ball movement, you will detect how many times that ball was tossed but miss the gorilla where as when you focus on the body movement, you will be able to detect the gorilla but miss the count of ball tossing.

The brain works hard to fill in the missing bits of an audio reproduction presentation and a read-out that shows the brain working less hard to do this would be an indicator of a better DUT.
This has what to do with audio DBT? Unless you are trying to reference McGurk Effect which was posted on this thread already.

Yes. And quite a spectacular case. One of the standards committee was about to approve a lossy compression standard where 100s participated in a double blind study before approving it and about to approve it when a single outsider came in and pointed out that the algorithm created a noticable whistle in the output. Then everyone heard it where they completely missed it before.

Perhaps someone can help with the name of the person (well known) and the incident.



Yes. And althou i have forgotten most of the statistics i took at uni (i have an honours degree in statistical mathematics) i well know that there are lots of pitfalls to avoid in such testing.
Please let me know when you have gathered the info on those so that the debate can continue.
 
<snip>

Yes. And quite a spectacular case. One of the standards committee was about to approve a lossy compression standard where 100s participated in a double blind study before approving it and about to approve it when a single outsider came in and pointed out that the algorithm created a noticable whistle in the output. Then everyone heard it where they completely missed it before.

Perhaps someone can help with the name of the person (well known) and the incident.

I suppose you´re referring to the famous swedish radio tests wrt transparency of audio codecs and Bart Locanthi identifies an idle tone that remained undetected in the swedish tests.

<snip>

How artificial is "somewhat artificial"?

It depends.... 😉

Which level matched audio DBT failed to reveal the audible difference that was really there because of "somewhat artificial" listening? Have you ever done level matched audio DBT?

As stated before, there are numerous examples and we have already discussed some of these thoroughly.
I´ve started with controlled listening tests back in the beginning of the 80s after reading some articles from Dan Shanefield.

What kind of controlled listening tests? Are those level matched DBT electronic audio components? And how did they figure out that the audible differences were really there?


What you are saying is called "straw man argument".

Do you have ties to audio electronics business?

In which way is stating a simple basic fact like presenting a straw man argument?

It is a simple fact, that _every_ experiment, which is used as a hypothesis test is prone to both error possibilies and it is good experimental practice against all error mechanisms, because we should be interested in _correct_ results.
False positives and false negatives are both incorrect results.

Audio is part of my business.
 
Last edited:
A mic/SPL meter reveals the reference output at your ear(s), unlike the signal going INTO your speaker boxes, albeit with a touch less precision.

B.

If you don't effect a change in any other part of the system (and you shouldn't!), the smarter and FAR MORE PRECISE/ACCURATE way of levels matching is to use the DVM, not a microphone. We're talking orders of magnitude more precise.
 
I suppose you´re referring to the famous swedish radio tests wrt transparency of audio codecs and Bart Locanthi identifies an idle tone that remained undetected in the swedish tests.
You seem to know more about these tests. So, can you please answer some questions:
When and where were these tests?
Were these double blind tests?
Which audio codec was used?
 
Google "Bart Locanthi swedish radio test" for links.

An extract from 1 of those:

First, Locanthi related his experience of listening to a DAT tape that contained examples of low–bit-rate encoded music. He had requested the tape from Swedish Radio, the organization which conducted the official listening tests of these systems (footnote 2). Almost immediately Locanthi heard several peculiar sounds in the music, the most obvious being an idle tone at 1.5kHz.

When Locanthi informed Swedish Radio of this problem, they were surprised that they had not discovered it, but they did hear the 1.5kHz artifact after it was pointed out to them. When Locanthi asked how such an obvious flaw could go undetected, the response was that he "knew what to listen for."

It is ironic that Swedish Radio's extensive listening tests, with over 20,000 separate trials and 60 "expert listeners," failed to detect a flaw immediately apparent to a single listener. Their listening-test methodology—called "hidden reference, double-blind, triple stimulus"—was beyond scientific reproach. Yet a single listener in "unscientific" listening conditions immediately identified this fundamental problem. A paper by Michael Gerzon described later in this report comments peripherally on this issue of double-blind listening-test protocols not revealing the very flaws they are designed to detect.

Swedish Radio had previously concluded that "Both codecs [data compression encoder/decoder systems] have now reached a level of performance where they fulfill the EBU [European Broadcasting Union] requirements for a distribution codec." In other words, the system in which Locanthi discovered the flaws had already been officially proclaimed sonically acceptable as the replacement for AM and FM radio broadcasting—a replacement that will likely be in place for many decades.

dave
 
I suppose you´re referring to the famous swedish radio tests wrt transparency of audio codecs and Bart Locanthi identifies an idle tone that remained undetected in the swedish tests.
Yes, thanx.

dave
That wasn't the same kind of audio DBT (ABX) we are talking about. BTW, that report was touted by Robert Harley, the peddler of high-end audio electronics who has been trying very hard to discredit the value of audio DBT so that his business can flourish via deception. 🙄
 
It depends.... 😉



As stated before, there are numerous examples and we have already discussed some of these thoroughly.
I´ve started with controlled listening tests back in the beginning of the 80s after reading some articles from Dan Shanefield.



In which way is stating a simple basic fact like presenting a straw man argument?
See the following example of response which still doesn't cite actual example of audio DBT.
It is a simple fact, that _every_ experiment, which is used as a hypothesis test is prone to both error possibilies and it is good experimental practice against all error mechanisms, because we should be interested in _correct_ results.
False positives and false negatives are both incorrect results.



Audio is part of my business.
Dancing around the question... Yup, that is one of the traits among shill posters online.
 
If you don't effect a change in any other part of the system (and you shouldn't!), the smarter and FAR MORE PRECISE/ACCURATE way of levels matching is to use the DVM, not a microphone. We're talking orders of magnitude more precise.
Or are we talking about smart ways to ensure you aren't making mistakes in the way you test things.

For sure, a DVM will not tell you if the two DUT have matching colourations. Pink noise will tell you that in an instant.

Using a mic or your ears to judge loudness and your ears to judge colouration (in instantaneous A-B-B-A comparisons) is plenty accurate enough and sets the foundation for getting on with the next steps.

Ben
 
Last edited:
Mic closer, ears no way.

Pink noise to tell coloration in a DAC?! Instantly? What sorts of pathological effects are we talking about here?! 😱

Umm, looks like we need to start with Testing 101: you need to start this kind of equipment test by ensuring there are no irrelevant differences such as loudness (louder always sounds better) or in the tone compass (many folks just gush over rolled-off treble you get with tubes*).

Before you can start making wise comments about imaging and this and that mystery, you need settle that loudness and colouration as the same.

There's zero validity to listener judgments unless you first match for loudness and tone balance.... and then conduct quick-substitute comparisons.

Ben
*yes, soft-treble music can be very nice and there's no disputing taste; but you can make your solid-state gear behave like that with a touch of EQ and ditto for making your inexpensive DAC sound exactly like an expensive one
 
Last edited:
Err.... match voltage levels coming off your DAC (in this case it's a separate piece of equipment) with a DVM. Feed it a straight sinusoid (1k?) or something from a test disc (burn a disc with test tracks if you need to!).

That is testing 101.
 
That wasn't the same kind of audio DBT (ABX) we are talking about.

We were talking about blind tests. ABX just happens to be a commonly used one (and one of the easiest to screw up IMO). Certainly the swedish one would have been much more rigorous than an ABX. And N=20k!!!

BTW, that report was touted by Robert Harley, the peddler of high-end audio electronics who has been trying very hard to discredit the value of audio DBT

Everybody who reported was just passing on the original info published in the AES (at least that was where i 1st saw it).

And what does that have to do with the fact that it happened? A serious, well executed blind test and 20k people missed something so obvious.

dave