DAC blind test: NO audible difference whatsoever

Status
This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.
Then I either look the other way or close my eyes when I don't want to know what I am listening to. I click the mouse really fast a bunch of times while focusing attention elsewhere so I don't know which track is playing.
I wonder whether subconsciously you do though, like waking up exactly the same time in the morning regardless of what time you went to sleep ;):) seriously though that sounds like quite a good method
In one file the volume level of the vibrato inflections was more uniform, and there was a tiny bit more clarity that probably wasn't really clarity but that's what it sounded like in terms of the mental experience. It may well have been a little bit of harmonic distortion, as 3rd harmonic in very low doses can make things sound more clear in a good way. It took me a few tries to lay my finger on what was different about them, and a bit more practice to get it right every time. However, i would say that was a very difficult test. Probably about the hardest I can still do at my age.
I accept that it may be possible to hear differences that aren't easily measurable. Have you attempted, when you have proved to yourself that you can hear a difference, to measure it?
 
I wonder whether subconsciously you do though, like waking up exactly the same time in the morning regardless of what time you went to sleep ;):) seriously though that sounds like quite a good method

Actually, I may reshuffle the randomization a few times just to avoid my System 1 from counting. It may help to remember System 1 can count up to six or so for most people and that's it. After that, numbers and math is genarally a job for System 2.
 
It's all about money after all, if I had more money than sense I'd probably buy the most expensive DAC I could find. I don't though, so for me, it would mean long hours of a crap job in order to buy one, I'd rather sit on my backside and listen to the music

I ordered one of those cheap ebay ESS9038 DAC boards (version 1.06) to try and to compare to the Benchmark DAC-3. Hasn't arrived yet though. I'm curious to see how they sound and if any simple mods have useful effect. When I have something to share I will post on the forum somewhere, hopefully in the appropriate place.
 
For my own use, I use the Reaper DAW. I use it to find a short segment of music to loop, and set it up to compare two tracks with one-click switching. Also use it for volume level matching.

Then I either look the other way or close my eyes when I don't want to know what I am listening to. I click the mouse really fast a bunch of times while focusing attention elsewhere so I don't know which track is playing. Then I let one loop several times then switch to the other track. Usually it is obvious right away the other track is different, but not always.

Sometimes it takes some very careful listening to one track and the other to really latch onto the differences. Then the task is the make that lock-in process work reliably enough for bullet-proof testing. One error is one too many when testing, so practice until ready for test.

Then I listen blind and make a prediction about what I will see on the screen when I look. If I have chosen the correct track, I repeat a few times to be sure it wasn't chance and I am certain I can hear the difference reliably.

If I get it wrong, I go back and keep practicing until I can do it right every time.

One problem is that in some cases there is a huge amount of information I am listening to that is going by fast, and auditory memory fills up quickly. In PMA's Toccatta violin files test, it was only very subtle difference in the sound of individual inflections of finger vibrato over a few seconds that I picked to listen to. In one file the volume level of the vibrato inflections was more uniform, and there was a tiny bit more clarity that probably wasn't really clarity but that's what it sounded like in terms of the mental experience. It may well have been a little bit of harmonic distortion, as 3rd harmonic in very low doses can make things sound more clear in a good way. It took me a few tries to lay my finger on what was different about them, and a bit more practice to get it right every time. However, i would say that was a very difficult test. Probably about the hardest I can still do at my age.

If there were two people available, it would be better to let one run the computer out of sight of the other person who is listening once it's time to test. For practicing and for finding a segment to loop, the listener should be able to run the computer if they like.
You describe my listening practice testing and self blind testing to a tee.....my exception is that I start from track beginnings when blind testing.

Dan.
 
TNT said:
I suppose you DF96 mean 2 DUT may not be physically equal (different) but still not distinguishable?
Yes. If two devices are both sufficiently close to perfect then they will be indistinguishable in sound even though they may be very different internally and slightly different in terms of the output signal.

Markw4 said:
. . . allows extremely low level discrimination of distortion . . .
If it is that low then it is fairly unimportant. Something that you have to strain to hear under certain special conditions is not going to be a problem when merely listening to music (for the purpose of listening to the music, rather than listening to the equipment).

planet10 said:
The scientific facts of the test If the null hypothesis is rejected it only applies to that particular test/day/session.
Yes. The same applies both ways: distinguishability and indistinguishability only apply to that test on that day with those people under those conditions etc.
 
If it is that low then it is fairly unimportant. Something that you have to strain to hear under certain special conditions is not going to be a problem when merely listening to music (for the purpose of listening to the music, rather than listening to the equipment).

No strain at all, it would be counterproductive. Also, there is a range of audibility. I chose to describe what I would consider resolving very small differences well enough to pass a test every time when close to a limit of audibility. It would be a mistake to assume everything is equally hard or easy.

Yes. The same applies both ways: distinguishability and indistinguishability only apply to that test on that day with those people under those conditions etc.


It depends how the experimental question is phrased. If the question is, "Can John Doe ever hear sound x under any conditions whatsoever?" then an affirmative test on a single day would answer the question persistently. With regard to the experimental question, a negative result would not prove anything.
 
Last edited:
It is interesting to note all the reasons put forward as to why we should ignore any result which shows that two devices are not distinguishable:
- its the wrong question
- its the wrong people
- its the wrong day
- its the wrong auxiliary equipment
- the people were untrained/stressed/biased/prejudiced
- the music samples were too long/short/simple/complex
- the statistics don't prove anything

Curious then that a test finding of 'distinguishable' is accepted almost without comment, even when the electrical difference combined with psychoacoustics means that the DUT ought (probably) to be indistinguishable.

I have dipped into this thread again. As things have not actually moved on I will dip out again.
 
It is interesting to note all the reasons put forward as to why we should ignore any result which shows that two devices are not distinguishable:

Curious then that a test finding of 'distinguishable' is accepted almost without comment, even when the electrical difference combined with psychoacoustics means that the DUT ought (probably) to be indistinguishable.

The point was that an experiment needs to be designed to answer a particular question. If well designed for that purpose, it should not be presumed to work equally well to answer a different question.

Regarding psychoacoustics, Earl Geddes said the findings apply to somewhere around 95% of the population. They were never intended to serve as hard limits for the entire human population, and especially not in the presence of training and practice.
 
Last edited:
The ability to hear differences largely depends on the personal expectations of the listener. On YouTube there are ridiculous clips about people listening to mains wires from NO. Not changing from a thin cheapo cable to a good one, no the audience can hear the huge improvement NO wire Mk2 shows over NO wire Mk1!
I do my personal listening tests in private, sometimes I am in the mode to listen carefully, sometimes I cant concentrate. An official listening test sure would be interesting for me, but the whole new situation probably distract me from the audible differences, if there are any.
 
It is a little bit sad, nobody here really cared about the DAC it started with. actually it´s about 156 pages OT...

What do you mean by, "...the DAC it started with...?"

Actually, the thread started with a story about some people who couldn't hear a difference between a few DACs, not one DAC. The story was okay by itself, but a claim was made by the teller of the story that nobody could possibly ever hear a difference between any modern DACs. While that may have been true in the teller's personal experience, it is not true in general.
 
As long as some people insist technical specs don't matter at all because all moderns DACs have inaudible distortion, it can never get to the point of tech being relevant for discussion. That belief of inaudibility, by the way, was largely based on 4 or 5 guys, who couldn't tell any DACs apart by ear, representing hearing ability of the entire population of the earth. An attempt was made to back it up with some old published research using fairly small numbers of untrained and unpracticed test subject. The author of some of that research, Earl Geddes, said it only applied to about 95% of the population anyway. There has never been any research conducted on the other 5%, or people such as highly trained and experienced mastering engineers. Why? Nobody cares to pay for it. It's hard to do, complicated, and expensive.

To further complicate matters, there are some issues with how ABX testing is often conducted, but there is not published research to prove it. At the most, there are some unquantified comments and impressions from researchers as to what tends to work better when testing for small differences.

Combine all of the above with firmly established beliefs on both sides, and you have a recipe for lots of forum activity.
 
Last edited:
This ABX tests IMO are highly undemocratic. If 99 persons can not find any difference between test objects, but only one is able to spot it with a high percentage of hits, 99 opinions are irrelevant.
Now one could discuss an allowable error margin, like, maybe, 10%.

In the end you got to except that 99% are wrong, which might be a problem...

Please don't see Hifi like politics, where a fraction of a percent can mean the decision between a boring and a mad candidate.
 
99 opinions are irrelevant.

Not necessarily. In most aural research the questions being asked are about the abilities or preferences of most people, not about a small group of only a few people. In such cases, averages are more important than best ever scores.


Please don't see Hifi like politics, where a fraction of a percent can mean the decision between a boring and a mad candidate.

It isn't that HiFi is like politics, it's that human brains are like human brains. When humans have already arrived at firm beliefs, it can be virtually impossible in many cases to get them to change their minds. Sometimes tribal effects are involved, some people simply seem to have more of a need for certainty in the world around them than others do, there is always psychological Naive Reality to contend with, etc.

Naive realism (psychology - Wikipedia)
 
Last edited:
Status
This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.