Let's make it clear:
The test just proved (to the OP and those who participated) that there are no audible differences under that specific test procedure and conditions, NOT that there are no audible differences in ANY conditions, and that it's impossible to hear them.
And if I understand correctly, the OP isn't interested to research for the later, because he only cares about tests under average "normal" listening conditions.
Which is understood and respected.
On the other hand, some people -myself included, are interested to know about the audible differences, even under "special" conditions.
For example, personally I'd suggest listening with electrostatic headphones (for their extreme clarity, transparency and neutrality), to proper music and sounds that would make the specific technological differences clearly perceived, selected with the process I've previously mentioned (starting with scientifically creating synthesized sounds, then use the results as a guide to select the proper music parts).
So, what if such a scientific approach would lead to a collection of specific (existing) music parts that would make those differences obvious to almost anyone and even under "normal" conditions?
1.Should we then consider another test with those parts invalid, and still consider the sound of the two devices indistinguishable because those parts are just a fraction of the possible sounds/music one can listen to?
2. Is there a threshold where the rarity of music parts that prove the differences can invalidate the test (and we shouldn't search for them)?
3. And what is "normal" listening conditions anyway, what's the "normal" listening budget threshold?
P.S
To avoid misunderstandings: I'm not supporting snake oil and ridiculously overpriced devices, in anyway.
Research cost though is high, and thus it is obvious that it will be added to special devices, even if the actual part cost is low. In high end though, we also have expensive parts and expensive (quality) labor, testing, and a rather small market and production line.
The test just proved (to the OP and those who participated) that there are no audible differences under that specific test procedure and conditions, NOT that there are no audible differences in ANY conditions, and that it's impossible to hear them.
And if I understand correctly, the OP isn't interested to research for the later, because he only cares about tests under average "normal" listening conditions.
Which is understood and respected.
On the other hand, some people -myself included, are interested to know about the audible differences, even under "special" conditions.
For example, personally I'd suggest listening with electrostatic headphones (for their extreme clarity, transparency and neutrality), to proper music and sounds that would make the specific technological differences clearly perceived, selected with the process I've previously mentioned (starting with scientifically creating synthesized sounds, then use the results as a guide to select the proper music parts).
So, what if such a scientific approach would lead to a collection of specific (existing) music parts that would make those differences obvious to almost anyone and even under "normal" conditions?
1.Should we then consider another test with those parts invalid, and still consider the sound of the two devices indistinguishable because those parts are just a fraction of the possible sounds/music one can listen to?
2. Is there a threshold where the rarity of music parts that prove the differences can invalidate the test (and we shouldn't search for them)?
3. And what is "normal" listening conditions anyway, what's the "normal" listening budget threshold?
P.S
To avoid misunderstandings: I'm not supporting snake oil and ridiculously overpriced devices, in anyway.
Research cost though is high, and thus it is obvious that it will be added to special devices, even if the actual part cost is low. In high end though, we also have expensive parts and expensive (quality) labor, testing, and a rather small market and production line.
Pictures are too coarse and lack contrast to see the part numbers. There are no schematics to tell how they're configured either. That aside, the difference in effort / circuitry put into the design is striking. I would expect the better DAC to measure significantly better. However, that does not mean we can hear it.
What's more astonishing is that both DACs ICs, probably sell for less than $3.
Attachments
Last edited:
So when he says he was very surprised by the result what he actually meant to say was that the result was entirely as he expected? When he said that this test was intended to be the precursor for tests on more expensive DACs what he meant to say was that this test was intended to eliminate the need for further tests?mmerrill99 said:
There are two reasons why two items can sound so similar that they are indistinguishable:bmwparts said:
1. they are both sufficiently good that they do not damage the sound except below the threshold of audibility under the test conditions
2. they are so similar that they damage the sound in audible but almost identical ways
For the latter you would probably need the same DAC chip (or at least the same technology with the same internal digital filter coefficients), same or similar opamps, and same or similar output caps, same or similar grounding errors, same or similar PSU noise etc. etc.
Occam's Razor tells us which of these two possibilities is most likely.
From my reading of his posts & responses to me it looks to me like he is on a mission rather than an scientific investigation but it would help if he stated what question he is trying to answer in the test - maybe it doesn't need scientific rigor?
Last edited:
You now accept the o/p is on a mission but it's OK because you claim those who give him advice on how to improve his testing technique or point out it's flaws are also on a mission!!
+1
I was one of the tester. This test is a shock as I was 100% sure that I could identify the Forssell in a snap. We made multiple listening sessions in two different setup including mine ( Lynx Aurora 8 DAC ). We both agreed about the superiority of the Forssell. Looking forward for the next ABX on another rig, hopefully more revealing this time
That might be
WMB8356EDS x
WMB8056EDS x
WMB805GEDS x
WM8805GEDS - S/PDIF tranceiver https://goo.gl/xf5PSq
WM88056EDS x
No DAC here, unless it is something else.
MV3581 x
MV358I returns LMV358 dual low volt. rail-to-rail op-amp.
Most likely, the DAC is this ..unknown hero:
The other pic says DAC: PCM1974A which it doesn't exist, but it might be a typo as PCM1794A does exist, it's a good Texas DAC chip and costs 12 or 15 euros (two versions).
"132-dB, 24-bit, 192-kHz advanced Segment, Audio Stereo Digital-to-Audio Converter"
"132dB SNR Highest Performance Stereo DAC (H/W Control)" goo.gl/3TSmgW
Of course, it's not only the DAC chip itself that counts.
Last edited:
OK, but next time try to use a cheapo DAC that we can recognize and download the PDF with its specs and architecture. Then we might be able to assist by suggesting specific revealing sounds for you.+1
I was one of the tester. This test is a shock as I was 100% sure that I could identify the Forssell in a snap. We made multiple listening sessions in two different setup including mine ( Lynx Aurora 8 DAC ). We both agreed about the superiority of the Forssell. Looking forward for the next ABX on another rig, hopefully more revealing this time
Last edited:
It's cheap enough, but what's the chip?
Who cares? I fail vs a 3000$ DAC. My life is a misery now.
I wouldn't underestimate looks. We don't like to admit they effect our opinion but they usually do at some level.
I was curious what chip was used and came across Inside FiiO D3 | H i F i D U I N O (which I believe is the dac used?) There are some other sites and a few youtube teardowns.
-----------
I suspect this thread wouldn't be 26 pages if the expensive dac won and that means maybe it's a device looking at - the cost to play is not exactly high by any standard.
Thanks for an interesting post JonBocani. I don't agree with the criticism of your conclusions for they lack the very understanding they assign to you.
"The OP doesn’t understand the distinction between: (1) being unable to discern a difference between A and B; and (2) concluding that A and B are identical. A key element of the test type used by the OP is understanding that rejecting a null hypothesis doesn’t equate to accepting, as fact, the opposite of the null hypothesis."
The null hypothesis is by default the assumption that there is no significant difference between test populations. The alternate hypothesis is that there is a difference. Experimenters test the alternative hypothesis. If there is not enough evidence to statistically prove the alternative hypothesis, they accept the null hypothesis - no statistical difference. This is what the OP did.
In general, I find the criticism of A/B/X testing to be without merit. The test was too long, or too short, or too much pressure, or unfamiliar surroundings or unfamiliar material, or the A/B/X box removed any real differences, or indigestion, or whatever.
Alan Shaw of Harbeth Speakers has a challenge that says if you can tell the difference between your well designed amplifier and his while auditioning his speakers you can have the speakers for free. 1) His speakers always make the recommended lists and 2) He hasn't paid out yet. $15K speakers for free sounds like a good deal to me. Where are the Michael Phelps or Labron James of the golden ear society?
Have a look at Floyd Toole's listening tests at Harman and the difference between sighted vs unsighted rankings of a group of speakers. This tells you all you need to know and by the way, for a component that EVERYONE agrees sounds different. I recognize that no one will change their mind on the value of A/B/X testing but the data is not on the side of the "I have to see what I'm reviewing in order to understand how it sounds" camp.
"The OP doesn’t understand the distinction between: (1) being unable to discern a difference between A and B; and (2) concluding that A and B are identical. A key element of the test type used by the OP is understanding that rejecting a null hypothesis doesn’t equate to accepting, as fact, the opposite of the null hypothesis."
The null hypothesis is by default the assumption that there is no significant difference between test populations. The alternate hypothesis is that there is a difference. Experimenters test the alternative hypothesis. If there is not enough evidence to statistically prove the alternative hypothesis, they accept the null hypothesis - no statistical difference. This is what the OP did.
In general, I find the criticism of A/B/X testing to be without merit. The test was too long, or too short, or too much pressure, or unfamiliar surroundings or unfamiliar material, or the A/B/X box removed any real differences, or indigestion, or whatever.
Alan Shaw of Harbeth Speakers has a challenge that says if you can tell the difference between your well designed amplifier and his while auditioning his speakers you can have the speakers for free. 1) His speakers always make the recommended lists and 2) He hasn't paid out yet. $15K speakers for free sounds like a good deal to me. Where are the Michael Phelps or Labron James of the golden ear society?
Have a look at Floyd Toole's listening tests at Harman and the difference between sighted vs unsighted rankings of a group of speakers. This tells you all you need to know and by the way, for a component that EVERYONE agrees sounds different. I recognize that no one will change their mind on the value of A/B/X testing but the data is not on the side of the "I have to see what I'm reviewing in order to understand how it sounds" camp.
The FiiO DAC is a Cirrus $2.55 from Digikey. CS4344-DZZR Cirrus Logic Inc. | Integrated Circuits (ICs) | DigiKey
The other DAC is a more advanced TI part cost $13 PCM1794A-Q1 132-dB, 24-bit, 192-kHz advanced Segment, Audio Stereo Digital-to-Audio Converter | TI.com . This is an expensive DAC.
Here is a similar part but lower cost at $2.90 PCM1795 costs
PCM1795 32-bit, 192kHz Sampling, Advanced Segment, Audio Stereo DAC | TI.com
The other DAC is a more advanced TI part cost $13 PCM1794A-Q1 132-dB, 24-bit, 192-kHz advanced Segment, Audio Stereo Digital-to-Audio Converter | TI.com . This is an expensive DAC.
Here is a similar part but lower cost at $2.90 PCM1795 costs
PCM1795 32-bit, 192kHz Sampling, Advanced Segment, Audio Stereo DAC | TI.com
Last edited:
It is not shown in the pics, but the author claims it is the Cirrus CS 4344.
It's because you compared a $30 device to a $3000 one.
If it's any consolation, the actual comparison is between a DAC that costs 2.5 euros vs one that costs 15 euros, that's a 6:1 difference, not a 100:1 one, therefore, mathematically you should now feel 16.7 times better!
P.S. The cheapo's PDF doesn't say much, I'll look if there is more info on other docs at a later time.
Without listening, why not record two identical clips A and B off each machine, synchronize and compare the two output signals A minus B, this result will tell you that there is a difference or not. Whether this difference is significant enough to be audible can then be debated.
From the OP's indication with his ABX test the error is not audible.
From the OP's indication with his ABX test the error is not audible.
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.