I feel like folks are moving the goal post here and losing sight of what I assume the point of the test is: Which one sounds better under normal conditions
If you can't determine the difference, then usually the take away is that it doesn't matter (so usually we choose the cheaper option)
When we start talking about running specific frequencies and tones to stress the DUT I wonder what the actual point is - we can easily hook up test equipment and see which device is superior on paper.
If OP is doing another test and people have some suggestions i'd say why not, generate some tones, select pieces of songs you think will sound different.
----------------------
One thing that should also be done is a test that 'fails'. Choose a really really bad dac and see if this group can successfully tell the difference and chooses 'correctly' (obviously the expensive one) which sounds better. That at way you can conclude your test group can tell the difference.
It would also be interesting if anyone has kids to bring them along see if they can determine the difference. (kids have better hearing)
If you can't determine the difference, then usually the take away is that it doesn't matter (so usually we choose the cheaper option)
When we start talking about running specific frequencies and tones to stress the DUT I wonder what the actual point is - we can easily hook up test equipment and see which device is superior on paper.
If OP is doing another test and people have some suggestions i'd say why not, generate some tones, select pieces of songs you think will sound different.
----------------------
One thing that should also be done is a test that 'fails'. Choose a really really bad dac and see if this group can successfully tell the difference and chooses 'correctly' (obviously the expensive one) which sounds better. That at way you can conclude your test group can tell the difference.
It would also be interesting if anyone has kids to bring them along see if they can determine the difference. (kids have better hearing)
Maybe i can micro-wave the FiiO or sink it in water prior of the test ?
Jokes aside, i started with the FiiO because i really thought that was the lowest possible (perceived) quality there is available, unless your goal is to find a defective or absolute worst in the world, which is not our case.
I am assuming everybody will rather seek for the best price/quality ratio.
19,99$.
Jokes aside, i started with the FiiO because i really thought that was the lowest possible (perceived) quality there is available, unless your goal is to find a defective or absolute worst in the world, which is not our case.
I am assuming everybody will rather seek for the best price/quality ratio.
19,99$.
Blind people are knowned to have better skills in that regards.
But kids, blind people or super-rare-golden-ears are not very helpful here, to be honest.
Again: that's not applicable in real-life, where audiophile's average would be north of 40 years old, male and not blind.
Finding ONE case/context that is successful would be nice, for the sake of curiosity, but wouldn't help really.
Thank you, jjrh88, you get it.
Superior or different on paper, i'm not even interested. That's not what an ABX test is about.
In fact, i feel -for a long time already- that Audiophilia is drowned under ''differences on paper'' and the whole domain is disconnected from the other side (human).
..and don't get me wrong: i have nothing against specs, technical data and measurements... But if we lose the sight of the endgame, what's the point?
Subjectivity is hard to demonstrate in prose or pictures (sorry, I mean graphs of test results) some (few) like me, don't have the means, competence (or interest, TBH) in measurement or simulation technology. I'm very interested in the findings of those that do, and even more interested to find that they are still open to interpretation and argument, this I find a source of hope and amusement in my ignorance
Maybe i can micro-wave the FiiO or sink it in water prior of the test ?
Jokes aside, i started with the FiiO because i really thought that was the lowest possible (perceived) quality there is available, unless your goal is to find a defective or absolute worst in the world, which is not our case.
I am assuming everybody will rather seek for the best price/quality ratio.
19,99$.
Agreed, goal is not to find out what the worst dac is however knowing your test group can differentiate between two things is useful. The reason is if something that really should sound worse can't be differentiated that most likely would point to a error in the test.
I use PCM2704USB sound card DAC decoder USB input coaxial fiber HIFI sound fever X8N5 | eBay at work - not great but better than my computers motherboard. There are some at ~$5. PCM2704 is 16bit however and if my google was right the fiio d3 has a Cirrus 4344 ( CS4344/45/48 | Cirrus Logic , Inside FiiO D3 | H i F i D U I N O ) which is 24bit. You probably have a cheap USB soundcard dongle kicking around which would work.
No agreed - once you have eliminated the average cases however finding the edge one is interesting. At this point it's just curiosity, this stuff is pretty neat.
For those who are visual and/or do not know me from previous work i did, i updated post #1 with some pictures.
DAC blind test: NO audible difference whatsoever.
FYI, that's obviously not a basement/residential type of context but a commercial one. The room was built for that purpose (audio tests/R&D), including the door that seals pretty well from the noises outside. Noise floor is not stellar but acceptable and can reflect our homes accurately.
DAC blind test: NO audible difference whatsoever.
FYI, that's obviously not a basement/residential type of context but a commercial one. The room was built for that purpose (audio tests/R&D), including the door that seals pretty well from the noises outside. Noise floor is not stellar but acceptable and can reflect our homes accurately.
An externally hosted image should be here but it was not working when we last tested it.
New speakers for the 3rd set-up.
(disregard the 3fe22, it's not connected)
Works very well, better than expected: the 99/94db mismatch doesnt measure as bad but rather close to 2db. Will be very easy to EQ. And also the tweeter handles well the low crossover points w/12db slope...
So far it sounds extremely detailed, very dynamic and everything i was expecting from those two drivers.
I don't think the 18FH500 is really the thing to watch here but the ribbon tweeters. That's really where the hopes are, IMO
An externally hosted image should be here but it was not working when we last tested it.
Last edited:
Did you read the link I gave for that one case you seem to be searching for?
Last edited:
If that's the point, then the elephant in the room has been assiduously avoided. To wit - blind isn't normal conditions.
There are many myths in audio.
Some people can hear differences in all sort of things.
I thought it was just me but I have trouble telling the difference between anything.
I have class ab, class d, quasi, IGBT, mosfet, bipolar etc etc and they all sound the same to me.
I guess it makes my audio buying cheaper.
Some people can hear differences in all sort of things.
I thought it was just me but I have trouble telling the difference between anything.
I have class ab, class d, quasi, IGBT, mosfet, bipolar etc etc and they all sound the same to me.
I guess it makes my audio buying cheaper.
I may have missed it along the pages, but are your results also valid for simple AB (not ABX) comparison? Out of curiosity, what if there is consistent preference for A over B? Would that suggest something?
This has happened in a very non professional and very casual test I conducted a couple of years ago comparing two cables, one diy and one from a known manufacturer. There were only two people participating and did not know beforehand that they were going to participate in a test. They also did not know what the devices under test were. Moreover, these guys are very close friends that had not seen each other for a long long time and more or less they didn't care at all about my test.
To cut a long story short, much to my disappointment, my super duper diy cable was consistently not preferred for the whole test, about 40 minutes. When I did not change the cables (although disconnected and connected them again to simulate a change), both participants answered correctly that there was no change (consistently, not even once wrong). Interestingly, when I asked them in the end what made them prefer one over the other, they mentioned more or less the same attributes. So, since I do not believe in golden ears and fairy tales, maybe, and I say maybe, is there any possibility that the more relaxed a listener is and out of the context of a test, the more he/she can discern?
On the other hand, I know well from personal experience how biased one can be based on expectation. Yes, I am talking about that frustrating moment when blind tested you realize that the differences you were sure you were hearing do not exist.
Just my two totally anecdotal cents and nothing more!
This has happened in a very non professional and very casual test I conducted a couple of years ago comparing two cables, one diy and one from a known manufacturer. There were only two people participating and did not know beforehand that they were going to participate in a test. They also did not know what the devices under test were. Moreover, these guys are very close friends that had not seen each other for a long long time and more or less they didn't care at all about my test.
To cut a long story short, much to my disappointment, my super duper diy cable was consistently not preferred for the whole test, about 40 minutes. When I did not change the cables (although disconnected and connected them again to simulate a change), both participants answered correctly that there was no change (consistently, not even once wrong). Interestingly, when I asked them in the end what made them prefer one over the other, they mentioned more or less the same attributes. So, since I do not believe in golden ears and fairy tales, maybe, and I say maybe, is there any possibility that the more relaxed a listener is and out of the context of a test, the more he/she can discern?
On the other hand, I know well from personal experience how biased one can be based on expectation. Yes, I am talking about that frustrating moment when blind tested you realize that the differences you were sure you were hearing do not exist.
Just my two totally anecdotal cents and nothing more!
That is precisely not the point of the test. The point of this test: do they sound different? As has been said, that is a necessary prerequisite for testing for preference. If you cannot tell them apart, then you cannot have a preference.jjrh88 said:
The next stage then depends on whether your aim is hi-fi or not. If hi-fi, the test is: does this sound like the real thing? If not hi-fi, the test is: do I like this sound?
We choose on grounds other than sound quality. Could be price, appearance, popularity, reliability, designer's name etc.
Any competent cable is good enough for the relatively undemanding task of getting audio from A to B over a distance of a few metres. Consequences of this are:iliverez said:
1. no cable can be significantly better than this
2. a cable has to be fairly bad to sound different from this
A consequence of '2' is that
3. many DIY cables are quite bad
Competent means:
- sufficiently low conductor resistance
- sufficiently high insulator resistance
- capacitance and inductance not too different from 'normal' cables
- appropriate construction (shielded for unbalanced, twisted for balanced) properly done (i.e. shielding near 100%, twisting fairly tight and uniform)
It is difficult to DIY a competent cable, but routine to be able to do it in an ordinary factory.
I agree & the link I gave to listening tests done over a year or so of different DACs (which the o/p is studiously avoiding) shows the outcome of this non-normal, forced-choice listening. The final blind test eventually showed the differences between DACs once a newcomer pointed out what to listen for. Once this happened a phenomena occurred which is worth noting - the people who throughout the year couldn't hear a difference, sighted or blind, between DACs could now hear differences.
The organiser, Whaleblue admitted that blind tests were a lot of the reason why they couldn't hear the differences during the year even with sighted listening i.e the blind test null results 'proved' to them that there were no differences & even their sighted listening was biased to no-difference. This new null bias was strong enough to overcome the many biases stipulated to push us towards hearing differences
You can't have it every way - if you maintain that hearing is biased & blind tests are used to remove/diminish bias then also admit that null results from blind tests bias your subsequent listening, sighted or blind to hearing no difference.
Be careful with your blind tests (particularly the usual forced choice ABX type) as they are very strong bias creation devices - exactly the opposite of what they are purported to address, removing/reducing biases.
If, as the o/p states, his test was just to show human capabilities & behaviour then reading the thread link I gave points to the significant behaviour changing technique of such blind tests which should be considered.
There is a good reason why there are standards & guidelines on how to run perceptual tests!!
Last edited:
The diy cable used solid silver 0.4mm 4N conductor, 1m length before twisting (0.75m after). The conductor was passed through a "PVC something tube" of 1.2mm. The two tubes (one signal, one return) were twisted and kept together with teflon tape. Then a copper braid was used for shielding and then a cotton tube. Typical male RCA connectors from the local electronics store at the end.
It may be bad or not, I really don't know and I don't have the required equipment to perform some measurements. To be honest, I believe it sounds pretty good and I still use it. Until I tested it with the two friends, I thought it was far better than the commercial one. After hearing their remarks, in subsequent auditions (not blind), I was not sure which one I preferred.
Eventually, and given a couple of more tests (e.g. how VTA affects vinyl playback), I don't trust my ears much anymore to state that a device/component is better than another, since my perception is heavily affected by expectation, mood and how much tired I am. So the only conclusion I can reach to now when listening is something like "I like what I hear now" or "something is not quite right with what I hear now" and of course this may change a day after...
Mr. C. Bateman has executed a very interesting research regarding the effects of cables from amp to speakers (cables_at_AF.pdf). At first glance, I get that the cable measurements can apply to interconnects. Maybe something like that could explain the perception/preference of the two other participants.
Anyway, enough with my story, back to the topic, my question or better curiosity remains:
- if the X factor is not suitable in audio tests, and if we could extract meaningful conclusions if an AB test, obviously in large scale, provides consistent results.
- if contrary to what other members have contributed, a more relaxed and "out of testing/comparing procedure" listening session can help someone to discern differences in audio (though I can see how contradictory it is to achieve something like that)
The point of the test is identification but the endgame is: which one sounds better. And even further: which one sounds best for the price, which implies nuanced ''better'', like noted wine.
Identification test is Step 1.
Appreciation A/B would be Step 2
Appreciation ranking A/B/C/D/E... would be Step 3
The only one that has scientific value (if done correctly and repeated) is Step 1.
Others have only statistical values. 63% prefered B over A, 88% prefered C over E and D was 1st of the list of 31%, etc... etc... Useful, interesting but more limited.
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.