I'm pretty sure they don't. The reason for that is I have tested people with variety of DACs and even with coaching most don't learn how to hear all the differences quickly or easily. It's more like they can lean how to listen for the most obvious differences with some effort quickly, but beyond that not much else. Even then they don't always care that they are hearing a bit more clearly than before, especially if it's expensive to get it.
For most people, if anything its more like they have learned to ignore any differences between DACs rather than to notice them. They focus on music only and don't listen to fine details of reproduction. For one thing, they are rarely exposed to finely detailed, very accurate systems, and people are notoriously bad at noticing what's missing, like missing distortion, etc. People are much better at noticing what's new, and it seems to work out that going from a very good system to less good system may be more noticeable than the other way around, at least for differences in DACs. It's different when going from a transistor radio to a system with chest-thumping bass, as that is a very noticeable big, new thing. Not something that takes effort to learn to give attention to.
Changing the subject a bit, may I ask if you have tried a Benchmark DAC-3? With a good set of cans? They go to a number of lengths to make their DACs more accurate than the stock ESS chips can do in minimal implementations. They also measure very well.
In the high end audiophile market they are considered mid-tier, but in the mastering and mixing market they are right up there in the top tier, I would say. It depends what people want. Mixing and mastering engineers are hired to produce results that sound as good as possible on many different systems, and to do that they need accurate systems. On the other hand, audiophiles care more about their listening pleasure and they want what they think sounds best to them.
I am more in the mixing and mastering camp in terms of my wants and needs. Therefore, I want measured accuracy and good sound, both of them. What you want might be different. You should not assume everybody wants what you want.
Sure a DAC or amplifier is part of a system which also includes speakers & room.
Each device in the complete system has it's own role & typical flaws
Once a certain quality of reproduction (subtle details aren't being masked or exaggerated) is achieved in a system then it's possible to differentially evaluate individual devices in that system by changing out the device for another one.
the phrase "a more realistic illusion" has many layers to it - often they are intertwined. Obviously this firstly depends on the recording - if we want to hear how well the recording room ambience is reproduced this needs to be recorded appropriately - electronica is not where it will be found.And remember there is almost always recording engineer manipulation of what is initially captured so we are generally listening to a mixture of the original musicians work overlaid with these manipulations. Unless we do a recording ourselves & know what we did we are faced with the "circle of confusion" as Toole calls it - we have no reference to judge what we are hearing.
But we have our inbuilt auditory model of how sounds behave in the real world & this is in operation at all times. So, at one level, we can judge how close the cymbals are to what we determine as realistic & how much like noise we perceive them. Similarly to any individual sound object in the scene - the sound of clapping hands, violins.
At the next level the "realism" is perceived in how solid & 3D the soundstage is presented (obviously again on the recording although studio manipulation also plays a part in this). How precisely located & unwavering in the auditory spatial scene is each auditory object portrayed? Usually the correct portrayal of the individual sounds in the above paragraph results in the soundstage realism too.
Maybe so but I'm not so sure that people don't notice when they are more engaged by the music & find it more interesting on one DAC when compared to another DAC which puts all the notes in the right place but lacks something? As you said before about system 1 Vs system 2 cognitive processes. I would extend the system 1 concept a bit by saying that even though it is the quick, intuitive part of the cognitive process, it also is second-guessed by system 2 processes. A lot of people do not have the knack of being able to turn off their analytical listening & are not able to access the system 1 aspect (as evidenced in this thread) - being convinced that the system 1 assessment has validity often takes some time & multiple exposure to different music/environments.
I haven't heard Benchmark but Meitner, Lampizator, Chord, dCs & others - most of these measure excellently!!
Sure, I shouldn't assume others want the same things but I keep going back to the preference tests by Harmon on speakers - it turns out that trained/untrained; experts/non-experts; students/mixing engineers(?) generally gravitate towards the same sound preference as far as this test evaluated & it turns out that this measured as a smooth frequency response on & off axis
If you are mixing & mastering then you would be interested in this mixing engineers report of his ABX testing on sample rate differentiation - it's a great insight into how difficult ABX testing is even by an experienced individual
Foobar 2000 ABX Test - Redbook vs 192/24 - Gearslutz Pro Audio Community
I know we mostly assume this but is it because we have some superiority complex about being audiophiles? See my mention of Harmon's preference testing of speakers!
I was thinking more of everyday exposure to the real sound world as opposed to only listening to our high-end audio equipment 
Also, hearing real live music, not sure how prevalent it is these days though, and I don't mean in the concert hall (chance would be a fine thing) a pub singalong would do. Even just conversations with people, how we learn to hear is very important in this regard.
Also, hearing real live music, not sure how prevalent it is these days though, and I don't mean in the concert hall (chance would be a fine thing) a pub singalong would do. Even just conversations with people, how we learn to hear is very important in this regard.Oh, I see what you mean - yes exposure to real world of sound is important but I reckon the majority of the groundwork/modelling/patterns are mostly established in early development - I think learning to speak in 'proper' sentences is related to this pattern learning so by the time we have mastered how to speak we have established the internal auditory models. I think it's pretty impossible to avoid exposure to real world sound.I was thinking more of everyday exposure to the real sound world as opposed to only listening to our high-end audio equipment
What you say is correct though as it's something I read recently - spoken Chinese uses intonation within the sentence to communicate different meanings & foreigners thus have a great problem communicating because even though they have the correct sounds their intonation often suggest a different meaning.
The interesting aspect to this is that there are a lot more pitch perfect Chinese people than found elsewhere.
I believe this is correct but can't find my source for it - I have a feeling it may have been from Scott Wurcer so forgive if I got it wrong?
Edit: It was Earl Geddes here Who makes the lowest distortion speaker drivers
Last edited:
I found it in the above link in another interesting thread - 123 pages here but still some interesting stuff & I feel there's some meeting of minds or understanding happening on both sides, maybe, perhaps, just possibly, am I reading it all wrong??
Ah right, yes, that's another interesting fun thread. You are not reading it wrong, I'm a little out of my depth here when it comes to knowledge and understanding, but I'm not one to give up and I try to have zero ego, It does me no favours and is a hinderance to just about everything as I see it. Like everyone I have my views and beliefs which are mine and I like to keep if possible! Communication is what it's all about, it's also not easiest in this medium where there is no tone of voice to help, another reason I don't get hung up on what people say or the way they say it.
I don't want to pick through a whole thread to look for particular salient-to-me factors. ABX brings its own set of problems for detecting small differences, but aside from that one has to be careful with sample rates on most operating systems, as OS's will usually perform automatic low-quality SRC on you without any warning. That can destroy most of whatever difference there may be between hi-res and CD quality recordings. Using ASIO drivers whenever possible is probably the best defense.
Last edited:
Ok, it's only 2 pages - others might like to read through it to learn about real ABX testing & what it involves
First thing to remember is that he has a preference for high sample rate sound from his long term exposure & work with different sample rates so he just set out to 'prove' his preference was based on something 'real' by doing ABX tests
Here are some salient excerpts:
Sorry if it seems like I'm copying over the whole thread but as I went through it I couldn't help but extract what I felt relevant
So, I didn't see where he ever said whether he was using an ASIO driver in Foobar or not. If not and you try to compare two files at different sample rates, one of them is going to undergo low-quality SRC by the OS. In that case, one isn't really listening to two different sample rates at all.
The reason I posted it was not to debate the ins & outs of sample rates but rather to show a reasonably good description of the effort he goes through & the difficulties he encounters in doing ABX tests
Just as another 'datapoint'
he posts again on another forum but this time about ABX testing the audibility of jitterJitter Correlation to Audibility | Page 8 | Head-Fi.org
I won't try extracting much but note this - he listens for a completely different aspect of the sound - not soundstage this time but an specific portrayal difference in the snare drum later I think he adjusts to something else?
Again, I'm just giving people a reference to read about real ABX testing & what's involved - far more detail than is given in this thread but it reiterates points made in this thread about ABX issues
Last edited:
time to give this thread a rest......
You stated that a statistical analysis neither could nor could not reject the nullhypothesis but in fact that is the core of the process.
It is formal statistical procedure and at the end there is a decision made by comparison with a predefined criterion. So to be correct - if the probablitiy for the observed data is below the predifined level of significance - it would be the null hypothesis could be rejected on SL = 0.05. There is sometimes given the advice to just report the p-value and let the readers decide, but even that doesn´t change the formal procedure.
It is a statistical analysis based on the _observed_ data (in fact of the observed and more extreme data); the statistical analysis can´t decide whether the data was gathered in a good/sound/meaningfull experiment well planned and executed.
The analysis just answered the question how probable the observed data are under the assumption that the nullhypothesis is true.
Therefore i emphasized the role of the main quality criteria (and the needed replication(s)), if an experiment/test was objective, valid and reliable it is justified to draw further conclusions based on the result, although the experimenter still does not know if the decision is correct or incorrect.
It is important to realize that the statistical analysis of any experimental result and the assessment of the quality (wrt to the hypothesis under examination) are totally different topics.
Let´s take some examples; two DUTs were examined in a controlled listening test (ABX or A/B, 100 trials, 65 hits, SL=0.05 ) therefore the statistical analysis - using the exact binomial test - with:
H0: p = 0.5
H1: p > 0.5 (it´s a different question if a test should be two-sided, this one is one-sided)
would result in "the null hypothesis can be rejected"; actual P for this observed result (under the assumption that the nullhypothesis is true) is:
P(X >= 65) = 0.0018 .
Looks impressive, but if nobody did measurements on the DUTs we simply don´t know if the result is something to be expected or a surprising result.
Let´s assume later measurements would show a level difference of 0.4dB between the DUTs, then the statistical analysis would still be absolutely correct, but would hardly be something special.
Let´s consider the same test but with a different hypothetical result of only 45 hits, then the statistical analysis would result in "the null hypothesis can not be rejected"; actual P for this observed data (under the assumption that the null hypothesis is true) is:
P(X >= 45) = 0.864
Which means the observed data are compatible with our null hypothesis (p=0.5, which means only random guessing is at play), so we can´t reject the null hypothesis.
We don´t know if the participants were really guessing and if nobody has tested (see the matter of positive controls) it before, we don´t know if the participants would have been able to detect any difference, but anyway the statistical analysis would still be correct.
But in neither of these two hypothetical test scenarios we would be able to draw further conclusions from the results.
I am not sure, if it helps to talk about "glee", because these are facts of testing.
Obviously there are questions of epistimology touched within these procedures and following that, we have to state that "prove" is not provided (either in case of positive or negative results) and i think the proponents of the idea of corrobation of "indistinguishability" do have a point; tests wich are objective, valid and reliable can imo indeed contribute to evidence of "indistinguishability" provided that sample sizes are of a sufficient order.
Last edited:
See the post above, the statistical test/calculation/analysis speaks for itself.
(Although i´d admit having some problems to extract which number of trials was done under the different test conditions).
Now for some semantics. If your null hypothesis was "there is a difference", then you are within 95% confidence of a random result, so you cannot statistically say "there is a difference". For most people the converse is also true that "there was no difference" detected.
I don't understand the controversy it generated.
For semantics, it´s "the observed data are compatible with the null hypothesis" (i.e. p = 0.5), but we don´t know if no difference was detected (generally) or if the listeners were really only randomly guessing.
Mainly because we haven´t really examined what the participants were doing, we only have analyzed the observed data under the assumption that the null hypothesis is true.
Furthermore we don´t know if this result occures because the difference is "indistuinguishable" or because the listeners would have not been able to spot a difference in general or because the listeners were not able to detect a difference under the specific test conditions.
The "confidence level" is usually associated with the construction of a socalled "confidence interval" (that´s what DonVK did), but in these audio tests it´s mainly a socalled "significance level" in calculation the p-value.
It is an ongoing discussion if it would be better to dismiss p-values in favour of confidence intervals and the afore mentioned philosophical questions are imo still unresolved.
That includes the differences between Bayesian and Frequentist approach to probability.
The construction of confidence intervals (and therefore the confidence level) are statistical procedures and there is probability associated (given that the assumptions are correct and met) if done repeatedly.
In the Frequentist´s world there can´t be a 95% level of correctness or probability assigned to a hypothesis, because in his view the correct result is included in the confidence interval or it is not included, so it´s either 0% or 100%.
I agree with a lot of what you say, but this varies considerably from one manufacturer to another. I can speak for myself as a manufacturer.
IME many of the audio measurements that are routinely done to characterize equipment are woefully inadequate to correlate to sound quality in high-performance systems. This is why even JA of Stereophile is sometimes stumped when a really bad measuring component gets a rave review by the reviewer and why he himself hears differences in both digital and analog cables that cannot be correlated to any measurements.
I for one have been wanting to correlate digital jitter in S/PDIF signal to sound quality. I feel that a single number representing the jitter of some master clock is insufficient and really useless. Even the plots that you commonly see are useless. No real correlation has ever been done, except maybe at a really gross level, like 10nsec of jitter versus 100psec, and even this is debated as the jitter being correlated or uncorrelated to the waveform or music.
Until now.
See these plots of jitter for various S/PDIF cables:
Can S/PDIF cable jitter be measured?
This shows that some aspects of the histograms are useful to correlate to sound quality. Others, like liquid vocals are simply a result of conductor material and quality, having little to do with jitter shape, spectrum or magnitude. It will take a bit more work and more plots, but I think this is useful. Because of these plots, I made changes to my best BNC-BNC cable and improved it significantly. It's already working for me.
Steve N.
Empirical Audio
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.