Your post makes it clear that you have not followed the thread. If you had, you would know that I've clearly stated multiple times that this is NOT a proper scientific investigation. Hans and I have never pretended that it was. I can't see how it could have been scientific. This is true for multiple reasons, all of which are practical in nature. Among the necessary factors for it to have been scientific are:
1) A sufficent number of participants to achieve statistical confidence.
2) All participants have exactly the same playback system and conditions.
3) Participants judge their subjective preference via blind A/B switching.
None of those factors is practical for an investigation involving an ad hoc group of internationally dispersed hobbyist volunteers, such as ours. The investigation not being scientific does not, however, automatically render it irrelevant, or incorrect even. It renders the findings questionable, which is a different thing.
Anytime an individual chooses one piece of new equipment over some other piece because they prefer the way it sounds to them, that choice is always scientifically questionable. Which does not prevent them from enjoying the benefits of their personal decision. It's the same for this thread. Should a participant judge that they have discovered something which produces a preferable sounding playback for them, the fact that their judgement is based on scientifically questionable findings will not prevent them from enjoying the benefits of their personal discovery.
Last edited:
Tfive,
That subject has been discussed at length before here in the forum. The short answer is that what I asked you for, to my current knowledge, does not exist. However, the term 'expectation bias' has been used by cognitive phycologists to describe a bias affecting professional experimenters, not their test subjects. In addition, the term apparently has been used in aviation to describe cognitive errors pretty unique to the conditions pilots are sometimes subject to. Neither definition of the term applies to hi-fi audio listening impressions; it just hasn't been properly researched, again, so far as I know. If you want, there is a 'list of cognitive biases' on wikipedia that you can google. If you search the page for 'expectation' it will take you to the entry and links to footnotes containing the original research regarding experimenters.
Overall though, I believe that 'expectation bias' as the term is commonly used in audio forums is actually a meme, a forum myth that people keep repeating without checking for supporting science.
Human perceptual testing, blind or not, is another matter altogether and is a complicated subject. If you want I could refer you to a couple of good books. Unfortunately, its too much to describe in a forum post. The short story on that is that human perceptual testing is hard to do right. It takes a lot of expertise that EEs are not trained for. It is it's own separate area of expertise.
That subject has been discussed at length before here in the forum. The short answer is that what I asked you for, to my current knowledge, does not exist. However, the term 'expectation bias' has been used by cognitive phycologists to describe a bias affecting professional experimenters, not their test subjects. In addition, the term apparently has been used in aviation to describe cognitive errors pretty unique to the conditions pilots are sometimes subject to. Neither definition of the term applies to hi-fi audio listening impressions; it just hasn't been properly researched, again, so far as I know. If you want, there is a 'list of cognitive biases' on wikipedia that you can google. If you search the page for 'expectation' it will take you to the entry and links to footnotes containing the original research regarding experimenters.
Overall though, I believe that 'expectation bias' as the term is commonly used in audio forums is actually a meme, a forum myth that people keep repeating without checking for supporting science.
Human perceptual testing, blind or not, is another matter altogether and is a complicated subject. If you want I could refer you to a couple of good books. Unfortunately, its too much to describe in a forum post. The short story on that is that human perceptual testing is hard to do right. It takes a lot of expertise that EEs are not trained for. It is it's own separate area of expertise.
Last edited:
But they do apply in objective listening test.
So that's why you just go for subjective listening impression. It's your choice. But that's still not a valid substitute for objective listening test.
There are tools for informal blind testing, such as Foobar ABX:
foobar2000: Components Repository - ABX Comparator
I think the last blind test on this thread already shows that people sometimes think they can hear more than they actually hear.
foobar2000: Components Repository - ABX Comparator
I think the last blind test on this thread already shows that people sometimes think they can hear more than they actually hear.
P-640 - The power of expectation bias - ScienceDirect
"Expectation bias (EB) occurs when an individual's expectations about an outcome influence perceptions of one's own or others’ behavior."
"Behaviour" being your delivered results (aka answers) as a participant in a study. E.g. a study as the one that have been performed here.
So there is absolutely something in science called EB but I would think that Mk4 isn't very keen of the notion that even *he* might (read: most probably, bordering certainty) be subject to this ill-will freak of nature.
Some more references - their context might not be audio but why its not there is just that science haven't found any reason (read: money) to spend resources on it. But I cant see why evolution would have left out hearing from being affected of this psychological effect.
- ATC Expectation Bias - SKYbrary Aviation Safety
- Expectation bias - Wikipedia
- Exaggerated Expectation Bias definition | Psychology Glossary | alleydog.com
- Biases in forensic experts | Science
- https://en.wikipedia.org/wiki/Observer-expectancy_effect
I absolutely accept that it is a player and try to supervise my reasoning and behaviour accordingly when I interact about the the subject of audio - like here, on a DIY discussion forum.
//
"Expectation bias (EB) occurs when an individual's expectations about an outcome influence perceptions of one's own or others’ behavior."
"Behaviour" being your delivered results (aka answers) as a participant in a study. E.g. a study as the one that have been performed here.
So there is absolutely something in science called EB but I would think that Mk4 isn't very keen of the notion that even *he* might (read: most probably, bordering certainty) be subject to this ill-will freak of nature.
Some more references - their context might not be audio but why its not there is just that science haven't found any reason (read: money) to spend resources on it. But I cant see why evolution would have left out hearing from being affected of this psychological effect.
- ATC Expectation Bias - SKYbrary Aviation Safety
- Expectation bias - Wikipedia
- Exaggerated Expectation Bias definition | Psychology Glossary | alleydog.com
- Biases in forensic experts | Science
- https://en.wikipedia.org/wiki/Observer-expectancy_effect
I absolutely accept that it is a player and try to supervise my reasoning and behaviour accordingly when I interact about the the subject of audio - like here, on a DIY discussion forum.
//
Let me give you my unpopular unscientific view on this:
Threads like this with the experiments we do, motivates and helps thinking about what we have today in our audio reproduction chain and what we could do to make it better in our ears, whatever that means. At the end it is a personal choice for more having more pleasure listening to music. It cannot be a general outcome for everyone to use. But again it could stimulate others to do similar things and end up enjoying their music more. Which is the overarching goal of all this audio hobby and DIY
Example: I would never have tried the HQPlayer software and the different filters and see what that does. This thread motivated me to do so.and that is great
And also, during the whole discussion journey I learned interesting aspects about the whole subject. It broadens my horizon…
So I enjoy it very much. Again a win.
That it is not scientific, I could not care less, as we are not solving world hunger or inventing a vaccine.
Doede
Threads like this with the experiments we do, motivates and helps thinking about what we have today in our audio reproduction chain and what we could do to make it better in our ears, whatever that means. At the end it is a personal choice for more having more pleasure listening to music. It cannot be a general outcome for everyone to use. But again it could stimulate others to do similar things and end up enjoying their music more. Which is the overarching goal of all this audio hobby and DIY
Example: I would never have tried the HQPlayer software and the different filters and see what that does. This thread motivated me to do so.and that is great
And also, during the whole discussion journey I learned interesting aspects about the whole subject. It broadens my horizon…
So I enjoy it very much. Again a win.
That it is not scientific, I could not care less, as we are not solving world hunger or inventing a vaccine.
Doede
Please see below a copy and paste of a highly relevant report from John Swenson, on a different forum nearly ten years ago. Ive bolded and underlined important points for speed readers convenience. I hope this copying isn't out of order and hope John and diyaudio folk don't object to this? Please ask a moderator to kindly delete this if it offends.
John had a similar question to this thread about what makes NOS sound different and built two NOS DACs one R2R and one noise-shaping (PCM1794 just like DDAC in above posts) to test his hypothesis:
John had a similar question to this thread about what makes NOS sound different and built two NOS DACs one R2R and one noise-shaping (PCM1794 just like DDAC in above posts) to test his hypothesis:
Last edited:
This is the first time I hear about “ dirty sound” produced by a NOS Dac at the high end of the audio range.
First thoughts are that this might be caused by intermodulation from speakers.
Other cause might be a somewhat inferior I/V converter.
But being both rich and dirty seems like an non digital issue.
Hans
First thoughts are that this might be caused by intermodulation from speakers.
Other cause might be a somewhat inferior I/V converter.
But being both rich and dirty seems like an non digital issue.
Hans
Dirty sound ?
Thanks Kazan for posting this.
It some how correlates with my finding in my previous post 1232 on using HQPlayer with the polynomial-1 filter
Sound was cleaner but did not lack the engaging NOS sound
Must say, that with FIR filtering in HQP this was not the case. The NOS sound was negatively impacted.
Thanks Kazan for posting this.
It some how correlates with my finding in my previous post 1232 on using HQPlayer with the polynomial-1 filter
Sound was cleaner but did not lack the engaging NOS sound
Must say, that with FIR filtering in HQP this was not the case. The NOS sound was negatively impacted.
Last edited:
Now, I think there might be something wrong with 176 files.... 88 files are a bit better in some regards, but not by much.
Still Got The Blues and Day 0:
The 44 files sound natural, spacious, I can clearly see into the music, especially with a solo vocalist and the choir clearly placed far (deep) behind him (Day 0), the choir sounded properly extended left<->right only with 44.1. All the 44.1 tracks sound vastly superior in every way, especially compared to 176. The difference is staggering when I used May DAC in NOS. Also, the bass (God Give Me Strenght) was properly defined only with 44.1; the other two (especially with 176) bass was loose - not in focus at all - like someone put the bass on toast and spread it... (The Birdcage comes to mind
)
I also checked the same files with OPPO205 and Sennheiser HD650 headphones. I thought that my Aleph J with only 2 gain stages, DC-coupled, was too revealing. But, the difference was even more prominent stereo separation-wise with 205 (which is by no means an audiophile DAC). In particular, the 176 files sounded half-as-wide, compared to 44.1 files. It is like the 44.1 sound was what I would expect to hear, with stereo soundstage extending outside the cans; but with 176... there was an almost 50% reduction in stereo separation (width)... which is what prompted me to write this post.
Day 0:
The audience clapping sounded like the real deal only with 44 file - really nice - impressive; maybe the files have not been recorded properly. The 44.1 files had more information (more everything), better placement - they sounded more dynamic -> very enjoyable.
This by no means is to understate the amount of effort and an excellent track selection, well done in that regard and thank you, Hans.
Still Got The Blues and Day 0:
The 44 files sound natural, spacious, I can clearly see into the music, especially with a solo vocalist and the choir clearly placed far (deep) behind him (Day 0), the choir sounded properly extended left<->right only with 44.1. All the 44.1 tracks sound vastly superior in every way, especially compared to 176. The difference is staggering when I used May DAC in NOS. Also, the bass (God Give Me Strenght) was properly defined only with 44.1; the other two (especially with 176) bass was loose - not in focus at all - like someone put the bass on toast and spread it... (The Birdcage comes to mind
)I also checked the same files with OPPO205 and Sennheiser HD650 headphones. I thought that my Aleph J with only 2 gain stages, DC-coupled, was too revealing. But, the difference was even more prominent stereo separation-wise with 205 (which is by no means an audiophile DAC). In particular, the 176 files sounded half-as-wide, compared to 44.1 files. It is like the 44.1 sound was what I would expect to hear, with stereo soundstage extending outside the cans; but with 176... there was an almost 50% reduction in stereo separation (width)... which is what prompted me to write this post.
Day 0:
The audience clapping sounded like the real deal only with 44 file - really nice - impressive; maybe the files have not been recorded properly. The 44.1 files had more information (more everything), better placement - they sounded more dynamic -> very enjoyable.
This by no means is to understate the amount of effort and an excellent track selection, well done in that regard and thank you, Hans.
Thanks for posting your finding. The soundstage collapse with the upsampled file seems massive.
I wonder if the soundstage collapse is from that particular filter upsampling which eg could conceivably change the phase? Or is it the Holo May DAC performing better with lower sampling rates?
Does anyone else have the luxury of a DIY R2R NOS DAC free of an internal programmable processor? Its hard to know what some modern commercial "NOS " DACs actually do with the PCM data, after the Denafrips "NOS" mode got busted recently.
I do confirm about 'dirty' sound on the poor amplifier/speakers with NOS DAC. I spent a lot of time selecting a right amp with my limited financial resources.
However when dirty sound is noticed specifically in the high end of the audio range, it indicate that Delta-Sigma DAC was used for tests. On the poor distorting audio chain R2R NOS DAC produce 'dirty' sound equally in the enire audio range.
Or... My NOS DAC has a very well designed output stage. /A second thought/
Last edited:
Of course there is. However, extrapolating that to subjective hi-fi audio listening impressions from a mish-mash of published opinion and or published science involves its own form of error-prone bias.
For example if one looks though the list of cognitive biases at wikipedia, the brief descriptions of some of the biases sound so similar that it can be hard to believe they aren't duplicate entries with slightly different wording. However if one studies the supporting research it turns out the biases are experimentally distinct.
Therefore until a serious study takes on hi-fi audio listening impressions, I don't think there is much of a scientific basis to stand on. Also, I think a close look at the kind of cognitive errors that do sometimes occur in hi-fi audio listening will show that they are not random. As tends to be ubiquitous in human biases, errors mostly take recognizable forms. Therefore I seriously doubt that human listening errors can be all attributed to a single bias. From my own observations I would say there are probably a few common cognitive listening errors, most of which can be overcome with some awareness and practice.
Regarding a point brought up about ABX testing, as Jakob2 has repeatedly pointed out, the published sensitivity of ABX is worse than A/B or other test protocols. Jakob2 also pointed out that ABX sensitivity can be improved with sufficient test subject practice. IME the problem with ABX is that for small differences it requires memorizing the audible differences by recruiting conscious awareness to effortfully do so. OTOH, informal listening impressions seem to be primarily formed by (hopefully well-trained) System 1 processing (where the two systems are as described by Kahneman). Thus ABX requires different brain processing than casual listening does; IMHO a well-trained System 1 is no longer sufficient.
The forgoing is entirely my own personal opinion, nothing more.
Last edited:
Thanks, for that detailed report. Can you confirm whether you listened to all files, including the 88 and 176, in NOS DAC mode or not?
I would like to remind everyone to please submit your reports directly to Hans via PM, to eliminate any possibility of it influencing the findings of others still conducting the experiment. Also, don't forget to indicate your preference/non-preference for each 176 file, and specify whether your indication is relative to it's respective 88 and/or it's 44 file.
Thank you.
Interesting. Our subjective findings regarding apparent audible artifacts from 'typical' OS, versus the lack of such artifacts from high performance OS, seem to support those of John Swenson. Regarding John's observation of NOS having a "dirty" sound, Abraxalito, in post#35 https://www.diyaudio.com/forums/digital-line-level/371931-makes-nos-sound-4.html#post6645400, mentions hearing something seemingly similar with non-sharply filtered NOS. Depending on the musical program content, and which disappears with 7th order analog anti-image filtering. So, this seems another mystery. What is it about the first image-band that could cause such an subjective effect?
Since all digital audio image-bands are ultrasonic, it seems this could only be due to some form of intermodulation. The immediate suspect is, of course, intermodulation within the electronic playback chain. However, this would seem to potentially be a problem affecting only a clear minority of insufficiently designed/implemented components. A secondary suspect may be an old one which we removed from our suspect list. Largely because the readily found professional research on it appeared very sketchy. Which is, the question of whether or not the ear structure itself could produce audible intermodulation products from otherwise inaudible ultrasonic frequencies?
Last edited:
Jakob2 posts with a business agenda and anyone who tries to prop him up as some kind of authority is also trying to push a business agenda.
"as audio development was and is part of my work"