I always perform blind test with real blind and deaf and drunk people ! The noise floor is always very low and the background always dark. So indeed , all the DACS sounds the same.
More seriously, there are two sorts of blind tests. One is for people with no hearing training or musicians involved for the mass markett. The second is with trained ears not involved in the designs and musicians, no drunks is better.
When the first not trained to hear group is at work, with unknown reccordings or tracks tests. The memory is even less precise and only gross difference or general feeling is percieved. Very usefull for mass markett to make just minimal job and avoid diminishing return. Marketing dpt does the job after to make it "sounding" better ! aka ads and biased magasine reviews whatever internet or paper (always look at the economic model of the place one take the "advices")
The second one is more tricky : microdynamic and notes are better heared. My little experience is when you make test your designs by such people is to work both with reccordings they know by heart and with unknown reccordings. It is rich and learning. The two groups don't feel and rank the same. First can not hear much difference sometimes or rank from feeling. There biass everywhere despite road map is given to them, the natural comes back with horses.
Second one is much more precise but can be biased by the fact the group is maybe not big enough (few people) and the same when it is your friends or people you know. But for I there is no doubt it gives good results. A problem is of course, most have just one hifi and room, so the result is subjectiv. It is better if you can to use two different sounding systems. And each time you must mitigate the results as what is percieved is a whole.
Also if the memory is not bad in a session, it is lost the day after if you wwant to benchmark two systems with different days, but it can be usefull to confirm the results (the victim for such borring tests for us is most of the time the wife).
What you learn is important : The best sounding devices for people has nothing to see with the THD-N .
Also after 30 minutes of music without rest, tests becomes not usefull. Measurements migth comes to reassure the technical people where the cartesian spirit side of the brain needs its food with the incoherency we all know.
It worths what it worths, but I am sure I am not the only one to have observed that. At the end all the systems are not the same and some devices sounds better than others in an hifi because of compensation and in room DI, age of the hearing, etc. But it is always to rank what is giving never good results enough.
I have good fun with a member here who is testing my last DAC pcb and eventually will buy it for few euros, with subttle setuping options : and the most important setuping seems to be the same but the little options that makes it sounding sligthy different are close but not the same : Bingo,not the same hifi, loudspeakers, etc. It migth be benchmarked I hope with a little Holo DAC by a friend to him. Here it is not a blind test if the Holo DAC is known, but at least it can be interresting while it biasing the final result, we of course all know that the brand, the people from who you buy, his name and so on is involved in the ranking.
I am learning nothing to you audio folk here for sure. But YMMV...
More seriously, there are two sorts of blind tests. One is for people with no hearing training or musicians involved for the mass markett. The second is with trained ears not involved in the designs and musicians, no drunks is better.
When the first not trained to hear group is at work, with unknown reccordings or tracks tests. The memory is even less precise and only gross difference or general feeling is percieved. Very usefull for mass markett to make just minimal job and avoid diminishing return. Marketing dpt does the job after to make it "sounding" better ! aka ads and biased magasine reviews whatever internet or paper (always look at the economic model of the place one take the "advices")
The second one is more tricky : microdynamic and notes are better heared. My little experience is when you make test your designs by such people is to work both with reccordings they know by heart and with unknown reccordings. It is rich and learning. The two groups don't feel and rank the same. First can not hear much difference sometimes or rank from feeling. There biass everywhere despite road map is given to them, the natural comes back with horses.
Second one is much more precise but can be biased by the fact the group is maybe not big enough (few people) and the same when it is your friends or people you know. But for I there is no doubt it gives good results. A problem is of course, most have just one hifi and room, so the result is subjectiv. It is better if you can to use two different sounding systems. And each time you must mitigate the results as what is percieved is a whole.
Also if the memory is not bad in a session, it is lost the day after if you wwant to benchmark two systems with different days, but it can be usefull to confirm the results (the victim for such borring tests for us is most of the time the wife).
What you learn is important : The best sounding devices for people has nothing to see with the THD-N .
Also after 30 minutes of music without rest, tests becomes not usefull. Measurements migth comes to reassure the technical people where the cartesian spirit side of the brain needs its food with the incoherency we all know.
It worths what it worths, but I am sure I am not the only one to have observed that. At the end all the systems are not the same and some devices sounds better than others in an hifi because of compensation and in room DI, age of the hearing, etc. But it is always to rank what is giving never good results enough.
I have good fun with a member here who is testing my last DAC pcb and eventually will buy it for few euros, with subttle setuping options : and the most important setuping seems to be the same but the little options that makes it sounding sligthy different are close but not the same : Bingo,not the same hifi, loudspeakers, etc. It migth be benchmarked I hope with a little Holo DAC by a friend to him. Here it is not a blind test if the Holo DAC is known, but at least it can be interresting while it biasing the final result, we of course all know that the brand, the people from who you buy, his name and so on is involved in the ranking.
I am learning nothing to you audio folk here for sure. But YMMV...
Last edited:
Lot of good things in your post.
But I have no idea what this means: 'I am learning nothing to you here for sure'....
Jan
But I have no idea what this means: 'I am learning nothing to you here for sure'....
Jan
Last edited:
Sorry for my basic english. I meant my post is not giving new informations. You all know it already. It is my experience only I wanted to give as a testimonial because it is always good to rewrite the basics and better said than not.
What may differs is the results, but at least it is what I experienced in these limited experiments within the perimeter I described. real ABX cost a fortune, not fun to perform.
What may differs is the results, but at least it is what I experienced in these limited experiments within the perimeter I described. real ABX cost a fortune, not fun to perform.
Hifi is like a photograph, it is never a copy of the reality. We hunt this goal, but it is a marketing mantra whatever we aproach this as we progress in science dpt.
A piccture is an aethetic view of the reality that magnify or undermagnify it, focussing on different things according what the photographer want to achieeve. Hifi is the same. What we try to do is to make it nice in the design whatever it is the reality or better (or worse) than the reality.
For the moment I nnever heard a hifi sounding like a cello in a room !
A piccture is an aethetic view of the reality that magnify or undermagnify it, focussing on different things according what the photographer want to achieeve. Hifi is the same. What we try to do is to make it nice in the design whatever it is the reality or better (or worse) than the reality.
For the moment I nnever heard a hifi sounding like a cello in a room !
First look for very good recording and after for a very good room and after for a very good speaker that match with your room and amp and last but no less important the source.
... and you'll will not have yet something sounding like a true cello but you willl be nearer to it VS if you pay no attention to all of that.
People need to be taught what to listen for. And even then not everyone will be able to discern subtle differences. Also what you are listening to will effect your results. Orchestral music that is prone to congestion or music with bells and horns will all allow a better dac to differentiate itself where as dynamically flat compressed music will not.
I have a HPA here (with no crossfeed) which measures -130dB distortion. It had a problem with a lopsided soundstage which was fixed by replacing the Alps pot with a Goldpoint attenuator. I then tried putting the HPA into my system to see if it could be used as a transparent line amp. Tried putting it between the dac and the existing preamp, and then tried putting it after the existing preamp but before the power amps. It was not transparent at all, and it affected the sound about the same way in both trial positions. Problem wasn't noise floor or nonlinear distortion. My guess based on some experiments with different types of thin metal film resistors in dacs, is that the HPA has a problem with (possibly current) noise intermodulating with the audio signal. Kind of has that sort of intermodulated 1/f noise sound signature to it. Of course, nobody who designs and optimizes using an AP 555x always bothers to test for noise intermodulation, since there is no pushbutton test on an AP for that.
Where am I going with this? The experience has me wondering if the reason some of the self-proclaimed objectivists who can't hear differences between their dacs might have designed their whole systems using an incomplete set of measurements combined with an absence of skilled listening tests. For them, all dacs probably do sound the same. Because their systems are so screwed up that everything sounds the same. Haven't had a chance to investigate further so I don't know the answer to my little puzzle question. Still wondering about it though.
What has this got to do with the present discussion about listening tests? Its this: it may not be enough to tell people about what they need to listen for. Maybe more effective to have them come to a system that is already very revealing. Invite them to bring music, some component from their system such as a preamp, then sit down, do some listening, try substituting some different devices in a casual way (or blind if you want), and see how things go.
That said, I already know from experience that some people are not going to hear/notice small difference at first after only one sitting. For some people it can take months or maybe around a year of starting to learn to notice more and more little details which were always there, just unnoticed. Of course, this always has to focus on discrimination and not on preference. Preference can be discussed to, but the skill is in developing discrimination for small differences (without fooling oneself too often along the way with vague imagined differences, which may come in part from trying to progress too far too fast).
Anyone else have any experiences like that?
Where am I going with this? The experience has me wondering if the reason some of the self-proclaimed objectivists who can't hear differences between their dacs might have designed their whole systems using an incomplete set of measurements combined with an absence of skilled listening tests. For them, all dacs probably do sound the same. Because their systems are so screwed up that everything sounds the same. Haven't had a chance to investigate further so I don't know the answer to my little puzzle question. Still wondering about it though.
What has this got to do with the present discussion about listening tests? Its this: it may not be enough to tell people about what they need to listen for. Maybe more effective to have them come to a system that is already very revealing. Invite them to bring music, some component from their system such as a preamp, then sit down, do some listening, try substituting some different devices in a casual way (or blind if you want), and see how things go.
That said, I already know from experience that some people are not going to hear/notice small difference at first after only one sitting. For some people it can take months or maybe around a year of starting to learn to notice more and more little details which were always there, just unnoticed. Of course, this always has to focus on discrimination and not on preference. Preference can be discussed to, but the skill is in developing discrimination for small differences (without fooling oneself too often along the way with vague imagined differences, which may come in part from trying to progress too far too fast).
Anyone else have any experiences like that?
Last edited:
My opinion (and I dunno if the science validate it) is : senses can be fooled somewhat as the acuracy is limited. But the skill acquired makes the discrimination easier to percieve. Where or when it is good enough is for a part cultural in our hobby, while a musician can tell we are far of the good enough still.
Some thougths :
- everyone has not the same perception skill. An important part of it is training and cultural (so biased for that last item, more on that later:*)
- sound is complex enough to be not easy to judge/weigth objectivly from a people to another. I mean this is easier if you take two people and ask them if they further see the same color (more on this later:*).
- People that acquire skill by experience and training developp the items involved : if you prefer they neuronal scheme is involved by that experience and somehow "neuronal roads" are created making them more skillled. There is a big difference between a musician, an audiophile and John Doe beyond at the beginnning of their life, differences certainly exist in their combo ears system acquisition +brain treatment. I think we are aware that what we hear are electrical/chemical interaction i our brain and not the sound itself (by chance differences between us are littles)
- I am convinced than most of people are not aware of scientifical method and mix the measurement and reality when it comes to prove. Measurement of course is a prove and is objective, but most of the time what we measure is limited (can be more precise in a field than a human, but it is the whole we don't know maybe how to measure as to arrive to a genuine ranking.
- As we know also : hearing beyond the cultural part of it, is adaptive (more on this later:*). We all have experienced the compression of spl and dynamic operated by the brain after few minutes of music (after some times the spl seems less that at the beginning when we playback the first track of an album.
*, the more on that later : We are aware of the bias that is eated here and there at all the sauces, often as easy dialectic. WHen science investigate about our senses, brain treatment and experiments are showing perception is a mix of external signals acquisation mixed with souvenirs database and treated as a whole (the well know Cave myth from Plato). There were for illustration tests on how the eyes can be fooled with colors : for instance a white apple can be seen as red by the tester not knowing it : its brain added the color to make it valid in relation to the experince the tester acquired in his life/culture.
A little like a Nose with perfums, there is a judgment of the feeled experience : this odor is like vanilla or s...t as an aproxiilation (the cultural part of you has a part of the answer and judge)
We know so, perception has a non negligible cultural part : japoneses for illustration will be more sensible to some timbres than others.
So the brain is sensible to the acquired skill by experience and repetition but this skill in the treatment is not virgin in relation to our cultural construction. I believe and maybe I am wrong, than skills in our hobby is superior to the bias than beginners in their musical journey are experincing when young enough (loss of our ears mic sensivity is a problem) VS the all bias brigade department whom are saying you never know, only THD-N is the verity. But when I take someone not involved in music and listening, a weird thing happen : they most of the time choose grossly the same unit/conf. What is strange to me is the experience is involved in the result as not trained people migth not discern some differences. Maybe their part of the brain that reconstruct the lived experience is adding a more important weigth of past experience than the trained people : aka this white apple is seen as red but actually it is white !
So when measuring, I am not sure what side of my brain has the last word : it could be the eyes proves of the datas are not enough but makes more sense to me because this is how I culturaly was teached : can be a bias too if the protocol measurement is not good enough. I.e you have datas but it is maybe not the most importants to make the experiment usefull despite the repeated results.
Same with art : for instance, spectography in the Louvre dpt is able to see much more colors than the eyes, whatever it is in a black and white only : sampling of the machine seems superior to the eyes but the whole picture is needed. See for instance hyperealism aerograph movt : seems like a photograp but is not.
Some thougths :
- everyone has not the same perception skill. An important part of it is training and cultural (so biased for that last item, more on that later:*)
- sound is complex enough to be not easy to judge/weigth objectivly from a people to another. I mean this is easier if you take two people and ask them if they further see the same color (more on this later:*).
- People that acquire skill by experience and training developp the items involved : if you prefer they neuronal scheme is involved by that experience and somehow "neuronal roads" are created making them more skillled. There is a big difference between a musician, an audiophile and John Doe beyond at the beginnning of their life, differences certainly exist in their combo ears system acquisition +brain treatment. I think we are aware that what we hear are electrical/chemical interaction i our brain and not the sound itself (by chance differences between us are littles)
- I am convinced than most of people are not aware of scientifical method and mix the measurement and reality when it comes to prove. Measurement of course is a prove and is objective, but most of the time what we measure is limited (can be more precise in a field than a human, but it is the whole we don't know maybe how to measure as to arrive to a genuine ranking.
- As we know also : hearing beyond the cultural part of it, is adaptive (more on this later:*). We all have experienced the compression of spl and dynamic operated by the brain after few minutes of music (after some times the spl seems less that at the beginning when we playback the first track of an album.
*, the more on that later : We are aware of the bias that is eated here and there at all the sauces, often as easy dialectic. WHen science investigate about our senses, brain treatment and experiments are showing perception is a mix of external signals acquisation mixed with souvenirs database and treated as a whole (the well know Cave myth from Plato). There were for illustration tests on how the eyes can be fooled with colors : for instance a white apple can be seen as red by the tester not knowing it : its brain added the color to make it valid in relation to the experince the tester acquired in his life/culture.
A little like a Nose with perfums, there is a judgment of the feeled experience : this odor is like vanilla or s...t as an aproxiilation (the cultural part of you has a part of the answer and judge)
We know so, perception has a non negligible cultural part : japoneses for illustration will be more sensible to some timbres than others.
So the brain is sensible to the acquired skill by experience and repetition but this skill in the treatment is not virgin in relation to our cultural construction. I believe and maybe I am wrong, than skills in our hobby is superior to the bias than beginners in their musical journey are experincing when young enough (loss of our ears mic sensivity is a problem) VS the all bias brigade department whom are saying you never know, only THD-N is the verity. But when I take someone not involved in music and listening, a weird thing happen : they most of the time choose grossly the same unit/conf. What is strange to me is the experience is involved in the result as not trained people migth not discern some differences. Maybe their part of the brain that reconstruct the lived experience is adding a more important weigth of past experience than the trained people : aka this white apple is seen as red but actually it is white !
So when measuring, I am not sure what side of my brain has the last word : it could be the eyes proves of the datas are not enough but makes more sense to me because this is how I culturaly was teached : can be a bias too if the protocol measurement is not good enough. I.e you have datas but it is maybe not the most importants to make the experiment usefull despite the repeated results.
Same with art : for instance, spectography in the Louvre dpt is able to see much more colors than the eyes, whatever it is in a black and white only : sampling of the machine seems superior to the eyes but the whole picture is needed. See for instance hyperealism aerograph movt : seems like a photograp but is not.