Evaluating
Level matching is critical for inexperienced listeners.
It's not so critical if you have experience in making comparisons and evaluating systems, cdps, amps, whatever.
You need to listen to the tune and not the gear.
You need to understand the music, follow every instrument and the whole.
The device where you find more difficult to understand the music and what every musician is playing is inferior.
Basically, what I'm saying is expressed very well here:
http://www.linn.co.uk/buy_linn/how_to_judge_a_system.cfm
I find this simple method quite efficient for evaluating and detecting differences.
With this method, sometimes you detect an instrument that was maskerated and unrecognizable on the other gear (amp or whatever).
One of the secrets is to pick just 30~40 seconds of a music and no more. Concentrate on that part.
Change the gear and listen again.
But concentrate on the music.
You will memorize more easily.
I can tell you a little secret: an excellent track to test a system is on Jeff Buckley's 'Grace' cd, track 6 - 'Hallelujah'.
Listen to that track with two different cdps, follow the music.
Stop a while after the guy begins to sing.
Repeat the test.
Concentrate.
How many instruments are there on that music?
Only a good cdp will make it clear!
And I'm talking of a not so good recording.
But listen to the music!
Level matching is critical for inexperienced listeners.
It's not so critical if you have experience in making comparisons and evaluating systems, cdps, amps, whatever.
You need to listen to the tune and not the gear.
You need to understand the music, follow every instrument and the whole.
The device where you find more difficult to understand the music and what every musician is playing is inferior.
Basically, what I'm saying is expressed very well here:
http://www.linn.co.uk/buy_linn/how_to_judge_a_system.cfm
I find this simple method quite efficient for evaluating and detecting differences.
With this method, sometimes you detect an instrument that was maskerated and unrecognizable on the other gear (amp or whatever).
One of the secrets is to pick just 30~40 seconds of a music and no more. Concentrate on that part.
Change the gear and listen again.
But concentrate on the music.
You will memorize more easily.
I can tell you a little secret: an excellent track to test a system is on Jeff Buckley's 'Grace' cd, track 6 - 'Hallelujah'.
Listen to that track with two different cdps, follow the music.
Stop a while after the guy begins to sing.
Repeat the test.
Concentrate.
How many instruments are there on that music?
Only a good cdp will make it clear!
And I'm talking of a not so good recording.
But listen to the music!
Resolution of the volume control is a big part of it. So is backlash and drift. Normally, to get a level match better than 0.1dB between sources, you have to either use a precision attenuator or an accurately trimmed voltage divider. Either way, it's tedious and a real hassle. But if you don't do that, you're not making any kind of valid comparison, you're just fooling around; tiny level differences are NOT perceived as such by our ear-brain system.
Now, I understand that there's a lot of fun-value and enjoyment in just fooling around, but it doesn't get you to any kind of real understanding.
Now, I understand that there's a lot of fun-value and enjoyment in just fooling around, but it doesn't get you to any kind of real understanding.
I believe I’ve seen the claim that double blind testing can resolve 0.1 dB level differences that exceed an octave in frequency range – so a full frequency response comparison is the only “sound” approach
Today a full frequency response comparison isn’t as daunting as it might seem, almost any PC soundcard will have the resolution (and probably the stability/repeatability) to make the comparison with freeware spectrum analysis sw such as RMA
Today a full frequency response comparison isn’t as daunting as it might seem, almost any PC soundcard will have the resolution (and probably the stability/repeatability) to make the comparison with freeware spectrum analysis sw such as RMA
I understand level matching is critical when one does A/B switching playing music continuously. I also agree that's the way for inexperience listeners.
However, I never do evaluation in that way. I rely on my memory of the presentation and I can easily decide which one is better and why (for me of course). In that case I have never found level matching critical.
However, I never do evaluation in that way. I rely on my memory of the presentation and I can easily decide which one is better and why (for me of course). In that case I have never found level matching critical.
This is explained quite clearly in the literature, especilly in the work of Lipshitz and Vanderkooy; it is well-established that minor relative level differences are not perceived by humans as level differences. That's true whether the humans are experienced or inexperienced, members of the "engineering community" or some other occupation.
Now, this is very different that talking about absolute levels; listeners should be able to vary them at will. But the relative levels of the two DUTs must be tightly matched. And again, we're not talking about preference, we're talking about ability to distinguish one DUT from another.
Peter Daniel said:
Peter,
Please don't take this personally, but as SY stated, it is quite established that even minor level differences are not perceived as such. They are instead perceived as qualitative differences. That means, if you don't match levels, you may quite perceive qualitative differences which lead to preferences which are based in fact on just plain level differences.
Now, I understand that that is very difficult to accept. After all, you CLEARLY hear a difference in quality, or let's say preference. But you must grow to the next higher level of understanding perception and understanding your own mental and sensory makeup and how this puts up a perceptive landscape in your mind.
Until that time, your preferences may be based on something that disappears when the situation is slightly modified. If, as can be the case, you preference is based on slight level differences in a comparison with another amp, than what is it worth really?
Jan Didden
SY said:
This, like the level issues relates to modes of human perception that can lead us astray, unless we are fully aware of them and compensate properly. It's a very interesting subject and it's nice to see such discussions. I have a more mundane question regarding tje test subject hearing (perceiving?) differences in amplifiers that do not correlate with the usual instrument tests; could some of that be due to the fact that the instrument tests are done with simple resistive loads (I don't know how they are actually done), while listening tests are necessarily done with reactive loads?
Sheldon
Sheldon said:
Doug Self measured the distortion of his amplifier with a loudspeaker load. The high - frequency distortion reduced compared to a resistive load, because the impedence increased due to coil inductance, and distortion reduces as load impedence increases. I intend to try this myself.
I take your points regarding level differences. So those CD players could actually sound the same, and two tuners which I thought sounded the same except that one was louder than the other, were actually different.
Did they ask the right questions?
I think asking consumers what they think about the product is a good idea but how the questions get asked, and what they are, is awfully important. Otherwise, we get results like those Japanese had.
Before we do so,we have to examine our assumptions and what we already know.
The division between "objectivists" and "subjectivists" is illegiitimate, in my view, because people can hear reproduced sound they don't like or get tired of very quickly. They can't describe, usually, what's wrong about the sound but they dislike it.
Some research suggests that whats's wrong with reproduced sound folk generally don't like involves parameters the audio industry and audio researchers haven't generally examined in a rigorous fashion, and those are the correlations between distortion products and human perception.
Read, for instance, the work done by Earl Geddes and Lydia Lee:
http://www.gedlee.com/distortion_perception.htm
Look particularly at the AES and SAE presentation and the results.
A small experiment we could undertake for ourselves to subjectively examine the masking effect is to take a reasonably flat audio system and and play around with equalization between 2 and 5 kHz while playing big orchestral music at realistic levels. A bit of a dip in the reponse, particularly @ 4000 Hz, makes for for a for more musical experience because our sensitivity to these frequencies is greater - the BBC fell on this years ago.
The Fletcher Munsen curves tell us either our total system response should follow, or that the recording systems should follow, the general shape of the F-M curves.
http://www2.sfu.ca/sonic-studio/handbook/Equal_Loudness_Contours.html
I think asking consumers what they think about the product is a good idea but how the questions get asked, and what they are, is awfully important. Otherwise, we get results like those Japanese had.
Before we do so,we have to examine our assumptions and what we already know.
The division between "objectivists" and "subjectivists" is illegiitimate, in my view, because people can hear reproduced sound they don't like or get tired of very quickly. They can't describe, usually, what's wrong about the sound but they dislike it.
Some research suggests that whats's wrong with reproduced sound folk generally don't like involves parameters the audio industry and audio researchers haven't generally examined in a rigorous fashion, and those are the correlations between distortion products and human perception.
Read, for instance, the work done by Earl Geddes and Lydia Lee:
http://www.gedlee.com/distortion_perception.htm
Look particularly at the AES and SAE presentation and the results.
A small experiment we could undertake for ourselves to subjectively examine the masking effect is to take a reasonably flat audio system and and play around with equalization between 2 and 5 kHz while playing big orchestral music at realistic levels. A bit of a dip in the reponse, particularly @ 4000 Hz, makes for for a for more musical experience because our sensitivity to these frequencies is greater - the BBC fell on this years ago.
The Fletcher Munsen curves tell us either our total system response should follow, or that the recording systems should follow, the general shape of the F-M curves.
http://www2.sfu.ca/sonic-studio/handbook/Equal_Loudness_Contours.html
janneman said:
It would be very nice if you could adjust your preferences by changing volume level. Let's say a piece of equipment sucks at 1/4 turn of the knob, but incresing volume 2 notches up cures the problem. Unfortunately, if something sucks, it sucks no matter what the level is.
I am not talking about immediate comparisons, as I don't even bother. I prefer to move to the next level of using my perception: listening to the sound and comparing it with the reference in my head: my image and the memory of the live acoustic sound. When I say original acoustic sound, I'm talking about my perception or image of that sound. In this context, it greatly depends on my taste and senses, which are the result of my musical experience.
Peter,
There is NO memory in your head. What happens is that your brain tries to re-create the neuronal connectivity between your various "maps" in the brain to get as close as possible to the connectivity that existed when you did the original listening. At a certain point, the brain signals: yes, this is it, this is how it was at that time, so this is how it sounded. Needless to say, it is inaccurate in the extreme. Relying on this kind of 'memory' to judge differences between equipment is a quite futile untertaking.
Let me try to give an exampole from visual perception. If I look out of my window, I see a few nice trees in the sun. Now suppose tomorrow, when I come back again, someone asks me: are those trees still there, and are they the same ones you saw yesterday? I would look and probably say yes, they are still there, they look like they are the same trees. But suppose someone had come by this night and clipped a few branches here and there, ripped off a bunch of leaves, etc. Do you think I would notice? Surely not, unless it was a significant change, because that was not in the detail that was stored in my memory system.
It is similar with sound perception. Subtle differences between say amplifiers are at the level of 'leaves' and for the life of you there's no way you can remember that even after a relative short time. And the visual system is vastly better in this respect than the auditory system.
Jan Didden
There is NO memory in your head. What happens is that your brain tries to re-create the neuronal connectivity between your various "maps" in the brain to get as close as possible to the connectivity that existed when you did the original listening. At a certain point, the brain signals: yes, this is it, this is how it was at that time, so this is how it sounded. Needless to say, it is inaccurate in the extreme. Relying on this kind of 'memory' to judge differences between equipment is a quite futile untertaking.
Let me try to give an exampole from visual perception. If I look out of my window, I see a few nice trees in the sun. Now suppose tomorrow, when I come back again, someone asks me: are those trees still there, and are they the same ones you saw yesterday? I would look and probably say yes, they are still there, they look like they are the same trees. But suppose someone had come by this night and clipped a few branches here and there, ripped off a bunch of leaves, etc. Do you think I would notice? Surely not, unless it was a significant change, because that was not in the detail that was stored in my memory system.
It is similar with sound perception. Subtle differences between say amplifiers are at the level of 'leaves' and for the life of you there's no way you can remember that even after a relative short time. And the visual system is vastly better in this respect than the auditory system.
Jan Didden
Re: Did they ask the right questions?
As I recall the article, what the Japanese manufacturer did was asking people to indicate their preference order for several speakers. These preferences were tabulated by a computer to get at an overall ranking.
Remember these are Joe and Jane Listener. If you go to more structured tests, asking specific questions aimed at getting to the differences between speakers, you are already gyrating towards a 'scientific' or 'objective' test.
Some people tell me I shouldn't use measurements or scientific tests because I don't know what to measure to correlate with sound quality. Now we see that using listeners to develop a product leads nowhere. So, what should I do next??
Jan Didden
FrankWW said:
As I recall the article, what the Japanese manufacturer did was asking people to indicate their preference order for several speakers. These preferences were tabulated by a computer to get at an overall ranking.
Remember these are Joe and Jane Listener. If you go to more structured tests, asking specific questions aimed at getting to the differences between speakers, you are already gyrating towards a 'scientific' or 'objective' test.
Some people tell me I shouldn't use measurements or scientific tests because I don't know what to measure to correlate with sound quality. Now we see that using listeners to develop a product leads nowhere. So, what should I do next??
Jan Didden
This discussion reminds me the old thread "blind listening tests and amps"... and it didn't end well, because there are several approaches to this.
In my oppinion and in my experience, independent on the volume, I can detect subtle details from one cdp to another one, independent on the volume setting.
It would be good that a bad cdp would match a good one just playing louder.
The fact is, with the better source or amp or whatever I listen more things, I listen clearly each instrument, I can detect sounds that were not perceptible.
And I like to play with the volume , I don't like that someone adjusts the volume level to a fixed setting at his will, I like to be free to listen as I like.
Don't take me wrong, but that method is for a pure A/B test, made for inexperienced listeners.
Switching back and forth doesn't let you concentrate on the music, and it's clearly not the best way to evaluate a system.
I'm sorry, I just have a different oppinion, I just hope this thread doesn't end like the other one.
In my oppinion and in my experience, independent on the volume, I can detect subtle details from one cdp to another one, independent on the volume setting.
It would be good that a bad cdp would match a good one just playing louder.
The fact is, with the better source or amp or whatever I listen more things, I listen clearly each instrument, I can detect sounds that were not perceptible.
And I like to play with the volume , I don't like that someone adjusts the volume level to a fixed setting at his will, I like to be free to listen as I like.
Don't take me wrong, but that method is for a pure A/B test, made for inexperienced listeners.
Switching back and forth doesn't let you concentrate on the music, and it's clearly not the best way to evaluate a system.
I'm sorry, I just have a different oppinion, I just hope this thread doesn't end like the other one.
janneman said:
... maybe your memory system is as poor as mine. Nevertheless, if the tree had one apple, shining red color in contrast with remaining parts, but disappeared this morning, I might notice it. However, it doesn't matter. The whole tree is still there. No difference. Only thing is that the tree gives me somewhat different look and feeling, depending on outside weather condition.
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.