Listening tests

Status
This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.
How do you all who are trying out various components/designs/modifications/whatever organize your listening tests? Presumably you can't afford to build multiple versions of everything to do side-by-side comparisons, so you have to rely on your memory of previous versions. How can you know with confidence that version B sounds slightly better or worse than version A, which you last heard a week ago? Is this just a skill that comes with practice? Or are there tricks you use to help bring the comparison into focus?
 
0.a Be very well aware that looks, in principle, fool and fool more than you might expect
0.b Be very well aware that your physical and phychological state varies more than you might expect
0.c Be aware of room trouble ; every room has a first and second base frequency

1. Use tracks you know very well
2. Never compare just A|B, but at least A|B|A
3. Try to understand a possible improvement; as we are DIYers -> open up the amp, think about the speaker design, etc
4. Anyone else?
 
Hi Rob, and welcome to the Forum.

I think that generally DIY'ers are not very scientific over such A/B testing.
In my case, I have tried changes to one channel only, but that only gives a vague picture of what's happening.
More often, I have come to notice an artifact, possibly only on one recording at first, but then when I listen more carefully, I can hear it on other things. Then I try to work out what is happening, and to find a cure. It's sometimes possible to switch back quicky to check, other times not.
Overall, there are enough variables to ensure that no one need agree what is right.:rolleyes: In the end, much comes down to personal preferences.

Cheers,
 
Hi,

A brief reply for now, as its time for bed here, but I will try and expand on it later if no-one else beats me to it.

There are some situations where it cannot be done this following way for the sake of practicality, but I would always recommend that you try to arrange it so that you can make immediate and multiple comparisons.

Generally, relying on memory, even say the day before, is not a good way to do this effectively.

Too many variables can occur and may change in the meantime and our memory is not too good for some of these details anyway.

Your own physical state, your frame of mind, whether you are tired, if you have had a drink etc., will all affect the matter, as will the state of your local mains electricity and atmospheric pressure etc.

The great good fortune with stereo is that you have two (should be!) identical channels to listen to, and this is also a help when troubleshooting, because then you can make some comparisons between the two channels and see what is different in the bad channel.

With a bit of practice, you will find it much more easy than you might originally think, because all you are looking for is any *differences* in the sound between the two channels, when you switch back and forth.

However, do bear in mind that no two speakers I have ever listened to are *exactly* alike, nor any two channels of an amp, or whatever, even in my own gear where every individual component is carefully hand- picked for similar values etc.

There will always be some, albeit minor, differences.

Therefore if the choices under consideration are very close, it may be wise before finalising any decision, to swap equipment around i.e. the speakers, or amp or preamp channels and interconnects etc., in order to reduce any outside variables.

If you can arrange it, I found it best to set up a test situation where you can make an immediate changeover several times between two different choices.

So set up two speakers side by side in the middle of a room, if possible, to avoid many unbalanced room reflections from affecting the issue too much, and try the change(s) to only one channel.

Only ever make one change (or type of change) at a time, as otherwise you will not know which was responsible for any differences, or if the two alterations (or more) are additive or subtractive. i.e. tending to cancel each other out.

Ideally, it is best to have a mono source, although with practice you will become so familiar with certain test material, you will probably know how each channel differs from the other without this, but mono sound is the easiest.

If your preamp doesn't provide for this, you need to make up a lead which effectively joins the two channels together, electrically, but if you are into this kind of tweaking you will probably know how to do this, anyway

Obviously the earlier on in the chain you can effect this 'interconnection' the more opportunities you have to do the comparison, as you can only compare items which are further down the chain (i.e. nearer the speakers) than where the signal becomes mono.

I have never tried it but I guess (if they are not commercially available) it is possible to burn your own mono CD which would be even better if you use CDs, but as I like vinyl for preference, I have several older mono records which I use. These two ways are really the best as you can compare (almost) everything in the entire audio chain from CD player or cartridge onwards.


If say it is just a new interconnect you wish to try, plug it in one side only, and listen for any apparent differences, whilst switching to and fro between the two channels. The time for each session will vary from a few seconds to a few minutes maybe, but don't go for too long, as you will not recall the sound so well.

I try to look out for specific notes in a tune or phrases in a song and then try the same notes or phrases on the other side. If you can set your CD to repeat a short section this would be a good way of doing this. Keep going back and forth until you can recognise the sounds well, and it will come with practice probably more easily than you may think.

However, if you are not a patient person, find another way to spend your time, as it does require a lot of dedication and perseverance to get good at this.

Incidentally you do need to be able to switch somehow from one channel to the other, but I guess most preamps will permit this.
If yours will not, you will need to insert a suitable switch in both channels to allow it, and the best quality component here will make the job more effective.

If say it is a new resistor to try out, it is often easier if you can exaggerate the situation by say changing several resistors in one channel at once to the 'new' types, as the cumulative effect will accentuate any changes, so they will be more easy to pick up.

When listening, you cannot take any account of the stereo soundstage this way of course, as you only have one channel to listen to at one time, but almost every other artifact can be readily compared, even including front to back soundfield.

I listen to the spectrum from top to bottom first, then consider attack, and how the whole sound gells together and the sweetness of sibilant sounds etc, in fact everything about the sound you can think of. With practice, you may not even need to consider each part separately, but a listen to the music as a whole will show you what you wish to know.

Personally, I use vocals a lot, as we are usually more familiar with these as we hear voices every day, whereas I only go to live concerts a few times a year now (live music must always be the true yardstick, of course), and some male voices cover a surprisingly large proportion of the normal audio spectrum.

Mainly, especially for higher frequencies and sibilant sounds, I like female voices the best as they are generally clearer and will accentuate any abberations more readily.

In time you will find certain recordings which show these areas up to the maximum, for example if you are interested in the bass spectrum, find something with perhaps a bowed double bass for the true rich 'colour' of the sound of a rosined bow on the strings, and maybe drums for attack etc.

I really must get off to bed now as it is after midnight and there will probably be many matters I haven't covered here, as I have only had time to scratch the surface of this topic. I have been doing it for 30 years now, and, accordingly it is second nature to me, so it is hard to immediately recall all the steps I take but I will think some more about it and will post these further thoughts if you wish, and if you consider they are of help to you.

Don't hesitate to ask any questions, and I will be pleased to answer them soon. I don't believe there are any instructions for these sort of trials, or at least, I have never seen any anywhere, so I am not suggesting that this is necessarily the best or only methodology to adopt.

It is, however the means of tackling a fascinating and very rewarding subject, which I have developed for myself over the years, and it works well for me. I hope some others will add a lot to this, and I will hopefully learn a few more tricks, myself! :nod:

Regards,
 
diyAudio Senior Member
Joined 2002
LISTENING TO AIR.

Hi,

Bob, your writing is just too long.. I got bored halfway. So maybe next time a truly brief version?

While it never bores me to read Bobs' scribblings...O.K.,I'll keep it short.

Much depends of course on what it is you want to compare.

For evaluating passive components such as resistors,caps,wire etc.,I always use my trusty Stax Lambda Pro headset.

As Bob suggested,keep the A/B test as short as you feel is needed.

Most people discern audible differences within a second or two,usually I spot them immediately.

The more you do this the more you'll train your hearing to pick up any differences.

While I agree with Bob that most commercial stuff such as amps and speakers usually are not matched between channels,they should be!

To me at least this is absolutely necessary to obtain a realistic stereo image.

If I were to spend,say,5K $ on an MC cartridge and it has a millivolt of channel inbalance it gets returned to the maker.
I take both my money and my music seriously as you can tell.

There's still a lot that can be said on hw to evaluate things here and Nielsio and Bob gave some very good pointers.

Cheers,;)
 
Learning Curve....

"As Bob suggested, keep the A/B test as short as you feel is needed.
Most people discern audible differences within a second or two, usually I spot them immediately.
The more you do this the more you'll train your hearing to pick up any differences"


Bob M,
Frank is perfectly correct here.
This is a learned skill, and it takes time and a million A/B/A/B/..... comparisons to fully develop these hearing skills, and once you have this skill you keep it forever.
Another skill is to be able to self turn off this listening mode and just kick back on the couch and enjoy the music regardless of system faults.
A million A/B comparos of all sorts of mods/improvements allows you to more easily identify and home in on and correct system faults that you hear.
This is a long learning process, and indeed required in the process of finding your own audio nivana. ( I have - at last).

As an example, on the weekend I tried an audio improvement on a friends PA system - mics, guitars, instruments, amplifiers and power feeds.
A colleague and I listened to the first session with the system
untreated, and modified the sytem during the first break.
On the first guitar note in the second session we both looked at each other an went 'Yeaaah', and 15 seconds later on the first vocal syllable we looked at each other again and both went 'YEAAAAAAAH, WOW'.

With practice, you can do this too.

Eric / - never tires of listening to good sounds.
 
I tried listening to one speaker only, but found out that listening to stereo is way easier to determine any differences and new sonic signature.

I use some favourite tracks, which over the years I got accustomed to and if there are any differences, I know it right away.

I used to do A/B switching in the beginning, but not anymore. Now, I just play a fragment of a known recording, try to memorise it, and then play it again after a change is done. I trained myself on countless occasions so for me it's almost a second nature.;)

I also know very well the sound of my main system, so if something changes, I can feel it: it just doesn't seem right.

There are certain days when the system is producing magic, but there are also days when I'm curious what makes it to sound so dull and boring on those other days. Since nothing changed, it must be me.;)
 
diyAudio Senior Member
Joined 2002
DON'T BLAME YOURSELF YET...

Hi,

Since nothing changed, it must be me.

Well,it could be you...it could also be the mains supply though.

We're all different of course but one thing I know is that if I've had a really stressfull day than there's no point in me firing up the system...I always end up NOT enjoying it wondering what's wrong.
No matter what I do to make it sound better then is just a waste of time and effort...better to relax first and wait 'till way past midnight and lo and behold there's the magic again.

Cheers,;)
 
Hi Frank, Peter,and Eric,

I couldn't agree more with what you say here, and as I already said, there is lot more I can add to what I said last time.

However, as the self-appointed spokesperson for this thread who has a short attention span, has already said that I am boring, and as presumably he doesn't yet know how to add me to his 'ignore' list, I will desist.

I don't use the more involved setup I outlined most of the time now, as I also don't find it necessary, except in certain rather critical situations.

However, putting myself in the shoes of someone who apparently has no previous experience of this subject, I feel that my suggestion will be the quickest, and most positive, way of identifying or experiencing these differences. One can keep on switching back and forth until any differences are noticed, whereas for a novice merely listening to some music and then listening again after a mod which could take from a few minutes to some hours, I am not so sure it would be very easy. :goodbad:

Of course, as with a rattle in one's car, once noticed, it is not then easily forgotten, and it is then very much easier to take some shortcuts and save a lot of time in setting things up.

Regards,
 
Bobken said:
Hi Frank, Peter,and Eric,

I couldn't agree more with what you say here, and as I already said, there is lot more I can add to what I said last time.

However, as the self-appointed spokesperson for this thread who has a short attention span, has already said that I am boring, and as presumably he doesn't yet know how to add me to his 'ignore' list, I will desist.

Don't let me stop you from writing another 1329 words (your first post, covering 3 (!) A4 pages when copy-pasted into Word).

Niels.
 
Hi Nielsio,

Is it possible for you to comprehend that I was merely trying to be helpful to someone else?

We all have our limitations, and I expect you could write a manual for a brain surgeon in just two lines, but, regrettably, I am not so clever as you appear to be.

I am sorry if you find my comments boring but they were directed towards the original enquirer, and I didn't see any need for your unwarranted intervention, which could hardly have been any less polite.


Regards,:)
 
Let me rephrase my original critique:

If you can find a way to bundle, summarize or shorten your advice, then the reader will have easier access to that information.

If after that, something is not clear, then people can ask for a further explanation or examplement on that something.

I am sorry if you find my comments boring but they were directed towards the original enquirer, and I didn't see any need for your unwarranted intervention, which could hardly have been any less polite.

Before submitting, I thought of writing a more polite version, but I chose not to because the situation seemed obvious.

Bob, your writing is just too long.. I got bored halfway. So maybe next time a truly brief version?

The situation being that you wrote an awful lot of text, and my remark being somewhat sarcastic (note 'a truly brief version', referring to your 'A brief reply for now, as ..').
 
Rob M said:
How do you all who are trying out various components/designs/modifications/whatever organize your listening tests? .... How can you know with confidence that version B sounds slightly better or worse than version A, which you last heard a week ago?

Almost no-one in the DIY audio or high end audio community does any kind of valid listening test. That explains some of the remarkably silly things you can read here. The simplest comparison of input versus output of devices can be quite illuminating.

A valid listening test is one which isolates the variable being listened for. I went through a phase where I did dozens and dozens of ABX type tests (one approach to valid subjective testing), where the listener had complete control of switching, timing, source material, volume, and reference to known A and B. These tests are a pain to set up, since level matching between tow devices to less than 0.1 dB (total, not plus/minus) takes a lot of effort and more patience than the fashion crowd normally has.

A year or two of that was enough to convince me that all audible differences were reflected in non-exotic measurement differences- e.g., frequency response, level, distortion. The jaw-dropping night-and-day stuff my friends told me about when using the lastest fashion accessories seemed to disappear when the variables of expectation, desire, and unconcious bias were removed.

Nowadays, I mostly do open A-B testing for my own stuff, sometimes single channel, sometimes against a reference piece of equipment. If I think I hear a difference, I'll blind it in a simple A-B and see if I can still hear it. If I can, then I go do more measurement to track down what I'm hearing.

In my professional work these days, I do a lot of sensory research. The idea that audio alone is immune to valid subjective testing is sheer voodoo nonsense.
 
Re: Re: Listening tests

SY said:


The idea that audio alone is immune to valid subjective testing is sheer voodoo nonsense.

It's almost like you are trying to say that you and me are hearing the same recording exactly the same way. What about emotional content and one's state of mind in a given moment? Those things are not so easy to measure and they vary all the time.
 
Re: Re: Re: Listening tests

Peter Daniel said:


It's almost like you are trying to say that you and me are hearing the same recording exactly the same way. What about emotional content and one's state of mind in a given moment? Those things are not so easy to measure and they vary all the time.

You are missing his point.

A valid listening test is one which isolates the variable being listened for
&
The jaw-dropping night-and-day stuff my friends told me about when using the lastest fashion accessories seemed to disappear when the variables of expectation, desire, and unconcious bias were removed

So: SY says that these ..

0.a Be very well aware that looks, in principle, fool and fool more than you might expect
0.b Be very well aware that your physical and phychological state varies more than you might expect

.. are too powerfull to come to a trustworthy conclusion and he has therefore reached out to measurement which, in his case, has proven to work.
 
Re: Re: Re: Listening tests

Peter Daniel said:


It's almost like you are trying to say that you and me are hearing the same recording exactly the same way.

That doesn't even vaguely make sense.

What about emotional content and one's state of mind in a given moment? Those things are not so easy to measure and they vary all the time.

These factors do indeed vary, and that's the whole point of eliminating them as variables.
 
Status
This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.