I thought you used Blue for, well, "My woman jus dun lef' me" stuff ... and Green for that cloying ambient crap ...
This was discussed earlier on by several people with experience setting up sensory tests (that's a very different skill than circuit design!).
Several good ways to set up the test where the effects of presentation order error, difference bias, and group dynamics can be minimized:
1. ABX forced choice. Yes, you can do that, but it's a lot of work on your part; doing good testing isn't easy. Record A, record B, and post them as references. Using a random number generator (this is EXTREMELY important, humans suck at randomizing), determine a pattern of X for 12 repetitions. For each of the twelve, do a recording with either bypass or opamp. Have people submit choices. So there's 14 separate recordings you need to do.
2. Triangle forced choice. Even more work, but even more significant. Similar to ABX, but in this case, you have for each trial three choices, two of which are the same (again, randomized by a random number generator). The listener picks the "different" file for each trial. You can have fewer trials to achieve significance, but you'll have a lot more recordings to do! If you want to add some power to it, add in two extra trials where all three are the same as a control.
3. Sorting. Random number generator time again. Now you'll have 12 presentations, each randomized between A and B, and the listener has to sort them into two "piles." There may or may not be 6 in each pile- in fact, for true random number generators, there is a higher probability that there won't be.
4. Paired preference. Use a random number generator. Now you'll have 12 pairs of files, where the listener each time chooses a preference. That's 24 recordings for you to do. Again, it's a good thing to slip in a couple pairs where there is no difference as a control. A variation is to have randomized pairs that are different or same. The statistics are a bit trickier since audiophiles will bias their results to "different."
I need not tell you that your files do have to be individually generated to prevent desk jockey gas bags from cheating via file analysis.
Last point: if you are doing a group test, you need to handle the statistics as a group test. That seems obvious, but there are some infamous examples where this was ignored. And even in this test, despite the lack of basic rigor, are the results significant?
This is all something of a pain and can be tedious for the guy setting up the test, but as with circuit design, if you want valid results, you have to put the work in.
Yes, I have already predicted it. That's why I wrote "HUUGE" difference between the two.
I have some reservations to 'scientific' methods of audibility testing. What we get as a result, for decades? Lossy compression, dynamic compression and information loss. And we are persuaded that the losses are inaudible, as the compression methods resulted from scientifically performed methods and are statistically valid. As a result, we have the worst sound ever, excluding classical music. These 'scientific' tests are, IMO, strictly market influenced. So what?
Last edited:
I guess it would be beneficial if PMA or Mooly published the listening impressions, so that the test has more significance, isn't it?
If we see WHY people prefered one file or the other we could have more indication whether the test showed real audible difference between blue and green or not.
(Two people with different preferences in sound reproduction could have voted differently, but still find the same characteristics in the sound of each file.)
If we see WHY people prefered one file or the other we could have more indication whether the test showed real audible difference between blue and green or not.
(Two people with different preferences in sound reproduction could have voted differently, but still find the same characteristics in the sound of each file.)
Three files are confusing to make a decision. It may work only in case the three files are very different, I believe it may work for lossy compression tests, when storage media cost is the argument and main reason for scientific testing.
If your assertion (without any cite or evidence) that "three files are confusing" is correct, then use one of the other valid methods. I mentioned two of them, there are others.
AB BA AA BB. Not a difficult concept, but as an experimenter, you have to be willing to do the work, not just put on the white coat and make believe.
AB BA AA BB. Not a difficult concept, but as an experimenter, you have to be willing to do the work, not just put on the white coat and make believe.
I think you'll all agree this has been a fascinating test.
These were some comments on the way things sounded... all names have been removed 🙂
---------------------------------------------------------
Fairly straightforward, only needed to use Nero player - no visuals to distract 🙂 ! Blue is quite superior, green sounds if it has has been processed in some way to remove sparkle and depth, though nominally the vocals sound slightly smoother - the latter has more a classic "hifi" sound.
There seems to be another subtlety, the quality of blue starts well, but degrades somewhat through the length of it, develops an edge by the end of the clip - this is a behaviour I've noted quite often with digital replay.
-----------------------------------------------------------
BLUE it is 🙂
-----------------------------------------------------------
No difference heard between tracks. Used Foobar 2000 to A/B the tracks on Senheisser HD555 headphones. Dell Inspiron I7 laptop (stock soundcard).
-----------------------------------------------------------
I'm not going to try much harder, but after a few comparisons, my preference is for green. Are the results going to be anonymous? 😀
Thanks!
-----------------------------------------------------------
I vote for green.
-----------------------------------------------------------
I can not decide between green and blue.
I don't really expect to be able to differentiate between systems with a similar level of quality unless the difference is something as unsubtle as a tone control set to one extreme or the other. These two are definitely a similar level of quality. I can't hear a tonal difference or a difference in distortion either but I'm prepared to believe that someone else (xyz123) might be able to.
In both of these samples (green and blue), the music is listenable and I can say with confidence that the quality of the chain and/or original used for green and blue is a hell of a lot better than the quality of the chain and/or original used for the previous fruit based op amp samples.
There is some obvious distortion in the male vocals which seems to be the same in both green and blue and might very well be an intentional part of the original or perhaps even the actual sound of his voice.
These two get an 8 out of 10 for me. They are smooth and listenable and I can't point out any obvious flaws but the music isn't to my taste, they don't really engage me and I didn't hear anything stunningly great in them.
It will be interesting to discover what the hardware differences are.
Personally, I found that eliminating all the op-amps from my audio (playback) chain was a major step forwards for my listening pleasure. I also don't really have to worry about which ones are the best any more 🙂
----------------------------------------------------------
My preference is the green file. The blue file has artifacts and more sibilance, especially in the passage
"...why is he standing...".
----------------------------------------------------------
blue
----------------------------------------------------------
Green
----------------------------------------------------------
Top SQ is "green".
Guessing in words:
"Green" is either direct clean signal, or chain has solid core cable connection (could be PCB kobber lanes), or it could be warm biased amp (class-A) maybe with green sound running through.
"Blue" have in chain either multistranded cablecore and/or amp with paralell devices, or a very short delay to hole soundstage is added from uS to say 2mS.
----------------------------------------------------------
Spoiler alert. Do not read further if you are still doing the test!
I have been trying to work on the waveform to see what might have been done.
I have thought of about 10 different possibilities but the latest thought was so cool I had to share:
What if one of the recordings was a soundcard digital loopback test (maybe involving a phase locked loop as a handicap for a bit more interest) and the other one was an analogue loopback test with the same soundcard.
I think that would qualify as a "not small" change in hardware and would give the low frequency phase differences from the high pass coupling capacitor in the analogue loop.
If not, and if anyone can reliably get the current test which, from looking at the waveform, looks pretty unlikely, I'd be tempted to try this on them next.
My next coolest possibility was just the introduction of a coupling capacitor into an analogue loop.
The other ideas were not so interesting.
-----------------------------------------------------------
Blue it is.
Green sounds out of phase.
My vote is blue.
-----------------------------------------------------------
I want to vote - but find it hard to decide. Both suck compared to my Gaucho CD. They are both muffled, crosstalk heavy and kinda phasey, Is there a "Both Suck" option? 🙁
Put me in the undecided camp, I suppose. Can't decide which is worse.
-----------------------------------------------------------
I would like to vote "I can not decide".
I would also like to predict that "blue" will win. Assuming that the number of people who can actually tell the difference is few or none, then people will base their choice on something else. In this case that could well be their favourite colour, which is statistically most likely to be blue.
-----------------------------------------------------------
Pretty difficult choosing a winner from my desk top ...
Blue ....
-----------------------------------------------------------
I cannot decide
-----------------------------------------------------------
I had another listen after the vote. Actually they don't suck, they just are lack-luster compared to the rip I have. Maybe it's a different master? Anyway, Blue sounds rougher on the top end, to me.
-----------------------------------------------------------
Hi Mooly,
I vote: green
-----------------------------------------------------------
cannot hear a difference
-----------------------------------------------------------
Green
-----------------------------------------------------------
Hi Mooly,
In responds to the ultimate listening test thread (with the blue and green testfiles), my choice for better sounding file is: green.
Most notible for me was the more spacious sound, sharper (sound)image and more dynamic/'snap' of the green.wave file.
Big thanks for setting up this very interesting blind test!
-----------------------------------------------------------
Hello! I've participated in the hearing test and I even used that plugin for Foobar2000 and I cannot hear any differences between those two files.
At first I thought maybe bass was different but I came to the conclusion that I just can't hear any difference.
I'm excited to know what the differences were and who got it right.
-----------------------------------------------------------
Green
-----------------------------------------------------------
FYI, this is quite interesting review ...
Blue :
Quite detailed, light, and a bit artificial and « Hi-Fi » sound, with separation between bass, midrange, and treble. The music feels a bit tensed and cool, voices sound a bit edgy, with a slight emphasis on the upper registers. Highs show a tendency to acidity. Good dynamics. Soundstage slightly restrained in width/depth.
Green:
More relaxed, and earthy sound. Voices sound more natural. Good reverb and decay. Soundstage quite expansive. Realistic reproduction of instruments harmonics.
My choice goes to green, as it feels more natural overall, without losing the details. But is it more true to the source? That I can’t say… I tend to find more micro-information in the green file, but that may well be psychoacoustics.
----------------------------------------------------------
I listenend to your BLUE/GREEN files and I would say GREEN sounds a little richer/fuller........
Is one of them stereo and the other MONO or is one done with analog equiptment and the other digital?? (Even though we are listening over digital stuff,you can hear a slight BETTER produxtion if analogue stuff was used)
----------------------------------------------------------
There are subtle differences but I can not reliably pick blue vs green
'I can not decide'
--------------------------------------------------------------------------------
When i voted for green i had some guessing which copyed here underneath. Since that with further thoghts green could be DC-coupled and blue AC-copled, or green with burned in caps and blue with non settled caps. Last i also think the differenc can be simple as green recorded with long buffer and blue recorded with short buffer.
Following is copy from first day guess.
Guessing in words:
"Green" is either direct clean signal, or chain has solid core cable connection (could be PCB kobber lanes), or it could be warm biased amp (class-A) maybe with green sound running through.
"Blue" have in chain either multistranded cablecore and/or amp with paralell devices, or a very short delay to hole soundstage is added from uS to say 2mS.
***********************The End***************************
These were some comments on the way things sounded... all names have been removed 🙂
---------------------------------------------------------
Fairly straightforward, only needed to use Nero player - no visuals to distract 🙂 ! Blue is quite superior, green sounds if it has has been processed in some way to remove sparkle and depth, though nominally the vocals sound slightly smoother - the latter has more a classic "hifi" sound.
There seems to be another subtlety, the quality of blue starts well, but degrades somewhat through the length of it, develops an edge by the end of the clip - this is a behaviour I've noted quite often with digital replay.
-----------------------------------------------------------
BLUE it is 🙂
-----------------------------------------------------------
No difference heard between tracks. Used Foobar 2000 to A/B the tracks on Senheisser HD555 headphones. Dell Inspiron I7 laptop (stock soundcard).
-----------------------------------------------------------
I'm not going to try much harder, but after a few comparisons, my preference is for green. Are the results going to be anonymous? 😀
Thanks!
-----------------------------------------------------------
I vote for green.
-----------------------------------------------------------
I can not decide between green and blue.
I don't really expect to be able to differentiate between systems with a similar level of quality unless the difference is something as unsubtle as a tone control set to one extreme or the other. These two are definitely a similar level of quality. I can't hear a tonal difference or a difference in distortion either but I'm prepared to believe that someone else (xyz123) might be able to.
In both of these samples (green and blue), the music is listenable and I can say with confidence that the quality of the chain and/or original used for green and blue is a hell of a lot better than the quality of the chain and/or original used for the previous fruit based op amp samples.
There is some obvious distortion in the male vocals which seems to be the same in both green and blue and might very well be an intentional part of the original or perhaps even the actual sound of his voice.
These two get an 8 out of 10 for me. They are smooth and listenable and I can't point out any obvious flaws but the music isn't to my taste, they don't really engage me and I didn't hear anything stunningly great in them.
It will be interesting to discover what the hardware differences are.
Personally, I found that eliminating all the op-amps from my audio (playback) chain was a major step forwards for my listening pleasure. I also don't really have to worry about which ones are the best any more 🙂
----------------------------------------------------------
My preference is the green file. The blue file has artifacts and more sibilance, especially in the passage
"...why is he standing...".
----------------------------------------------------------
blue
----------------------------------------------------------
Green
----------------------------------------------------------
Top SQ is "green".
Guessing in words:
"Green" is either direct clean signal, or chain has solid core cable connection (could be PCB kobber lanes), or it could be warm biased amp (class-A) maybe with green sound running through.
"Blue" have in chain either multistranded cablecore and/or amp with paralell devices, or a very short delay to hole soundstage is added from uS to say 2mS.
----------------------------------------------------------
Spoiler alert. Do not read further if you are still doing the test!
I have been trying to work on the waveform to see what might have been done.
I have thought of about 10 different possibilities but the latest thought was so cool I had to share:
What if one of the recordings was a soundcard digital loopback test (maybe involving a phase locked loop as a handicap for a bit more interest) and the other one was an analogue loopback test with the same soundcard.
I think that would qualify as a "not small" change in hardware and would give the low frequency phase differences from the high pass coupling capacitor in the analogue loop.
If not, and if anyone can reliably get the current test which, from looking at the waveform, looks pretty unlikely, I'd be tempted to try this on them next.
My next coolest possibility was just the introduction of a coupling capacitor into an analogue loop.
The other ideas were not so interesting.
-----------------------------------------------------------
Blue it is.
Green sounds out of phase.
My vote is blue.
-----------------------------------------------------------
I want to vote - but find it hard to decide. Both suck compared to my Gaucho CD. They are both muffled, crosstalk heavy and kinda phasey, Is there a "Both Suck" option? 🙁
Put me in the undecided camp, I suppose. Can't decide which is worse.
-----------------------------------------------------------
I would like to vote "I can not decide".
I would also like to predict that "blue" will win. Assuming that the number of people who can actually tell the difference is few or none, then people will base their choice on something else. In this case that could well be their favourite colour, which is statistically most likely to be blue.
-----------------------------------------------------------
Pretty difficult choosing a winner from my desk top ...
Blue ....
-----------------------------------------------------------
I cannot decide
-----------------------------------------------------------
I had another listen after the vote. Actually they don't suck, they just are lack-luster compared to the rip I have. Maybe it's a different master? Anyway, Blue sounds rougher on the top end, to me.
-----------------------------------------------------------
Hi Mooly,
I vote: green
-----------------------------------------------------------
cannot hear a difference
-----------------------------------------------------------
Green
-----------------------------------------------------------
Hi Mooly,
In responds to the ultimate listening test thread (with the blue and green testfiles), my choice for better sounding file is: green.
Most notible for me was the more spacious sound, sharper (sound)image and more dynamic/'snap' of the green.wave file.
Big thanks for setting up this very interesting blind test!
-----------------------------------------------------------
Hello! I've participated in the hearing test and I even used that plugin for Foobar2000 and I cannot hear any differences between those two files.
At first I thought maybe bass was different but I came to the conclusion that I just can't hear any difference.
I'm excited to know what the differences were and who got it right.
-----------------------------------------------------------
Green
-----------------------------------------------------------
FYI, this is quite interesting review ...
Blue :
Quite detailed, light, and a bit artificial and « Hi-Fi » sound, with separation between bass, midrange, and treble. The music feels a bit tensed and cool, voices sound a bit edgy, with a slight emphasis on the upper registers. Highs show a tendency to acidity. Good dynamics. Soundstage slightly restrained in width/depth.
Green:
More relaxed, and earthy sound. Voices sound more natural. Good reverb and decay. Soundstage quite expansive. Realistic reproduction of instruments harmonics.
My choice goes to green, as it feels more natural overall, without losing the details. But is it more true to the source? That I can’t say… I tend to find more micro-information in the green file, but that may well be psychoacoustics.
----------------------------------------------------------
I listenend to your BLUE/GREEN files and I would say GREEN sounds a little richer/fuller........
Is one of them stereo and the other MONO or is one done with analog equiptment and the other digital?? (Even though we are listening over digital stuff,you can hear a slight BETTER produxtion if analogue stuff was used)
----------------------------------------------------------
There are subtle differences but I can not reliably pick blue vs green
'I can not decide'
--------------------------------------------------------------------------------
When i voted for green i had some guessing which copyed here underneath. Since that with further thoghts green could be DC-coupled and blue AC-copled, or green with burned in caps and blue with non settled caps. Last i also think the differenc can be simple as green recorded with long buffer and blue recorded with short buffer.
Following is copy from first day guess.
Guessing in words:
"Green" is either direct clean signal, or chain has solid core cable connection (could be PCB kobber lanes), or it could be warm biased amp (class-A) maybe with green sound running through.
"Blue" have in chain either multistranded cablecore and/or amp with paralell devices, or a very short delay to hole soundstage is added from uS to say 2mS.
***********************The End***************************
You can look at it in another way. What if the sound coming out of the DAC is "harsher" (digital..?) than what we expect to hear coming out of e.g. a guitar amp. And somehow, coupled with the op-amp; it sounds more like how it "should" sound when naturally heard.
Don't get me wrong, I love looking at graphs and numbers 😀 but what if the way we measure things now, is inadequate to how our hearing behaves?
Compare it to the Acropolis in Greece. That building was built with an optical illusion. A building should have 90 degree angles, straight lines, however; the Acropolis was built with arcs in it, to offset the "distortion" our eyes have. We see it as straight; our mind tells us: 'it's straight'. But it isn't; when measuring it. My take is: we might not have the right measuring tools. Yet.
Hi Mooly,
thanks for posting the listening impressions! This is important, at least to me. To me it is more important than click on A = X, B= Y, False, true.
Anyone of us is able to make his A-B-X with the posted files in foobar 😉
thanks for posting the listening impressions! This is important, at least to me. To me it is more important than click on A = X, B= Y, False, true.
Anyone of us is able to make his A-B-X with the posted files in foobar 😉
I had pretty much the same impressions as all those who picked Green. It seemed smoother. The difference was tiny, but there, so I voted undecided.
Pavel says he usually prefers the Green circuit as well. But is it a preference only when re-recorded - or also when playing direct to the speakers? It's easy to understand why it might help the re-record loop. But of course we didn't hear the original file, so don't know how either compares to it. Maybe this file needed a little smoothing off. 🙂
Thanks for the test.
Pavel says he usually prefers the Green circuit as well. But is it a preference only when re-recorded - or also when playing direct to the speakers? It's easy to understand why it might help the re-record loop. But of course we didn't hear the original file, so don't know how either compares to it. Maybe this file needed a little smoothing off. 🙂
Thanks for the test.
Also in case I play music from CD or turntable. It does not seem to depend much on sound source.
Harsh is not the same like fatiguing. My speaker is very revealing, yet it can accept any music and displays beauty.
That's my point. Except that I'm not guessing. We need to know importance of each figures we can calculate.
Yes green is smoother, blue is abnormal (don't know where this abnormality came from).
Preamp versus no preamp is even a bigger issue because the difference is not tiny. Preamp has everything to be favored except... You know it if you are a passive preamp user.
Given the wide variety of opinions, some of them directly opposing, is it not beyond possibility to admit that a lot of what we "hear" is just in our heads?
It seems that many now are just looking for excuses / justifications.
I could not hear a difference, BTW, but must admit that I spend most time listening to music rather than sound effects!
It seems that many now are just looking for excuses / justifications.
I could not hear a difference, BTW, but must admit that I spend most time listening to music rather than sound effects!
The comments were indeed imaginative, but they did lack the "wife in the next room" meme. Too bad, I should have contributed that if I had known it would somehow be missed.
Pano, did you try ABXing them?
Pano, did you try ABXing them?
- Status
- Not open for further replies.