Go Back   Home > Forums > >
Home Forums Rules Articles diyAudio Store Blogs Gallery Wiki Register Donations FAQ Calendar Search Today's Posts Mark Forums Read

Everything Else Anything related to audio / video / electronics etc) BUT remember- we have many new forums where your thread may now fit! .... Parts, Equipment & Tools, Construction Tips, Software Tools......

Can you hear a difference between 2 solid state preamps?
Can you hear a difference between 2 solid state preamps?
Please consider donating to help us continue to serve you.

Ads on/off / Custom Title / More PMs / More album space / Advanced printing & mass image saving
View Poll Results: Can you hear a difference between the two test files
I can hear a difference, but have no ABX result 12 50.00%
I cannot hear a difference and have no ABX result 6 25.00%
I can hear a difference and have an ABX result 4 16.67%
I cannot hear a difference and have an ABX result 2 8.33%
Voters: 24. You may not vote on this poll

Reply
 
Thread Tools Search this Thread
Old 27th November 2019, 01:30 PM   #331
PMA is offline PMA  Europe
diyAudio Member
 
PMA's Avatar
 
Join Date: Apr 2002
Location: Prague
Quote:
Originally Posted by Jakob2 View Post
The basic rule is that "cherry picking" is prohibited, period.
However, "learning phase" is allowed, in many DBT tests. What is the difference if you call say first 5-10 attempts as "learning phase" or "unsuccessful trials", and all other trials after learning phase count and are above some threshold, in this case 6/8 and better. Regardless if 2 protocols or more are posted. Is it cherry picking as well?
  Reply With Quote
Old 28th November 2019, 10:47 AM   #332
Jakob2 is offline Jakob2  Germany
diyAudio Member
 
Join Date: Jun 2004
Location: germany
Quote:
Originally Posted by PMA View Post
However, "learning phase" is allowed, in many DBT tests. What is the difference if you call say first 5-10 attempts as "learning phase" or "unsuccessful trials", and all other trials after learning phase count and are above some threshold, in this case 6/8 and better. Regardless if 2 protocols or more are posted. Is it cherry picking as well?
The term "cherry picking" means just that, to pick the best from a badge for the reports; sometimes happening if people don't know about the problem, sometimes happening intentionally (one of the reasons for the so-called replication crisis in some fields).

Of course, nothing wrong with training, if the training phase is really parted from the "real" test phase.
We have mentioned before Feynman's famous speach and it is well worth to remember his line that "the easiest one to fool is oneself" .

If you state _before_ doing a test if it will be a training test (and therefore not be counted) or a real test (that will be reported/used for statistical analysis) there is no problem.
  Reply With Quote
Old 28th November 2019, 11:23 AM   #333
mmerrill99 is offline mmerrill99
diyAudio Member
 
Join Date: Aug 2014
Quote:
Originally Posted by Jakob2 View Post
Not sure, why you've posted this.
The basic rule is that "cherry picking" is prohibited, period.
If one does for example 21 8-trial ABX tests and got two with 5/6 hits then it would be cherry-picking to only report these two with combined results.

If one does for example 11 16-trial ABX tests and got one with 12 hits then it would be cherry-picking to only report this one.

One must use all the data that is available for the statistical analysis, but it is always possible to do the evil "cherry picking" (and others ) regardles of the number of trials in each test.
I hope that clarifies it?!
Jakob, my point is that if one 'cherry picks' & reports 11 correct out of 16 trials (in one test) from a set of listening tests (all tests of 16 trials each) that then this is more statistically significant statistically 'cherry picking' a 5 out of 8 run & a 6 out of 8 run from a set of ABX listening tests - it's statistically more difficult to get one run of 11 correct in 16 than it is to get a 5 & 6 in runs of 8 trials.

Agreed, no cherry picking is allowed but sometimes people are not even aware they are cherry picking & that getting 5 out of 8 & 6 out of 8 randomly, is not terribly difficult if enough ABX runs are considered

Foobar ABX was changed a while back in a number of ways - one of which was to stop reporting to the test subject, the ongoing total of correct results - this was to avoid people stopping the test when they had got a good run of positive results - a form of 'cherry picking'.

As you can see from the posts here people still think ABX testing is simply about "hearing" & fail to understand the significance of the statistics behind the test

Last edited by mmerrill99; 28th November 2019 at 11:45 AM.
  Reply With Quote
Old 28th November 2019, 11:33 AM   #334
mmerrill99 is offline mmerrill99
diyAudio Member
 
Join Date: Aug 2014
Quote:
Originally Posted by PMA View Post
However, "learning phase" is allowed, in many DBT tests. What is the difference if you call say first 5-10 attempts as "learning phase" or "unsuccessful trials", and all other trials after learning phase count and are above some threshold, in this case 6/8 and better. Regardless if 2 protocols or more are posted. Is it cherry picking as well?
PMA, if the 'real', non training ABX runs show a statistical significance then they are considered a positive result. if you report 5/8 & 6/8 & "2 or more protocols" (meaning ABX tests?) were run then are all these subsequent ABX tests either 5 or 6/8 threshold.

In other words if you are consistently scoring above a threshold & you report just that threshold then that is fine.
  Reply With Quote
Old 1st December 2019, 05:57 PM   #335
Jakob2 is offline Jakob2  Germany
diyAudio Member
 
Join Date: Jun 2004
Location: germany
Quote:
Originally Posted by mmerrill99 View Post
Jakob, my point is that if one 'cherry picks' & reports 11 correct out of 16 trials (in one test) from a set of listening tests (all tests of 16 trials each) that then this is more statistically significant statistically 'cherry picking' a 5 out of 8 run & a 6 out of 8 run from a set of ABX listening tests - it's statistically more difficult to get one run of 11 correct in 16 than it is to get a 5 & 6 in runs of 8 trials.<snip>
Generally it depends; according to PMA's description it has to be 5/8 and 6/8 in consecutive runs, probability to get such a result is comparable to the probability to get a 11/16 result if both are doing 8 tests.
If it doesn't have to be in consecutive runs then you are right, probabilities are higher for the 5/8 and 6/8 case.

But we shouldn't use the term "statistically significant" for "cherry picking".
  Reply With Quote
Old 2nd December 2019, 02:35 PM   #336
Jakob2 is offline Jakob2  Germany
diyAudio Member
 
Join Date: Jun 2004
Location: germany
Quote:
Originally Posted by billshurv View Post
The blowtorch is never the best place for a serious discussion though
It used to be different before grumpy old dogmatics decided that it wouldb be better to destroy any friendly discussion where people don't agree to their handmade hypothesises about what people can hear.

But anyway, usually someone'e post about audible differences between opamps (in nonpathologically behaving circuits with comparable measured numbers) would be enough to trigger a flood of posts mocking audiophiles, demanding "level matched double blind listening tests" and even more "extreme" requirements. But this time?

Quote:
By the time I would find the time to do that sort of evaluation I'd be too old to trust my ears at all!
It's not that time consuming at all; just to find out why and where differences (that might have led to preferences) exist. A bit of additional analysing overall.

Quote:
know it has been stated, but this is the first time I have seen any evidence of it in action. Maybe I haven't been looking hard enough or maybe I haven't been participating enough for it to become more visible.
I was just surprised reading that, because I (in this case) underestimated that personal experience of a fact is much more convincing than to read about it.

Quote:
Nevertheless I have learned something .....
And so did I.

I'd hope that this example and the opamp case in the Blowtorch thread might help to prevent members from "BS - shouting" more often when others describe their listening impressions.
  Reply With Quote
Old 2nd December 2019, 02:42 PM   #337
scottjoplin is offline scottjoplin  Wales
diyAudio Member
 
scottjoplin's Avatar
 
Join Date: Dec 2015
Location: Penrhyndeudraeth
How often are sound files, measurements and schematics included with listening impressions in order for it to be fully open to participation? I notice the usual critical members posting in this thread have not fully participated, I wonder (a little bit) why that is.......
  Reply With Quote
Old 2nd December 2019, 02:50 PM   #338
Markw4 is offline Markw4  United States
diyAudio Member
 
Join Date: Jul 2016
Location: California
Quote:
Originally Posted by Jakob2 View Post
I was just surprised reading that, because I (in this case) underestimated that personal experience of a fact is much more convincing than to read about it.
That is why I have tried to invite people out to visit, so they can see for themselves.

Quote:
I'd hope that this example and the opamp case in the Blowtorch thread might help to prevent members from "BS - shouting" more often when others describe their listening impressions.
+1
  Reply With Quote
Old 2nd December 2019, 03:19 PM   #339
billshurv is offline billshurv  United Kingdom
diyAudio Member
 
billshurv's Avatar
 
Join Date: Mar 2014
Can you hear a difference between 2 solid state preamps?
Quote:
Originally Posted by Jakob2 View Post
It used to be different before grumpy old dogmatics decided that it wouldb be better to destroy any friendly discussion where people don't agree to their handmade hypothesises about what people can hear.
That would have been before my time . I should be clear. I don't believe I could tell the files apart if they were shuffled in some way. If I get round to testing them and I can, then the real headscratching begins.


Quote:
But anyway, usually someone'e post about audible differences between opamps (in nonpathologically behaving circuits with comparable measured numbers) would be enough to trigger a flood of posts mocking audiophiles, demanding "level matched double blind listening tests" and even more "extreme" requirements. But this time?
There is bad behaviour on both sides. If someone claims night and day differences with some random component rolling or having paid someone to empty the kitty litter into their DAC with no before or after measurements then they are fair game in the lounge IMO.


Quote:
It's not that time consuming at all; just to find out why and where differences (that might have led to preferences) exist. A bit of additional analysing overall.
Right now that is time I don't have. My critical listening time is a couple of hours a month at the moment. Come the spring that should change, but such is the self-inflicted burden of a second brood.


Quote:
I was just surprised reading that, because I (in this case) underestimated that personal experience of a fact is much more convincing than to read about it.
I've been uneasy about the fact that I can't translate 90% of what audio reviewers are saying into something that relates to me since the 80s. Online it's worse. I've told all my children not to touch hot things and all of them have got burned at least once so I can confirm humans need to experience things . I am no closer to a common lexicon but at least now I can be sure I focus on different things. Wrong or not, they are my preferences and I make no claims for if anyone else would enjoy my system.
  Reply With Quote
Old 2nd December 2019, 03:44 PM   #340
Markw4 is offline Markw4  United States
diyAudio Member
 
Join Date: Jul 2016
Location: California
Quote:
Originally Posted by billshurv View Post
That would have been before my time . I should be clear. I don't believe I could tell the files apart if they were shuffled in some way. If I get round to testing them and I can, then the real headscratching begins.
Bill,
Possibly your system isn't good enough to make the differences as clearly audible as they otherwise might be. That is why my old Bryston 4B power amp had to go, and was replaced with a Benchmark AHB2. Now I find the Benchmark DAC-3 is coming very near to the end of its usefulness. It will go when the AK4499 dac on my workbench is ready.

Happy to let you hear for yourself if and when you get around to visiting Northern California.
  Reply With Quote

Reply


Can you hear a difference between 2 solid state preamps?Hide this!Advertise here!
Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
Germanium in the World of solid state (preamps) Bojan Hajdinjak Solid State 115 11th May 2015 06:05 PM
Preamps for the great solid state power amps audiostar Solid State 2 4th September 2012 07:41 PM
The Solid State Wiki - Your solid state reference guide Jason Solid State 0 25th June 2002 06:26 PM


New To Site? Need Help?

All times are GMT. The time now is 09:50 AM.


Search Engine Optimisation provided by DragonByte SEO (Pro) - vBulletin Mods & Addons Copyright © 2020 DragonByte Technologies Ltd.
Resources saved on this page: MySQL 14.29%
vBulletin Optimisation provided by vB Optimise (Pro) - vBulletin Mods & Addons Copyright © 2020 DragonByte Technologies Ltd.
Copyright ©1999-2020 diyAudio
Wiki