I am very interested in this. If you can, please provide links to the papers or other related info.
Yes. And I don't see it worthwhile or productive to quibble about whether irrelevant strictly means innocuous or means unimportant in this situation.
As I brought up previously. And it certainly seems possible. But I also see this biased subject issue as the result of the sides being relatively vocal and visible members of the audio community. Therefore there is a somewhat combative overtone to the proceedings. And the aftermath. People being people, it's to be expected at some level or another.
Just to clarify, are you looking for ABC or D preference, or group A vs. group B preference? Or?
And when they show no preference?
In itself I don't have an issue with such a challenge, nor do I find the ethics questionable (unless they are kidnapping subjects and strapping them in a chair).
Personally, I see it not as a product problem but a market problem. IMO people will still buy what they want irregardless of any data from ABX, Consumer Reports, diyAudio, or whatever. I just cannot accept the no-money-for-tests explanation. Use student volunteers. Give the inmates a little more yard time. I fear it's more to do with the vocal & visible group mentioned already. That is the focus.
Even the one making the observation? On this forum that isn't always the case. People declare such opinions as fact, thereby becoming part of the vocal & visible group. So we all know what comes next...
Hi,
I am looking for any data I can get, but preferably we have a clear preference across the listening group for one or several out of the items presented blind over the others. Each item gets multiple presentations, so we get enough data that we can analyse the data statistically.
Then I conclude that items being tested are for the specific application essentially identical and interchangeable.
I do, as such challenges are especially good in maximising all the factors that mitigate against the tests sensitivity, combined with a tests that by methodology is already biased we have a very small chance of anyone passing the "challenge".
This overall in all practical terms places such challenges on par with circus sideshow mentalists, fortune tellers, Nigerian Letters and practitioners of shell games, that is among confidence tricksters. To be honest, I have no issue with confidence tricksters and think that those who fall for them are only lying in the bed they made for themselves.
What I have issues is when such confidence tricksters attempt to pass themselves off as scientists.
Throughout this thread a number of studies have been linked or referenced that did exactly that. Earl Geddes's distortion studies for example used Student Volunteers, while Juergen Ackerman used paid volunteers and was clearly able to finance his tests somehow. So the problem clearly can be overcome, if there is a will to do so.
However, I note that one specific, extremely vocal and militantly "anti high end" group does not do so, but instead:
And due to the high profile they have gained with their sensationalist and militant approach their tests are commonly referenced as being relevant or indeed promoted by some DIY Audio Members who are either incredibly gullible or share this groups agenda and methods...
I think that most of make such observations do not make scientific claims and are aware that they are sharing personal observations, opinions and views.
However, the starkness of what is written together with the view of academics that wish to see everything in a frame like a peer reviewed academic journal (which an open discussion board obviously neither is, intends to be or can be) may cause some to attach more significance than was intended and find they wish to challenge such views that they disagree with in a manner that would be appropriate in an academic journal, but leaves real people both bewildered and offended. And then they argue back...
It does take two sides, however the conclusion of one of the two sides if debate happens and the other notes "I was only giving my personal opinions" come back routinely with "if you cannot back it up with an ABX tests you should not say it" and challenges become attempted censorship of ideas.
Ciao T
I am looking for any data I can get, but preferably we have a clear preference across the listening group for one or several out of the items presented blind over the others. Each item gets multiple presentations, so we get enough data that we can analyse the data statistically.
Then I conclude that items being tested are for the specific application essentially identical and interchangeable.
I do, as such challenges are especially good in maximising all the factors that mitigate against the tests sensitivity, combined with a tests that by methodology is already biased we have a very small chance of anyone passing the "challenge".
This overall in all practical terms places such challenges on par with circus sideshow mentalists, fortune tellers, Nigerian Letters and practitioners of shell games, that is among confidence tricksters. To be honest, I have no issue with confidence tricksters and think that those who fall for them are only lying in the bed they made for themselves.
What I have issues is when such confidence tricksters attempt to pass themselves off as scientists.
Throughout this thread a number of studies have been linked or referenced that did exactly that. Earl Geddes's distortion studies for example used Student Volunteers, while Juergen Ackerman used paid volunteers and was clearly able to finance his tests somehow. So the problem clearly can be overcome, if there is a will to do so.
However, I note that one specific, extremely vocal and militantly "anti high end" group does not do so, but instead:
- prefers to rely on statistics that do not balance the rist of type 1 and type 2 statistical errors but instead minimise the risk of type 1 statistical errors at the expense of a risk of type 2 statistical errors that approaches near certainty
- prefers to rely on individuals that are most likely highly biased
- prefers to use methods (Challenges) that maximise test stress
- refuses to use any positive and negative controls
And due to the high profile they have gained with their sensationalist and militant approach their tests are commonly referenced as being relevant or indeed promoted by some DIY Audio Members who are either incredibly gullible or share this groups agenda and methods...
I think that most of make such observations do not make scientific claims and are aware that they are sharing personal observations, opinions and views.
However, the starkness of what is written together with the view of academics that wish to see everything in a frame like a peer reviewed academic journal (which an open discussion board obviously neither is, intends to be or can be) may cause some to attach more significance than was intended and find they wish to challenge such views that they disagree with in a manner that would be appropriate in an academic journal, but leaves real people both bewildered and offended. And then they argue back...
It does take two sides, however the conclusion of one of the two sides if debate happens and the other notes "I was only giving my personal opinions" come back routinely with "if you cannot back it up with an ABX tests you should not say it" and challenges become attempted censorship of ideas.
Ciao T
I got a chuckle out of your opening, and the first couple of responses were very well stated. But then the rhetoric begins to turn heavy. Lots of studies get linked here, and Geddes' is one I want to take time to read thoroughly, though prompted by a different thread. But you are aware of the extremely vocal and militantly "high end" group that exists.
Of your 4 points, the first is "statistics talk" that I understand to be related to point 4, but otherwise isn't something I'm familiar with. Point 2 and 3 I also understand to be related. It's about publicity. Get some big (biased) names, call it a "Challenge" (the media will do it if no one else), and everybody gets flown out to California with a hotel room and a per diem for XYZ cable show. Easy funding.
BTW, sensationalist and militant isn't the question. Accurate and effective is.
I think it's fairly obvious things aren't always presented as personal opinions and views.
Peer-reviewed should be the frame. What form it takes from there isn't as important. It's to this site's enormous credit that everyone from the newbie to the gray-haired tube engineer has peers. I hope I'd be included with real people, and I'm not so much bewildered as just watching the sausage being made. It ain't always pretty, but things are for the most part kept contained.
What I'm bewildered about is the practical difference between academics and statistics in the present context.
Of your 4 points, the first is "statistics talk" that I understand to be related to point 4, but otherwise isn't something I'm familiar with. Point 2 and 3 I also understand to be related. It's about publicity. Get some big (biased) names, call it a "Challenge" (the media will do it if no one else), and everybody gets flown out to California with a hotel room and a per diem for XYZ cable show. Easy funding.
BTW, sensationalist and militant isn't the question. Accurate and effective is.
I think it's fairly obvious things aren't always presented as personal opinions and views.
Peer-reviewed should be the frame. What form it takes from there isn't as important. It's to this site's enormous credit that everyone from the newbie to the gray-haired tube engineer has peers. I hope I'd be included with real people, and I'm not so much bewildered as just watching the sausage being made. It ain't always pretty, but things are for the most part kept contained.
What I'm bewildered about is the practical difference between academics and statistics in the present context.
Hi,
Yes, this is the point.
There are two groups with views as unreasonable and mutually opposed, yet sides of the same coin and both are vocal.
Looking at both sides though there is more "militancy" on the "anti" side, which is unsurprising. Those against something are essentially expressing their negative side and negative emotion.
So while one side is primary enthused and full a will to share these positive things, the other is cynical, morose and contrarian and at least appears to want nothing else except to spoil the other sides's fun.
It is not just publicity, even when the people are only local audiophiles or believers of "everything sounds the same" (as they often are) the subjects are biased, the statistics are biased.
With regards to the specific group of self proclaimed debunkers in audio their work is certainly accurate, but highly effective in their own intentions.
Why? This is not academia. This is not your ivory tower. It is place where people with common interests hang out and talk about their hobby, no more. Trying to impose silly requirements before people can say something leads to resistance and bad feeling.
What is clear to me is that you are neither really aware of the people involved or of the controversies (which could have been remedied by perusing some of the references I gave) and the fact that this whole debate has been ongoing for over 3 decades.
And the crux of the matter is simply that they insist no-one should make statements of personal observations in Audio that A and B are different if they cannot pass THEIR SPECIFIC (and fatally flawed) test.
It is a simple as that.
Ciao T
Yes, this is the point.
There are two groups with views as unreasonable and mutually opposed, yet sides of the same coin and both are vocal.
Looking at both sides though there is more "militancy" on the "anti" side, which is unsurprising. Those against something are essentially expressing their negative side and negative emotion.
So while one side is primary enthused and full a will to share these positive things, the other is cynical, morose and contrarian and at least appears to want nothing else except to spoil the other sides's fun.
It is not just publicity, even when the people are only local audiophiles or believers of "everything sounds the same" (as they often are) the subjects are biased, the statistics are biased.
With regards to the specific group of self proclaimed debunkers in audio their work is certainly accurate, but highly effective in their own intentions.
Why? This is not academia. This is not your ivory tower. It is place where people with common interests hang out and talk about their hobby, no more. Trying to impose silly requirements before people can say something leads to resistance and bad feeling.
What is clear to me is that you are neither really aware of the people involved or of the controversies (which could have been remedied by perusing some of the references I gave) and the fact that this whole debate has been ongoing for over 3 decades.
And the crux of the matter is simply that they insist no-one should make statements of personal observations in Audio that A and B are different if they cannot pass THEIR SPECIFIC (and fatally flawed) test.
It is a simple as that.
Ciao T
You´ll find the Geddes/Lee papers at their website:
Perception
An example would be the ITU-R BS.1387 paper:
http://www.itu.int/dms_pubrec/itu-r/rec/bs/R-REC-BS.1387-1-200111-I!!PDF-E.pdf
An example in practice is the discussion on the Pras/Guastavino paper i´ve cited in this thread before:
44.1 vs 88.2 ABX report at AES - Hydrogenaudio Forums
See post #4 where Arny Krueger talks about the "resample" issue.
Normally one should think that a look into the datasheets of the AD-converters would give a reasonable answer to any question regarding a possible influence of the different sample rates.
Well, the 'prove it' crowd may be more cranky, but they certainly are in a tiny minority, so I really don't see the cause for alarm/paranoia (the dark forces of the ABX mafia?).
Look around! Just in the popular online forums (diyaudio, audiokarma) there are thousands having a good time talking about cables, caps, burn-in, pointy feet, tube rolling, vintage gear, etc etc. Their enjoyment doesn't seem to be much reduced by the 'prove it' group.
BTW, throwing out insulting descriptions of one of the 'two sides' as you have done really uncovers a lot of your own biases. So much for the 'I only want data' idea.......
Perhaps you think advising an audio newcomer to pay $1000 for a power cable, or elevate the speaker cables off the floor with tiny tripods is a 'positive thing' shared enthusiastically; I don't. And, yes, it does happen- often.
Yes, if reduced to two sides, either could be descriptively worded to be the "positive" and the "negative."
I'm just disappointed that you made no reference to the practical difference.
I don't know where you get the idea that peer review is limited to academia. Two pubescent girls doing each other's hair in the mirror is peer review. "I'm a noob. What do you think of my circuit?" is peer review. And if Pass, Curl, Cordell, Geddes, et al are hobbyists, I'm not even a blip on the radar. And in that light, the "ivory tower" stuff isn't even good rhetoric.
Should I be? I'm interested in good data. I don't care who provides it. I have, though, no reservations to conceding your point. I've never been a regular reader of "stereo magazines." The closest I've come is Larry Klein's Audio Update column in the old RE.
I'm just disappointed that you made no reference to the practical difference.
You're repeating this, when even a cursory investigation shows it to be untrue. If it isn't scientific claims being made, then the scientist poseurs wouldn't have opportunity to be "militant".
Hi,
We may argue that discussing anything or comparing anything (like two looters in the recent Riots in London - 'What did you get?' - 'A 50" Pioneer Kuro TV.' - 'Awesome, I only got a cheap china made 32" LCD TV.') is a "peer review".
However I was instead referring to the somewhat more formal and more limited scope process that attends publication in certain kinds of Magazines.
I think you should be.
If some "data" is published it worth knowing who is in the byline.
If someone called Bernard Madoff published data regarding investments, it would be worth knowing who Bernard Madoff (with me money) is...
Even cursory reading of this set of boards here proves the opposite, so I contend it is shown to be true....
Well, let us consider the following situation.
Two old codgers named O'Brian and Nolan discussing the weather over a pint of Stout. "Terrible rain and wind for this time of year," O'Brian ventured.
"Ah, faith," Nolan replied, "I do not believe it is this time of year at all, at all."
At this, Murphy spoke up. "Ah, Jaysus," he said, "I've never seen a boogerin' normal day." He paused to set down his pint, then added thoughtfully, "And I never met a fookin' average man neither"
Are any scientific claims being made?
These boards here are NOT a peer reviewed scientific journals where every casual remark or observations is elevated to a "scientific claim". They are more like the Pub, O'Brian, Nolan and Murphy tended to hang out.
Ciao T
(About Sean Murphy nothing else appears in the record except a remark gleaned by Prof. LaPuta from one Nora Dolan, a housewife of the vicinity: "Sure, that Murphy lad never did any hard work except for getting up off the floor and navigating himself back onto the bar-stool, after he fell off, and he only did that twice a night.")
We may argue that discussing anything or comparing anything (like two looters in the recent Riots in London - 'What did you get?' - 'A 50" Pioneer Kuro TV.' - 'Awesome, I only got a cheap china made 32" LCD TV.') is a "peer review".
However I was instead referring to the somewhat more formal and more limited scope process that attends publication in certain kinds of Magazines.
I think you should be.
If some "data" is published it worth knowing who is in the byline.
If someone called Bernard Madoff published data regarding investments, it would be worth knowing who Bernard Madoff (with me money) is...
Even cursory reading of this set of boards here proves the opposite, so I contend it is shown to be true....
Well, let us consider the following situation.
Two old codgers named O'Brian and Nolan discussing the weather over a pint of Stout. "Terrible rain and wind for this time of year," O'Brian ventured.
"Ah, faith," Nolan replied, "I do not believe it is this time of year at all, at all."
At this, Murphy spoke up. "Ah, Jaysus," he said, "I've never seen a boogerin' normal day." He paused to set down his pint, then added thoughtfully, "And I never met a fookin' average man neither"
Are any scientific claims being made?
These boards here are NOT a peer reviewed scientific journals where every casual remark or observations is elevated to a "scientific claim". They are more like the Pub, O'Brian, Nolan and Murphy tended to hang out.
Ciao T
(About Sean Murphy nothing else appears in the record except a remark gleaned by Prof. LaPuta from one Nora Dolan, a housewife of the vicinity: "Sure, that Murphy lad never did any hard work except for getting up off the floor and navigating himself back onto the bar-stool, after he fell off, and he only did that twice a night.")
"Looking at both sides though there is more "militancy" on the "anti" side, which is unsurprising. Those against something are essentially expressing their negative side and negative emotion.
So while one side is primary enthused and full a will to share these positive things, the other is cynical, morose and contrarian and at least appears to want nothing else except to spoil the other sides's fun."
With all due respect, this is disingenuousin the extreme.
The problem is you drop into threads and bow beat everyone who does not share your view.
So while one side is primary enthused and full a will to share these positive things, the other is cynical, morose and contrarian and at least appears to want nothing else except to spoil the other sides's fun."
With all due respect, this is disingenuousin the extreme.
The problem is you drop into threads and bow beat everyone who does not share your view.
Ok last call. I'm sure someone wants to discuss DBTs more directly.
But it does make an interesting analogy for the first couple dozen posts in this thread.
Lastly, if you consider two sides to the debate, each side is for something and against something else. One side calling the other negative is silly posturing.
Yes, I could see peer review in a sense. But what I find more interesting is that you voluntarily and unilaterally constrained peer review to "the somewhat more formal and more limited scope process that attends publication in certain kinds of Magazines" and then denounced it as too academic. Some might view that as a straw man.
Ah, the qualifiers. If investment data is published, it is only important to me if the data is useful and reliable. If I consider handing over some of my money, then it is important to know if the potential recipient is crooked.
Informally, yes. This one: "Terrible rain and wind for this time of year," which can be compared with weather data from past seasons to determine if indeed the wind and rain is worse than an average year. The responses are strange for any conversation. The bartender overhears it all and figures it's time to cut off Nolan and Murphy.
But it does make an interesting analogy for the first couple dozen posts in this thread.
Lastly, if you consider two sides to the debate, each side is for something and against something else. One side calling the other negative is silly posturing.
Hi,
Really, I should think they may find this something more akin to the common definition of peer review and may instead view your approach as exceedingly broad:
First, if the person giving the advise is a crook, it may cast immediate doubts on the data's usefulness and reliability, would not say so?
And if a recommendation for implementing a test is published and results of such tests are published is it not worthwhile to understand that if claims are made for these are from individuals that have a history in providing useful and reliable information or not?
NO. NOT EVEN INFORMALLY. NO CLAIM IS MADE. NO SCIENCE IS INVOLVED.
That is my point. Normal people talk about the weather, their cars mpg and speed and many other topics without making any scientific claims in fact often being ignorant of any science involves.
It is you who projecting that "an informal scientific claim was made"...
This would presume such data exists for the very specific spot being discussed. I often encounter situation where only a few 100m from where I experienced sunshine I find evidence of very recent rain. Where I was the rain may not be terrible, but where I was not it was terrible.
Not really. They are extremely sensible and filled with common sense, especially those from this Murphy Lad...
You may not have noticed, I am not taking sides, I'm only showing yellow or red cards where I see a foul...
I personally think either side is just a bunch of maniacs and ejits who are mostly wrong in their claims and posturing, tough each side does have the occasional grains of truth that make reading them worthwhile...
Ciao T
Really, I should think they may find this something more akin to the common definition of peer review and may instead view your approach as exceedingly broad:
Wikipedia said:
First, if the person giving the advise is a crook, it may cast immediate doubts on the data's usefulness and reliability, would not say so?
And if a recommendation for implementing a test is published and results of such tests are published is it not worthwhile to understand that if claims are made for these are from individuals that have a history in providing useful and reliable information or not?
NO. NOT EVEN INFORMALLY. NO CLAIM IS MADE. NO SCIENCE IS INVOLVED.
That is my point. Normal people talk about the weather, their cars mpg and speed and many other topics without making any scientific claims in fact often being ignorant of any science involves.
It is you who projecting that "an informal scientific claim was made"...
This would presume such data exists for the very specific spot being discussed. I often encounter situation where only a few 100m from where I experienced sunshine I find evidence of very recent rain. Where I was the rain may not be terrible, but where I was not it was terrible.
Not really. They are extremely sensible and filled with common sense, especially those from this Murphy Lad...
You may not have noticed, I am not taking sides, I'm only showing yellow or red cards where I see a foul...
I personally think either side is just a bunch of maniacs and ejits who are mostly wrong in their claims and posturing, tough each side does have the occasional grains of truth that make reading them worthwhile...
Ciao T
"A process of evaluation involving qualified individuals within the relevant field" is precisely my working definition. I also notice that academia is somewhat differentiated in that definition.
On second thought, I'll explain. Either O'Brian is familiar with local weather patterns at that time of year (ie such data exists) and assumes Nolan is also, or he's making polite conversation of the mindless variety. The latter then must have been your intention.
Really, though, I like you ThorstenL and have no hard feelings on this end. I've enjoyed the conversation but I'm feeling like a threadjacker. And that wasn't on my misdeed list. I see some good and bad things on each side also. My favorite diy comeuppance story involves Forrest Mims, NASA, and atmospheric studies. Are you familiar with it? It proves that the pointy-headed guys in lab coats don't have all the answers. DBTs seem to me to be an effective (and even fair) way to evaluate playback performance. What I'm for is a better understanding of the underlying science of it all. As I've said, I don't foresee it causing any type of revolution in audio. Equipment companies will still find creative ways to distinguish themselves in the marketplace. People will still buy, build, and listen to what they want to.
It may. It may not. I'm sure Phil Spector still knows a thing or two about producing records. William Schockley is another name that could be mentioned. Bernie Madoff may well be an outstanding investment advisor. I wouldn't want to be friendly with any of them, but I'd prefer to keep their deeds and misdeeds in the proper context.
I guess you either see it or you don't.
Explain to me under what circumstances such data didn't exist and that statement is made.
On second thought, I'll explain. Either O'Brian is familiar with local weather patterns at that time of year (ie such data exists) and assumes Nolan is also, or he's making polite conversation of the mindless variety. The latter then must have been your intention.
Militant objectivists, militant subjectivists, militant referees. Ya gotta love it.
Really, though, I like you ThorstenL and have no hard feelings on this end. I've enjoyed the conversation but I'm feeling like a threadjacker. And that wasn't on my misdeed list. I see some good and bad things on each side also. My favorite diy comeuppance story involves Forrest Mims, NASA, and atmospheric studies. Are you familiar with it? It proves that the pointy-headed guys in lab coats don't have all the answers. DBTs seem to me to be an effective (and even fair) way to evaluate playback performance. What I'm for is a better understanding of the underlying science of it all. As I've said, I don't foresee it causing any type of revolution in audio. Equipment companies will still find creative ways to distinguish themselves in the marketplace. People will still buy, build, and listen to what they want to.
You´ll find the Geddes/Lee papers at their website:
Perception
An example would be the ITU-R BS.1387 paper:
http://www.itu.int/dms_pubrec/itu-r/rec/bs/R-REC-BS.1387-1-200111-I!!PDF-E.pdf
An example in practice is the discussion on the Pras/Guastavino paper i´ve cited in this thread before:
44.1 vs 88.2 ABX report at AES - Hydrogenaudio Forums
See post #4 where Arny Krueger talks about the "resample" issue.
Normally one should think that a look into the datasheets of the AD-converters would give a reasonable answer to any question regarding a possible influence of the different sample rates.
Thanks, I'll take a look.
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.