use one of the abx testers instead of just listening. Trying to sit and listen and find the difference is going to be too unreliable. I too "thought" I heard a difference, but then after repeated attempts at abx comparisons, I never did better than chance. I didn't do so on the best of reference listening setups, and will try again later in a room with less ambient noise and when I'm not so tired, but as I said, I didn't do better than chance.
It's also not all that hard to cheat on the ABX by making slight adjustments. I had never used software abx like the ones suggested, but the Foobar is the one I did best with (could have just been practice too). Anyway, you could fudge it by putting different wav files into each part, so different that they are easy to tell. You could insert a boost, a delay, etc. to all ensure the difference. I also did better with the short pieces than the long pieces, for whatever reason.
It's also not all that hard to cheat on the ABX by making slight adjustments. I had never used software abx like the ones suggested, but the Foobar is the one I did best with (could have just been practice too). Anyway, you could fudge it by putting different wav files into each part, so different that they are easy to tell. You could insert a boost, a delay, etc. to all ensure the difference. I also did better with the short pieces than the long pieces, for whatever reason.
Yes, you're right, in WinABX 'play A' plays the sample that you loaded first and 'play B' plays the sample that you loaded second. Then try 'play X' and determine if it is A or B. After choosing A or B click 'next trial' and you see whether you were correct or not. Then you can listen again to sample A and/or B or directly listen to X again. You can also first listen to X, then to A and/or B, choose then, or first listen again to X, do what you want. X remains the same during a trial until you choose and go to the next trial.pjpoes said:
In all ABX testing that I did myself during the past years, I preferred short samples, but this could be different for other people. I use ABX-testing to find out if I can hear a difference. This is the first thing to do. If you cannot hear a difference, it makes no sense to find out which sample sounds better to you in my opinion.
Attachments
You're doing good work here jeroen_d! I hope many people will take the small effort to do the listening tests. The summed outcome of these tests might surprise many of us.
It sure makes one wonder: what DOES actually still make a difference in speaker design. More and more I get the feeling there's little more to it than getting a smooth and reasonably flat frequency response curve on- and off-axis.
It sure makes one wonder: what DOES actually still make a difference in speaker design. More and more I get the feeling there's little more to it than getting a smooth and reasonably flat frequency response curve on- and off-axis.
keyser said:
Sure, "little more to it than ... on- and off-axis." but just try that! Its not so easy!!
It interesting to note by reading my "waveguide" thread that many still insist that Constant Directivity isn't important, that its distortion and phase that count (both of which have been tested for audibility and found to be irrelavent).
IMO its a few critical things that matter. First and formost IS constant directivity, but not any CD, but a fairly narrow CD to avoid early room reflections; second is smoothness of the CD (axial response is pretty unimportant); third is diffraction - highly audible especially at higher levels; fourth is thermal modulation (controlling which tends to lower nonlinearity). After that you are down into the mud of effects that affect perception.
In short a narrow directivity CD system which has good thermal capability and low diffraction will be a winner. Now if only somebody made a speaker like that!!
So far I haven't had any experience with CD-waveguided speakers. The closest thing was an Usher D-2, of which I've never seen any measurements (that brings to mind, neither have I ever seen any measurements of the Summa! <<hint>> ).
I do have experience with well designed dipoles with relatively good constancy of directivity, and boxspeakers with that same property. Those types of speakers sound good to me, but hey, that might just as well be suggestion at work
.
Myself and a couple of other DIY'ers are trying to get a pair of Summa's to the Netherlands because we're very interested if they live up to the hype. However, not being able to audition before purchase, is still holding us back. Earl, would you know of somebody in the Netherlands who has your speaker?
I do have experience with well designed dipoles with relatively good constancy of directivity, and boxspeakers with that same property. Those types of speakers sound good to me, but hey, that might just as well be suggestion at work
. Myself and a couple of other DIY'ers are trying to get a pair of Summa's to the Netherlands because we're very interested if they live up to the hype. However, not being able to audition before purchase, is still holding us back. Earl, would you know of somebody in the Netherlands who has your speaker?
keyser said:So far I haven't had any experience with CD-waveguided speakers. The closest thing was an Usher D-2, of which I've never seen any measurements (that brings to mind, neither have I ever seen any measurements of the Summa! <<hint>> ).
I do have experience with well designed dipoles with relatively good constancy of directivity, and boxspeakers with that same property. Those types of speakers sound good to me, but hey, that might just as well be suggestion at work
Myself and a couple of other DIY'ers are trying to get a pair of Summa's to the Netherlands because we're very interested if they live up to the hype. However, not being able to audition before purchase, is still holding us back. Earl, would you know of somebody in the Netherlands who has your speaker?
There are measurements of all of my speakers on my web site.
No loudspeaker system that uses pistons can be CD because the sources are not CD. Dipoles are not CD. ONLY a waveguide can be CD. Its possible to design a piston based loudspeaker system to be almost CD with a wide directivity, but NOT a narrow directivity, and its never exactly CD.
I think that "hype" is an incorrect word. Better would be "reviews".
I don't think that there are any of my designs in the Netherlands, closest might be Sweden, Spain or Switzerland. The old model of auditioning loudspeakers is going to have to go away however because it drives costs up to the point of impracticality. If you learn to read the measurements and rely on them you won't go wrong and you'll save a lot of money to boot!
gedlee said:
You could also implement headphone auditioning (as you've hinted at before) by posting WAV file recordings of music or sound being played on various speakers properly setup in a good room.
Variables could be:
Speakers: CD waveguide / point source / dipole / line array
Recording: acoustic music / acoustic sounds / sound effects
Level: low / movie / concert
I think the cost of that type of audition would be much more acceptable.
That is absolutely true and is, I believe, the future of loudspeaker auditions and sales. I spoke with Floyd Toole about this a few months back and he agreed and said that basically Harman does all of their listening tests over headphones with bi-naural recordings. He said that he mentioned this idea to marketing, but, as should be expected, they were leary of loosing control. I will do this at some point, but the ramp up of comparisons will be slow as it is very time consuming and I simply don't have any free time to do anything but build loudspeakers.
gedlee said:
Just to verify i am not completly out of it ..
let's assume a line array ( vertical for purposes )
consisting of many many drivers in OB dipole config
would have advantages toward what you are pointing at ... thermal = reduced because of the # of drivers
working for the same db
and ob cylindrical type of directivity with reduced side roof reflections ...
is that a good example ?
then you are talking about the way pistonic drivers can't achieve what we are looking for,
does any other technology or idea does it better?
contrary to most here that seem to try and do the best possible with what they have or with very limited conditions ( wich is normal and wich i respect and understand )
i try to think out of the box for eveyrhting i do in life
i usually work by setting goals and trying to achieve them outside of the normal paths ..
do you suggest that, for audio reproduction to advance to the next level, we would all need to change path
from what were seem to be "stuck" in ..??
gedlee said:
In such case those tests are flawed. You claimed a 50% 3rd order THD would pass as undetecatable and I scored 10/10 in the test in this thread which has way lower levels... and no comment from you? You just avoid to adress things that proove you're wrong? Did you take the test yourself?
You said you could post files but when several of us asked for them we didn't see anything.. but then Jerome posted his test files which clearly proove you're wrong (which was no news to me).
Phase is typically not very "important" but there are limits as to how much phase distortion or group delay we can tolerate in different registers.
A "piston dipole" can easily be made that has practically CD up to 3kHz and up from there you can use a pistonic dome tweeter in a waveguide (or just use it on a baffle which JohnK prooved to have very smooth in room response). This would be a system that has more uniform room response than your speakers that goes from omni to narrow dispersion over a decade.
I'm very surprised to read that. Such tests can be interesting for some things but is not very wise for others. This should be obvious for anyone with basic insight in physics and human hearing.
/Peter
MartinQ said:
If you want to investigate a speakers performance you should listen to the speaker first and foremost.
When you use a binaural recording and headphones you have introduced a whole chain of devices between the actual sound and what you hear. Also a dummy head doe not have a HRTF that suits everybody and that's a compromise even if the same dummy head is used with all speakers in a test.
So we have:
- A dummy head with imperfect HRTF
- Microphones with distortion and noise
- Mic preamps with distoriton and noise
- AD with possible colorations
- DA with possible colorations
- Headphone amplifiers with possible imperfections
- Headphones which can have higher distortion than the speakers and a frequency response that may (likely) alter the results.
/Peter
gedlee said:
You're right. My bad. I hadn't seen them before.
gedlee said:
Of course, at some angle a speaker will cease to be CD. Well designed boxspeakers can be pretty much CD up to large angles. Actually, 'Constant' isn't really the right word here; I'd rather call it 'Smoothly Increasing-' or something like that, save for a better term. But of course the same goes for a Summa type speaker. If true Constant Directivity is utopic, then what roll-off slope at what angle would be 'optimal', and why?
gedlee said:
Whatever we call it, the Summa and your viewpoints sure are getting a lot of attention on the forums these days. Not in the least part because of your own numerous contributions, which (I must say!) I very much enjoy reading.
gedlee said:
I don't think audiotioning physical speakers in a room will disappear any time soon. A small part of the community, presumably a part with some technical back ground, might agree it is an easier and perhaps better way. But in a world of $10000 cables, tube amps, demagnetisation devices for CD's (yeah, Compact Discs that is, mind you!) etc. I don't really see that happening at a large scale. And then think of the general public.
Even for those who know how to interpret measurements, at some point they'll still have to listen to a speaker in their room. Frequency response curves are not very hard to read for what they are. Knowing how they correlate with what you will hear is more difficult. How do I know if your speaker, which has an imnipolar radiation pattern at low frequencies and a large amount of directivity at high frequencies, isn't going to sound liveless, with a lack of high frequencies?
What I'm trying to say is, you can't capture everything in measurements and numbers. You won't know in what way a dipole sounds different from a conventional box speaker untill you hear it! The same thing goes for a speaker like the Summa. Sure, the measurements look nice. But I couldn't possibly tell how it would compare to other well measuring speakers. How can I tell by looking at the graphs that narrow directivity will sound better than wide directivity (the kind of speaker Toole and his friends propose) or figure of 8 kind of radiation? This has to do with psycho-acoustics and this is never easy.
Therefor, I won't be 'saving myself a lot of money' before I get a chance to audition your kind of speakers. I think few people would spend a couple thousand dollars on a speaker the couldn't first listen to.
Ok I've now run 3 people through a total of 15 experiments with 10 trials each. Two other people tried 3 different headphone combinations (so 6 "experimental situations") and then I made up the bulk of the rest of them. I did the best of anyone thus far, getting 7 out of 10 right. When I tried 20 trials the number became 16 out of 20. While I got more than half right, it's still statistically too low. I have a friend coming over today who will try as well, and I'm sending the necessary files to my brother to give it an attempt, but I doubt he will have the patience to do it right.
Peter, either you have much better hearing than all of us, or there is something wrong with what I'm doing. Given that I've done these as part of my work, I'm pretty sure I'm executing the experiment correctly. Biggest problem I have is ensuring the sound quality is "clean" enough. I found something was wrong with my headphone amp so right now I have Shure headphones plugged into my pre/pro's headphone output. My laptop is connected digitally to the pre/pro, and the processor is run in "bypass" mode, so should be a relatively transparent signal path.
Anyone else take the time to do 10 or more trials with one of the mentioned ABX softwares do better than chance? Technically you will need a full 10 out of 10 to be statistically better than guessing, but I would take 9 out of 10 (If you get that, try again with 15 or 20 trials, and see if you can do better than a 5% chance of guessing).
Ok so the 3 % distortion added some compression to the signal, if I recall the earlier conversation correctly. Does this account for the measured average gain difference between the two tracks. The track gain is about a full db less in one over the other, wanted to make sure that's what was accounting for it.
Peter, either you have much better hearing than all of us, or there is something wrong with what I'm doing. Given that I've done these as part of my work, I'm pretty sure I'm executing the experiment correctly. Biggest problem I have is ensuring the sound quality is "clean" enough. I found something was wrong with my headphone amp so right now I have Shure headphones plugged into my pre/pro's headphone output. My laptop is connected digitally to the pre/pro, and the processor is run in "bypass" mode, so should be a relatively transparent signal path.
Anyone else take the time to do 10 or more trials with one of the mentioned ABX softwares do better than chance? Technically you will need a full 10 out of 10 to be statistically better than guessing, but I would take 9 out of 10 (If you get that, try again with 15 or 20 trials, and see if you can do better than a 5% chance of guessing).
Ok so the 3 % distortion added some compression to the signal, if I recall the earlier conversation correctly. Does this account for the measured average gain difference between the two tracks. The track gain is about a full db less in one over the other, wanted to make sure that's what was accounting for it.
Pan said:
Peter, have you had equal success with the longer samples that were posted?
I did some listening to the Liszt_norm_.wav files last night but was unable to hear an obvious difference so I discontinued. I was using my Sennheiser HD540 headphones plugged directly into the motherboard and the Foobar2000 ABX feature.
There is some system noise coming through and I haven't tested audio capabilities of this system yet so I'm not exactly sure what I'm up against. I will take another look and have a more serious try at it.
JinMTVT said:
Well its not CD so from that perspective its not a good example.
JinMTVT said:
Of course! You should know the answer to that question, but if you mean other than a waveguide, yes an electronically controlled array can be CD, but it has other problems.
JinMTVT said:
Well thinking "out of the box" is fine if you like vogue phrases, but its sometimes good just to think along the lines of what the science tells us. That works too, but, of course, you have to actually know the science to do it that way.
I think that I would agree with your last sentence. Why do I get so much opposition on these forums? It's not because what I say is so complimentary to the current dogma. It's the exact opposite. What I am saying is heresy and many people who are deeply invested in the old ways and beliefs are going to have a real problem with it. But you have to consider that either I am a total quack - as they say - or maybe I do have a "better way". Look at my career and my publications and see if you can find some data to support the "quack" theory.
Ok so the 3 % distortion added some compression to the signal, if I recall the earlier conversation correctly. Does this account for the measured average gain difference between the two tracks. The track gain is about a full db less in one over the other, wanted to make sure that's what was accounting for it.
@pjpoes: The added distortion reaches 5% at max amplitude. The gain of the track is not altered, however, at full scale the distortion results in a reduced amplitude of the peaks in the signal.
There appear to be people (apart from pan) who I know and who I trust, that are able to reliably detect the audible difference between the tracks 1 and 2. For me personnally, I do not care if they are able to detect what the original was by selecting the one they prefer. I want reproduction to be as true to the original as possible, so if a certain amount of distortion changes the sound (although hard to detect) it should be avoided. So in designing systems I would select speakers with 3rd order harmonic distortion well below 1% and make sure that I stay away from their excursion limits.
@pjpoes: The added distortion reaches 5% at max amplitude. The gain of the track is not altered, however, at full scale the distortion results in a reduced amplitude of the peaks in the signal.
There appear to be people (apart from pan) who I know and who I trust, that are able to reliably detect the audible difference between the tracks 1 and 2. For me personnally, I do not care if they are able to detect what the original was by selecting the one they prefer. I want reproduction to be as true to the original as possible, so if a certain amount of distortion changes the sound (although hard to detect) it should be avoided. So in designing systems I would select speakers with 3rd order harmonic distortion well below 1% and make sure that I stay away from their excursion limits.
keyser said:
If I understand the question, there is a lot of evidence to say that at 45 degrees the response should be down about 6 dB. There is also support for a 90 degree waveguide being optimal. This is why I've stuck with this pattern through all of my designs.
keyser said:
This has been and will continue to change. Poeple aren't buying that BS anymore - either physically or euphemistically.
keyser said:
Floyd Toole and Sean Olive claim that they can tell you what 95% of people will think about a loudspeaker just from measurements. Of course you COULD BE in that 5% category! In which case I am very sorry, because you will always have to pay more for being so unique.
I firmly believe that measurements can get 95% of the listening experince and that I can tell from the measurements what a speaker will sound like to within that 5% variance. I could not differentiate two speakers that were very close, thats true, but the fact is that speakers are seldom very close. There is usually a vast difference.
keyser said:
The vast majority of people who have bought my speakers do so without ever having heard them. I should and probably will give a money back guarantee because no one has complained. Indeed the testimonials speak for themselves.
yeah I also realized that I miscalculated the p values from my guesses, should have just looked at the program. I did better than chance in my 20 trials attempt. Looks like, while maybe not a perfect 10 out of 10, I managed to do so as well.
The reason why I suggested that the noise in the system would make distinguishing the difference difficult is that if the threshold for hearing the distortion would be if the volume of the system puts the distortion below the noise floor. For instance, if I listen at 60db's, and my noise floor is 40db's, then it easily would. I made those numbers up to make a point, as I've not taken the time to measure my computer room.
The reason why I suggested that the noise in the system would make distinguishing the difference difficult is that if the threshold for hearing the distortion would be if the volume of the system puts the distortion below the noise floor. For instance, if I listen at 60db's, and my noise floor is 40db's, then it easily would. I made those numbers up to make a point, as I've not taken the time to measure my computer room.
- Status
- This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.