Can you tell original file from tube amp record? - test

[scottjoplin]

I've heard Americans have difficulty with British sarcasm

 

[mmerrill99]

But will Henry the Horse be there?

 

[DPH]

I've heard Americans have difficulty with British sarcasm

This Yank is having zero problems here.

 

[PMA]

BTW, the DacMagic+ is not bad.

 

[Pano]

You should understand that your ABX results were invalid, right? Why? Because after you did the ABX test which produced a null result (& people interpret this to mean no audible difference could be heard), you then did a sine wave test which showed that, actually it was your equipment that was at fault & the first interpretation of your ABX results were now incorrect - in fact your results were invalid. This sine wav post test exercise was in fact a way of checking the ABX test run by the people who participated & it revealed that most of the people returning their impressions or ABX results don't have a setup (or they themselves ) are not capable of hearing known audible differences incorporated in the Sine wav test.

OK, thank you for explaining you reasoning. But I will have to say that it is false reasoning
.
My ABX test was valid for the equipment used in the test. This is equipment that I normally use to listen to music, not something special for the test. With 4 different DACs, 1 headphone, 2 earbuds, coax speakers and a set of laptop speakers I could not hear the distortion on the music or the sine waves. That's valid, because it's what I have currently been using to listen to music. When I listen to music under normal circumstances, I can't hear the distortion. Valid test.

After measurement showed that my headphones are down 9-10dB at 2K, and that Grado headphones have a strong peak right there, I figured that was the reason I couldn't hear it. That proved to be the case. Again, valid.

However after a new EQ for the Fostex headphones, I was able to hear the H2 of the distorted sine. But I still was not able to consistently identify distortion in the other files. Also a valid test.

If your objection is that the ABX isn't valid for any and all audio gear and listeners ears, that seems pretty obvious for any test like this. That was never a claim of the test. Nor was the test proposed as definitive. The test simply asks if we can hear the distortion on the gear we choose to use. Mooly often chooses to use his laptop speakers and has a history of picking out differences. Does that invalidate his results because he isn't using "good" speakers?

Nothing you've posted invalidates the test. The Foobar ABX protocol works just as it should. The equipment used is normal, day to day listening gear. Only taking special measures allowed me to hear the tone. You claim that "most of the people returning their impressions or ABX results don't have a setup (or they themselves ) are not capable of hearing known audible differences incorporated in the Sine wav test." Yes, that's the point of the test. However others did have the equipment and/or the ears capable of resolving the difference. We can't throw out their results as invalid. The test did exactly was it was intended to do - allow us to discover whether a certain distortion profile was audible on our normal audio setups.

I see an argument here that is all too common on these pages and others. "Your gear is not good enough" or "Your ears are not good enough." Implying of course that superior people with superior gear are the true arbitrators of what is actually audible. Lesser people need not apply.

 

[cliffforrest]

Eruditely argued, Pano.

 

[mmerrill99]

OK, thank you for explaining you reasoning. But I will have to say that it is false reasoning
.
My ABX test was valid for the equipment used in the test. This is equipment that I normally use to listen to music, not something special for the test. With 4 different DACs, 1 headphone, 2 earbuds, coax speakers and a set of laptop speakers I could not hear the distortion on the music or the sine waves. That's valid, because it's what I have currently been using to listen to music. When I listen to music under normal circumstances, I can't hear the distortion. Valid test.

After measurement showed that my headphones are down 9-10dB at 2K, and that Grado headphones have a strong peak right there, I figured that was the reason I couldn't hear it. That proved to be the case. Again, valid.

However after a new EQ for the Fostex headphones, I was able to hear the H2 of the distorted sine. But I still was not able to consistently identify distortion in the other files. Also a valid test.

If your objection is that the ABX isn't valid for any and all audio gear and listeners ears, that seems pretty obvious for any test like this. That was never a claim of the test. Nor was the test proposed as definitive. The test simply asks if we can hear the distortion on the gear we choose to use.

OK, it's a case of defining the objective of the test & then strictly staying with this stated goal. if the goal for this test had been stated as what you say it was I would agree with you but here's what PMA said when he released the results & I saw nobody contradict or amend his stated conclusions. Nowhere in this quote is there mention of the possibility that some equipment & possibly some members are not capable of differentiating known audible differences

The results indicate that the sound difference between the test files was not very big (= not easily detectable in a DBT), but might have been detectable for some participants.

The hybrid amplifier was chosen for this test for two reasons:
tube gain stage has “typical” distortion profile with decaying amplitude of harmonic components and only the 2nd (and 3rd questionably) harmonic is above possible audibility threshold on pure tones,
class A transistor power buffer is free of cross-over distortion and has very low distortion in absolute terms, below 0.001%. Its output impedance is fairly low 0.23 ohm, which should be low enough not to modulate frequency response by the speakers connected.

It seems that distortion of the amp under test, which was H2 up to about 0.7% for the test file amplitudes, is not easy to tell by ear on the music samples used. In my opinion, the so called “tube sound” is not the result of just non-linear distortion, but rather of the high output impedance of tube power amplifiers in combination with often not very good S/N ratio. That's why I have used the class A solid state output stage, to avoid high output impedance influence, and why I tried to achieve the highest possible clean dynamic range of the amplifier. The goal was to concentrate on distortion itself.

If I understood him correctly, Scott Wurcer talks about sighted tests "requires a lot of integrity and discipline from the participants" & there is usually a great forensic examination of any reports of audible differences which jar with 'received wisdom' but I don't see any such examination of these null results - they are accepted as of sufficient quality & validity that PMA & others generalize the results - no examination of the test itself is usually entered into.

This is the first time I have seen a follow up test & it wasn't initially intended as a test of the participants, rather PMA put it out there thinking that it would be easily audible - "I made this small test to show that though it is quite easy to tell the sound difference on the pure sine tone, it is not so even on a single tone of a relatively spectrally simple instrument like the recorder flute is." Nothing at this stage was mentioned about testing the participants & system with known audible differences.

Only sometime days later when it became clear that only one person (Mooly, I think) could show that they were able to audibly differentiate the sine wav distortion, did PMA actually posted

My problem is that I automatically supposed that users make tests of their systems. I can see that it is not the case so I think that you know what kind of conclusions I have made for myself. On the other hand, I think it may be a positive lesson for many participants, because shared experience almost never works, one has to go through himself and then it is forever. So, if 2 - 3 users have learnt something from this exercise, to me it was worth efforts.

And later he seems to realize this important point:

I think that there should be (I tend to say must) an audible difference between the sine waves that I posted. If not, I would say it indicates to sound device not suitable for the test, or hearing loss, or impatience to find appropriate listening level for the sine test. If it was the first (bad sound device), then I would say the test cannot bring a meaningful result.

So you are directly contradicting the originator of the test & his posted statements .

Again if your definition of the objective was correct then why don;t you object to those who try to generalize the results into some fact - ScottJoplin already tried this & I often see null ABX results being used to support a case made that the accumulation of null results indicates that there is no actual audible difference, again no mention of the quality of the participants/systems/test itself.

Do you find this to be correct Mooly often chooses to use his laptop speakers and has a history of picking out differences. Does that invalidate his results because he isn't using "good" speakers?

You seem to be mixed up on a number of fronts - positive ABX results in themselves are a strong indication that he hears an audible difference. Now if you are making some case that his equipment introduces or amplifies distortions which then allow him to easily audibly differentiate differences then you are going down the same road as DF96 but it is a possibility & would make his results invalid

Nothing you've posted invalidates the test.

I don't even need
a post PMA already stated the same as me on this matter

The Foobar ABX protocol works just as it should. The equipment used is normal, day to day listening gear. Only taking special measures allowed me to hear the tone. You claim that "most of the people returning their impressions or ABX results don't have a setup (or they themselves ) are not capable of hearing known audible differences incorporated in the Sine wav test." Yes, that's the point of the test.

No, it wasn't the point of the original test - as PMA stated "The goal was to concentrate on distortion itself." between a signal that had passed through a tube/SS hybrid amplifier Vs the original

However others did have the equipment and/or the ears capable of resolving the difference. We can't throw out their results as invalid.

I never suggested that - valid results are valid results & should be accumulated - just as invalid results are invalid results & should be discarded

The test did exactly was it was intended to do - allow us to discover whether a certain distortion profile was audible on our normal audio setups.

Doesn't matter how many times you state it was a valid test, PMA & I agree with him, states that certain of the results are not meaningful - "the test cannot bring a meaningful result."

I see an argument here that is all too common on these pages and others. "Your gear is not good enough" or "Your ears are not good enough." Implying of course that superior people with superior gear are the true arbitrators of what is actually audible. Lesser people need not apply.

No, again you are not making any sense, just above you stated that the test was to discern if people's gear/ears were "good enough" & now you are demeaning that objective in what you just said. I have made no mention of expensive gear/or golden ears - none of what I say is about that - it's about understanding the test itself, it's objectives & possible conclusions that can be drawn

 

[turk 182]

Pavel i'm not used to using Foobar (and still not sure if there's a setting for it) i think what was happening was Foobar restarting a short play list.

i guess the dsp eq doesn't work in ABX .

the only way i could hear the difference in the sine files was to use the EQ to bring the difference between the fundamental and harmonic to within 30 db but it's hard to tell what degree i can resolve without ABX. come to think of it would that allow user's to confirm to what level of resolution there hearing has...or have i had to much scotch...
conclusions?...

 

[Pano]

have made no mention of expensive gear/or golden ears - none of what I say is about that -

You're not convincing anyone. See below:

it revealed that most of the people returning their impressions or ABX results don't have a setup (or they themselves ) are not capable of hearing known audible differences incorporated in the Sine wav test

I.E., they and their equipment aren't good enough. And yet some passed. Strange.

 

[Pano]

it's like asking people to evaluate the photographs from two different cameras but not knowing if they are looking at these pics on mobile phones, large screens, printouts on inkjet printers, laser printers, etc - the variety & unknown quality of the viewing process is part of the problem but there are other potential issues that should be resolved as far as possible

No, it's not like that. I would be if no one reported what they were using to listen to the files, but most do report. Thus it's like people saying "I can't see the jpeg artifacts on my iPhone 4 screen" and "I see them perfectly well on my calibrated 70" Sony 4K monitor". The listening test here is not done without any reporting of the gear involved. And not all the people who could pass the test had great equipment. That's an interesting find (and is true of some other tests)

If you really want to object to tests like this, object to all those who chime in claiming to hear a difference with no proof whatsoever. That's the lack of protocol that should be criticized. That's misleading.

 

[mmerrill99]

Piano, that's all you have in reply to my post?

Nothing about how you started that was the purpose of the test - to find out if people/their equipment was capable.

Nothing about how you directly contradict PMA's post where he states that a lot of the results were "not meaningful results"?

 

[Pano]

No, because none of that is valid. It's in my previous post.

 

[mmerrill99]

No, it's not like that. I would be if no one reported what they were using to listen to the files, but most do report. Thus it's like people saying "I can't see the jpeg artifacts on my iPhone 4 screen" and "I see them perfectly well on my calibrated 70" Sony 4K monitor". The listening test here is not done without any reporting of the gear involved. And not all the people who could pass the test had great equipment. That's an interesting find (and is true of some other tests)

If you really want to object to tests like this, object to all those who chime is claiming to hear a difference with no proof whatsoever. That's the lack of protocol that should be criticized. That's misleading.

I would have equated this blind test to have equal validity as any sighed test except that we know most of the results are "not meaningful" as PMA at.

If you want to try to make more of them then let's see your argument & logic?

 

[mmerrill99]

No, because none of that is valid. It's in my previous post.

So let me get this straight - your reply to my post is already in the post I replied to? That's some trick - remind me how you do that again - you can reply to a post before the post is even made?

 

[Pano]

Don't be silly. I already covered your objection in a previous post. I didn't need to do it again.

 

[Pano]

Mmerrill: I don't see anything in your objections other than you find the test inconclusive because we don't know the details of people's hearing or playback gear. That seems a strained objection. There was no claim that this is a definitive scientific test. It's simply a test asking if forum members can hear a difference in a blind test. Some could, some could not. Some explained the reasons that they might not have heard it.

You claim the tests are invalid for reasons that have little or nothing to do with the tests.
You also have not reported your results. That makes your objections suspiciously weak. If you truly want to show that the ABX test is not a valid measure of distortion audibilty, take the test and pass it. You'll then have firm ground to stand on.

 

[indra1]

Because misinformation is being posted which needs correcting for the benefit of all. Understanding tests & how to use them correctly, their pitfalls & issues, would, I imagine be of some consequence on DIYAudio forum?...

The way I see the situation is that the fruit test was a request to the diy community by Pavel to gain an opinion on his particular setup. I also understand your stated opinion on the lack of elements to make the test result more universally valid.

However, Pavel is of the opinion that the result was good enough for his particular purpose, no point for him to ask other members do any more than what he needs. Very few members will participate in a complicated test. Also the test result is by no means capable to benefit all, not everybody is interested in any tube stage whatsoever.

You can dismiss the fruit test if you do not find it useful for you. You are also allowed to open a new thread if you want a test for your own needs or to disseminate correct information for all to benefit.

 

[johnego]

Comment by indra1 to mmerrill99:

I also understand your stated opinion on the lack of elements to make the test result more universally valid.

Mmerrill99 is probably honest in his objective to educate people on the real meaning of the ABX by giving 'critics', but it doesn't seem to work for two reasons; first, very few knows what he knows about the issue such that people will be confused of what he is trying to say; second, the approach is highly questionable.

This isn't a dissertation project; no need objective and background statements. This test is just a random event where at the end of it we will try to draw some conclusions. The more detailed the test, the more useful conclusions can be gathered.

Conclusions can be numbered and people can comment on its validity. The test is always valid. The conclusion can be invalid.

 

[PMA]

The way I see the situation is that the fruit test was a request to the diy community by Pavel to gain an opinion on his particular setup.

Hello Indrawan,

yes in a way, however the main amplifier in my audio system is this one:

PMA CFA current feedback amplifier

The fruit test is a result of my long term attempt to investigate how distortion profile is audible on real music samples. I was doing some tests with mathematically added distortion which showed high tolerance of human ear to just only pure non-linear distortion added.
The distortion of the open-loop tube circuit used is very close to a pure mathematical polynomial distortion (in contrary to multi-loop feedback solid state circuits), so that's why I used it as a next step. During the tests on myself with a tube link gain stage, I was surprised that I could not tell the difference in the ABX test on any music sample. So I decided to prepare a test and ask the audience at diyaudio what they can hear and asked if they could possibly add an ABX report. The first test was a DAC==>ADC loopback vs. inserted tube link gain stage
Do you like tube distortion? - listening test
but because of possible or existing objections on DAC==>ADC contribution to a possible sound difference I have decided for the current test, where we compare original rip vs. complete hybrid tube-SS amplifier. Again, I asked the participants if they hear the difference and asked if they could possibly add an ABX report as an option.

I do not think the hybrid amplifier would remain as my main amplifier, as it it has not enough output power. However, I have no objections to sound when it plays within its power limits.

 

[PMA]

Conclusions can be numbered and people can comment on its validity. The test is always valid. The conclusion can be invalid.

Hello John,

I agree and I would like to add that this was not intended as a scientific test. It is what it is, an off-line test for the public forum, but even as such it is helpful and interesting to me and I appreciate and thank everyone who have participated in it.