As I have stated very early on this thread (post #11), it (What kind of evidence do you consider as sufficient?) is the one that sellers / shills object the most to.
I thought that once folks were talking about the ITU specification there might be an agreement on a test. I've often stated that I'm not insistent on ABX but now this seems to be just another attempt to discredit blind testing. I suspect much of the industry CODEC research is proprietary so the details of how the testing was done can't be quoted here. I suspect it followed the ITU spec pretty closely and was blind possibly even ABX.
I would be happy if there was an agreement on a test that simply showed how much work and care it takes and how subtle the differences are (if they exist at all).
Last edited:
Sure. But, nobody has created such a test for people to see if it works. Without a prospective test to validate, you would be wanting people to come to agreement on a hypothetical test. I still would suggest what needs to happen is somebody needs to come up with a test and start testing people. If it proves to be sensitive enough and if test subjects say it was fair then that is the point at which you might see people start to agree on it. But, it takes money for a programmer to make a test for a researcher to start using, etc, etc. It's not happening. All IMHO, of course.
Last edited:
Sorry but that is true for nearly every term. It does not help to simply ask other people about what they believe those terms should denote. You must present an example (or call it a reference) that has the specific virtue you would like to denote with a certain attribute.
So if you meet with Max Headroom you should ask him for an example that triggers the "vibe and groove" feeling and other that do not and then you´ll start to know what the meaning is. If you then both agree to use it further or if you agree to another term that seems to be even more appropriate is another question.
There is a set, even with accompanying sound samples (PEQS) free for download from the EBU:
EBU Technology & Innovation - EBU CDs now online
and the according handbook containing the informations about attributes and sound samples:
https://tech.ebu.ch/docs/tech/tech3287.pdf
the other set more related to technical artefacts is the SQAM programme:
EBU Technology & Innovation - Sound Quality Assessment Material recordings for subjective tests
contains links to the sound samples and the handbook.
Generally the PEQ set is already quite specific and tries to address certain details of a sound event.
It is not so much related to attributes describing the so-called main audio quality. Usually a set of 9-10 attributes is sufficient to describe differences wrt the main audio quality (overall) between DUTs in a meaningful way, but to agree on such a set needs usally a meeting combined with a listening session to work it out.
Those samples date from 1988 and are actually 44.1/16 and possibly undithered (I don't know for certain maybe Mark can tell). Picture of track 12 at the decay of a flute tone. Hardly what we want to compare ESS to Benchmark.
Attachments
Exactly my point! What I said was that an agreed set of words with agreed meaning is needed. I didn't discuss how that would be done, but sound samples is a good idea.
Of the two EBU links - at a quick read the first is ok for live (orchestral) music reproduction, which is probably the intent, so offers a partial solution. The second doesn't seem to agree terminology.
Secondly - I am answering your original query, in part.
What I would need is:
Agreed and understood terminology.
An agreed DBT testing method with adequate statistical relevance - ie enough tests and enough participants, suitable source material, listening environment, etc. A lot of variables there to control.
But in reality - I'm not selling to people, so what's most important is what I think of a system, and it's effectiveness for my use cases and my source material. I don't have to please anyone else...
It's not perfect, but you're throwing the ABX protocol of Foobar pretty far under the bus, many of which are a limitation to ANY test protocol. This is why it's important to accept that one needs to play around with it for a while to become familiarized. Would re-sampling be a nice feature? Absolutely. Would expanding to 2-AFC/3-AFC/4-AFC and other various protocols be nice? Sure. It is what it is, and can provide plenty of useful information for an individual's needs. Might the tool be too insensitive for the minutia? Yes--and I'll happily argue that almost any test protocol an individual puts together is, unless they go to heroic levels of experimental design. I find it great for trying a few different DSP curves for digital crossovers, which are dB changes, not PPM changes.
Testing whether X opamp sounds better in circuit or two DACs that are, both better than -90 dB SINAD under their nominal operating conditions? Give me a break, that's asking for an area-under-curve (AOC) of better than 0.95 on a receiver operator curve (ROC), if not notably better. It's hard to build a decent prior plausibility, but remember that a lot of people struggle to hear differences that are in the -50 dB range, and we're talking a 1/100th the energy at -90 dB.
It's nice and pretty to talk about the last echelon of human perception, which may be of academic benefit, but for practical purposes in one's own living room, all this back and forth is below the sensitivity and selectivity that we can experimentally set up (and maybe even universally, as humans limit the experimental performance envelope).
Might seem like being stubborn but it´s simply a question for clarification. As you can read in this thread Waly for example already considers the claim "be able to detect an audible difference between two mains cables" as extraordinary while according to your criterion it is not.
Not at all. If you read most of what I post about Foobar ABX it is to the effect that it is very close to fine. A few minor modifications should fix it. Then people like mastering engineers who tried it and say it doesn't work will change their minds and say it does work. Big difference in getting people to be able to come to agreement on it or something like it.
The post you are questioning here was only an imperfect attempt to start to explain something that is complicated and not fully explainable in a single post. Sitting in my living room is very different from needing to perform perfectly on a test 20 times in row. In a very relaxed setting it is easy to hear many familiar yet subtle sounds that make a big difference to how I enjoy sound and are definitely part of what I want and expect from a system.
Last edited:
Once again, uselessly stirring the pot and nitpicking on an example. But even so, except for pathological cases of a bad/improper cable construction, the "sound of mains cables" falls into the hidden variables/information category. According to the proponents, there is "something" that makes the sound of a mains cable (from the equipment to the mains outlet) dominant (so that it could be changed by changing the cable) compared to the wiring from the mains outlet to the power generator.
Now you are going to stubbornly ask what is a "bad/improper cable construction", isn't it?
Mains cables are a borderline case just like power conditioners. The extraordinary comes with the claims made for how they work. In fact one should not dismiss out of hand that a poorly designed or shielded amplifier might show a subtle effect. Why a manufacturer would sell a $15000 amplifier not prepared to get its full performance out of the box and plugged in is beyond me.
There are plenty of other cases far more obvious.
Last edited:
I have found that it is simply because they dont know how or they dont know what they dont know.
For example, I have spent days and hours debugging EE's prototypes with ground-loops/noise between the test system and DUT and grounding issues within the DUT. They just are not aware of practical issues not taught in college.
Shielding? How does that work? When and how? Does it affect grounding?
Emphasis is on circuit theory and CAD and mfr spec sheets. I have a talk today with two newly minted EE's about some of the pit falls in implementation. Should be interesting for both of us.
THx-RNMarsh
@Waly & scott wurcer,
thanks for the reminder as i actually missed the "hidden variables/informations" and the "etc." parts.
What does the term "hidden variables/information" really means in this context? And what about the "etc."? If it was inserted to cover claims being contradictionary to first principles and cases where really overwhelming experimental evidence exists, i can agree otherwise it needs clarification too. (stubbornly i know )
thanks for the reminder as i actually missed the "hidden variables/informations" and the "etc." parts.
What does the term "hidden variables/information" really means in this context? And what about the "etc."? If it was inserted to cover claims being contradictionary to first principles and cases where really overwhelming experimental evidence exists, i can agree otherwise it needs clarification too. (stubbornly i know
)If some one replaced a power cord with one using twisted wires and shields..... and that some one detected a change..... those wishing to make and sell similar power cords will claim you too Might detect a change. Because there is no way to know if only one person's situation or more than one will detect any change. The only way to know is to buy it and use it in your system.
Many products are sold/marketed this way.
THx-RNMarsh
Many products are sold/marketed this way.
THx-RNMarsh
An example would be that the data on a file somehow "remembers" its jitter history even though the bits are identical. In other words the claims involve denial of basic information theory which in turn is based on the second law of thermodynamics (both use the concept of entropy).
I borrowed the hidden variables term from physics an example might be a claim of a hidden unknown process that can separate signal from noise with no a priory knowledge, in other words Maxwell's demon.
This is in contrast to statements that are simply wrong like a linear time invariant circuit of ideal R's and C's can create new frequencies.
As I said yesterday simply saying my #6 power cord twisted in a certain way might make your PA sound better is not in itself an extraordinary claim. I also think that many if not almost all of these claims would not pass a blind test.
I have to thank you for finding that EBU spec and I'm surprised that there were no comments. If this was somewhat a standard the fact that perceptual tests were done with undithered 44.1/16 sound clips should give one pause.
Last edited:
- Status
- Not open for further replies.