New Ideas in ABX Double Blind Testing
ABX Home ABX Books ABX Periodical ABX Publications

ABX Double Blind Test on a CD -by Arny Krueger
Recordable CD's facilitate doing ABX comparisons without extra ABX equipment. CD's can be custom-produced as functional blind, level-matched tests. The listener is presented with a CD that contains several tracks of the same program material. Some of the tracks have been processed to create a potentially audible effect, and others are unprocessed. To make the test blind, tracks are not identified or readily identifiable as to their content, except by listening for the sound of the specific processing being tested. To make the test fit the ABX format, two reference tracks are provided and clearly identified as to their content.
Obviously, this technique is best for potentially audible effects that can be created by digital processing. It is also good for audible effects that can be properly presented in a recording.
Kinds of audible effects that this technique readily applies to include:
  1. Perceptual coding
  2. Nonlinear distortion
  3. Linear distortion
  4. Dynamic range modification
  5. Detection of masked program material
Kinds of audible effects that may be presented as a recording include:
  1. Room acoustics presented by a binaural recording.
  2. Autosound system comparisons presented by a binaural recording.
Kinds of audible effects that seem difficult or impossible to apply it to may include:
  1. Sound due to physical components such as amplifiers or wires.
  2. Subtle "problems" of many kinds - small amounts of jitter, D/A converters, etc.
To make CD's of this kind you need access to the following:
  1. A means for obtaining or creating digital program material.
  2. A digital audio editor
  3. A digital means for performing some kind of alteration to the program material
  4. A CD burner
Here is a procedure to use for doing this:
  1. Obtain digital program material from the web, by digitally reading an audio CD ("Ripping"), by transferring digital program material from a DAT using a AES/EBU or SPDIF digital audio interface card, or by real-time recording using microphones and a high quality sound card. This material will then be on the PC as a .WAV file.
  2. Create a copy of the program material that has been processed by the means you wish to evaluate.
  3. Ensure that the basic level, timing, etc. characteristics of the processed and unprocessed file are the same to the extent that a casual observer is not going to tell the difference between the files, except by the audible characteristics of their content.
  4. Create a Audio CD of the files, with the unprocessed and processed files first, and then a random selection of unprocessed and processed files following.
  5. In making all these copies, don't forget to respect the creators' copyrights. Most material is probably okay to use in personal listening tests, but don't sell your AB test CD's. To be sure what's okay, check the copyright law, with the copyright holder, or with an attorney.
Here is an example: We wish to examine the audibility of MP3 coding. A CD track is provided as being one that tends to make the effects of MP3 coding most audible such as the first 20 seconds of "We Shall be Happy" which is track 1 of Ry Cooder's CD entitled "Jazz".
  1. The CD track is "ripped" using CD Worx, a shareware program that can be downloaded from the web.
  2. A critical 20 second passage of this track is selected using CoolEdit96, a shareware program that can be downloaded from the web.
  3. The passage is encoded into a MP3 file using MP3 Compressor, a freeware program that can be downloaded from the web.
  4. The resulting MP3 file is decoded back into a .WAV file using L3Dec, a shareware program that can be downloaded from the web.
  5. The processed .WAV file is analyzed and editing using CoolEdit96 as necessary to ensure that its length, etc. have not been changed to make it readily detectable, other than by listening to program material.
  6. A random sequence is developed by flipping a coin 16 times and writing down the results. This information is kept secret from the listeners.
  7. The unprocessed and processed .WAV files are recorded on the first and second tracks of a recordable CD.
  8. The random sequence from (6) is used to additionally re-record the processed and unprocessed files.
  9. The resulting CD is QC'd to ensure that it performs as desired.
  10. The CD(s) are sent out to the listeners. If you fear collusion between the listeners, you may make each one a unique CD with a different random sequence of tracks.
  11. The listeners record their conclusions about the sound quality of the randomly sequenced tracks on the CD, as compared to the reference tracks, and return their conclusions to you.
  12. You analyze the listener's decisions for correctness and statistical significance in accordance with usual ABX procedures.
ABX Home ABX Books ABX Periodical ABX Publications
Send questions or comments on the ABX Web Page to David Carlstrom.
Text by Arny Krueger. Web Page Created by David Carlstrom. Last Modified: 1/26/2011