Blind A/B crossover switching script for Camilla DSP

[Dave Bullet]

I've developed a simple script to apply an A/B test between crossovers.

I did this to remove any confirmation bias I have when trying to fine tune or "voice" a crossover. It also allows a group to evaluate during the same playing session.

I'm using ArchlinuxARM (on a Raspberry Pi 3) but any Linux distro should have the required packages.

Dependencies

• flite - text to speech so the test run can be "spoken" allowing test subjects to mark which crossover they think they hear
• websocat - CLI tool to send CamillaDSP instructions to switch config
• MPD - this assumes you are using MPD as a music server. You can comment out any "mpc" commands if not. These simply reduce the playback volume of MPD so you can hear the text to speech

The crossovers (YAML files) and scripts are all assumed to be in the same directory.

#!/bin/sh

# Performs an AB random load of 2 crossover configs
# running the number of times requested for the period
# requested between switches.  Uses a Text to speech package
# (flite) to "speak" the iteration being switched into the
# default ALSA device (e.g. "One, Two, Three etc...".
# It will drop the volume in MPD so you can hear the change
# Assumes the default ALSA device is a dmix device, thus
# will play merged in with the current song.
# When finished, it will restore the original Camilla DSP loaded
# The output for each test is written into /tmp so it can be
# checked after the test is done
#
# requires these packages for text to speech:
# festival
# festival-english
#
# or
#
# flite (quicker to run - recommended)
#
# Arguments (all are mandatory and no command line error checking):
# $1 = number of tests - that is times crossover config is to be switched
# $2 = duration of play between switches
# $3 = Y or N = Y repeats the section played each switch, N continues song
#      whilst switching
# $4 = fully qualified filename for Camilla DSP crossover #1
# $5 = fully qualified filename for Camilla DSP crossover #2
#
# eg:
# ab_test.sh 5 5 Y bps_r6.yml bps_r6_mid_only.yml
#
# runs 5 tests (switching between the 2 YAML files 5 times) with a 5 second period listening
# to each switch.  Y = reset the position (in MPD) back to the time the test started
# allowing subjects to hear the same "sample period".

OUTPUT_FILE="/tmp/ab_test_"`date +%Y-%m-%d\-%H\:%M\:%S`
#echo $OUTPUT_FILE
BASEDIR=$(dirname "$0")

if [ $3 == "Y" ]
then
    CURRENT_SONG_POS=`mpc | grep playing | cut -d' ' -f 5 | cut -d'/' -f 1`
fi

# save the config so we can restore it at end of test
ORIGINAL_CONFIG=`$BASEDIR/get_config.sh`

# To be doubly blind - we take the 2 crossovers in the
# command line and randomly assign them as the A and B
# so the person / caller has no idea which one is which
UPPER=2
LOWER=1
DIFF=$(($UPPER-$LOWER+1))
R=$(($(($RANDOM%$DIFF))+$LOWER))

CURR_VOLUME=`mpc volume | cut -d":" -f 2 | sed 's/\%//g'`

if [ $R = 1 ]
then
    FILE_A=$4
    FILE_B=$5
else
    FILE_A=$5
    FILE_B=$4
fi

for i in `seq $1`
do
    DIFF=$(($UPPER-$LOWER+1))
    R=$(($(($RANDOM%$DIFF))+$LOWER))
    if [ $R = 1 ]
    then
        TEST_FILE=$FILE_A
        TEST_CONFIG=A
    else
        TEST_FILE=$FILE_B
        TEST_CONFIG=B
    fi
    #echo "$i" | festival --tts
    echo "Test #$i: $TEST_CONFIG: $TEST_FILE" >> $OUTPUT_FILE
    mpc volume $(expr $CURR_VOLUME - 50)
    flite -t $i
    $BASEDIR/switch_config.sh $TEST_FILE
    mpc volume $CURR_VOLUME
    if [ ! -z $CURRENT_SONG_POS ]
    then
        mpc seek $CURRENT_SONG_POS
    fi
    sleep $2
done

$BASEDIR/switch_config.sh $ORIGINAL_CONFIG

 

[Dave Bullet]

Dependent scripts follow

get_config.sh
Allows the test to revert to the crossover at the start

#!/bin/sh

SCRIPTPATH="$( cd -- "$(dirname "$0")" >/dev/null 2>&1 ; pwd -P )"
source $SCRIPTPATH/websocat_cli.sh

echo '"GetConfigName"' | $WEBSOCAT | cut -d":" -f 4 | sed 's/\"//g;s/\}//g'

switch_config.sh
Loads the desired config. Assumes /opt/camilla-dsp location if no path qualified (and assumes script in same location

#!/bin/sh

SCRIPTPATH="$( cd -- "$(dirname "$0")" >/dev/null 2>&1 ; pwd -P )"
source $SCRIPTPATH/websocat_cli.sh

# see if the config file has a path character
# and if not - add the path
if [[ $1 == *"/"* ]]
then
    # already has path
    CONFIG=$1
else
    # add path
    CONFIG=$SCRIPTPATH/$1
fi
#echo $CONFIG
echo '{"SetConfigName": "'$CONFIG'"}' | $WEBSOCAT
echo '"Reload"' | $WEBSOCAT

websocat_cli.sh
Common "source" script to set the websocat command line and Camilla DSP listener port. Assume localhost configured for Camilla and port 7070

#!/bin/sh

WEBSOCAT='websocat ws://127.0.0.1:7070 -t'