BruteFIR setup for windows?

Status
This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.
Too bad it does not process files in parallel, but it write to output all at once at the end, so is unsuitable for real time (i.e applying FIR to streamed pcm from Qobuz or Tidal).

With Large WAV files (600 MB) it takes few seconds on my i5 with windows 10.

Any Idea?

I doubt a 600MB file is being processed in series twice by the fir's to completion before you get output. As others have suggested there are probably pipe related things you don't have control of. If you run the fir on a 600MB stereo file and pipe it to sox.play what is the latency?

The fir filter is implemented by the overlap and add FFT technique and will have some latency but I would think that would be < sec.
 
I doubt a 600MB file is being processed in series twice by the fir's to completion before you get output. As others have suggested there are probably pipe related things you don't have control of. If you run the fir on a 600MB stereo file and pipe it to sox.play what is the latency?

The fir filter is implemented by the overlap and add FFT technique and will have some latency but I would think that would be < sec.

,
Not sure to well understand what you are asking me to do, but:

sox -M -t sox "|sox.exe Adagio.wav -p remix 1,1 fir shelvingHigh.txt" "|sox.exe Adagio.wav -p remix 2,1 fir shelvingLow.txt" -b 24 Adagio_out.wav

takes 21 seconds, being Adagio.wav 680 MB and Adagio_out.wav is empty until the very last second, then it fills up all at once.

then, playing Adagio_out.wav is immediate.

If I run sox -M -t sox "|sox.exe Adagio.wav -p remix 1,1 fir shelvingHigh.txt" "|sox.exe Adagio.wav -p remix 2,1 fir shelvingLow.txt" - | player.exe

where player.exe is squeezebox server module that get the input from his stdin, it starts playing (not really, should need some format adjustment here, but it starts receiving input, ) after 21 secs.
 
The pipe operator is "|" and the chain effect operator is ":" .

I originally tried using the effect chain as it made more sense to me, but could not get the syntax to get it to work.

@phofman : could you provide an example of channel splitting, fir effect chain with unique fir to each channel, then merging back ?
 
,
Not sure to well understand what you are asking me to do, but:

I see that's not good it looks like SoX can't set this up efficiently without help.

You could recompile SoX from source with a stereo fir filter (like foo_convolve). The fir would be a .txt file with two entries per line. I doubt this solution appeals to you.

IIRC SoX supports Ladspa plug-ins, maybe there is one that can do this.
 
My fault, I did not realize you needed a different effect to each channel. I do not think sox can do that without the the piped multiple input feature.

IMO the reported behavior on windows is direct result of the way pipes work in that OS (e.g. Information Technology: An Introduction for Today’s Digital World - Richard Fox - Knihy Google ) - synchronous first all in, then all out. I have no Windows available to test now.

The piped input is a nice way on systems with standard pipes since all the processing runs in parallel. Sox itself is single-threaded - a longer serial chain would take longer.
 
My fault, I did not realize you needed a different effect to each channel. I do not think sox can do that without the the piped multiple input feature.

IMO the reported behavior on windows is direct result of the way pipes work in that OS (e.g. Information Technology: An Introduction for Today’s Digital World - Richard Fox - Knihy Google ) - synchronous first all in, then all out. I have no Windows available to test now.

The piped input is a nice way on systems with standard pipes since all the processing runs in parallel. Sox itself is single-threaded - a longer serial chain would take longer.


I was wrong, the problem is elsewhere, I should investigate where, but the command:

sox -M -t sox "|sox.exe Adagio.wav -p remix 1,1 fir shelvingHigh.txt" "|sox.exe Adagio.wav -p remix 2,1 fir shelvingLow.txt" -

produces an (almost) immediate output to stdout, so seems it works in parallel, even in windows.

Now I face another problem: how to feed the two pipes from the common stdin instead than from a file (use case is a stream form Qobuz or Tidal)

I tried this:

sox -M -t sox "|sox.exe - -p remix 1,1 fir shelvingHigh.txt" "|sox.exe - -p remix 2,1 fir shelvingLow.txt" -t wav -

that seems to work, but the result is a weird stereo file in output, completely out of sync


F:\SVILUPPO\06 - sox>sox --info stereo_in.wav

Input File : 'stereo_in.wav'
Channels : 2
Sample Rate : 44100
Precision : 16-bit
Duration : 00:00:20.00 = 882000 samples = 1500 CDDA sectors
File Size : 3.53M
Bit Rate : 1.41M
Sample Encoding: 16-bit Signed Integer PCM


F:\SVILUPPO\06 - sox>sox --info out.wav

Input File : 'out.wav'
Channels : 2
Sample Rate : 44100
Precision : 32-bit
Duration : 00:00:10.40 = 458752 samples = 780.19 CDDA sectors
File Size : 3.67M
Bit Rate : 2.82M
Sample Encoding: 32-bit Signed Integer PCM​


here the real command and the log:


F:\SVILUPPO\06 - sox>"F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\flac.exe" -dcs --force-raw-format --endian=little --sign=signed -- "F:\SVILUPPO\01 - SqueezeboxServer Plugins\musica campione\20_Sec\wav_16_044100.flac" | "F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe" -V6 -M -t sox "|"F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe" -V6 -t raw -c 2 -b 16 -r 44100 -e signed-integer -L - -p remix 1,1 fir shelvingHigh.txt" "|"F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe" -V6 -t raw -c 2 -b 16 -r 44100 -e signed-integer -L - -p remix 2,1 fir shelvingLow.txt" -t wav -c 2 -b 32 -r 44100 out.wav
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe: SoX v14.4.2
time: Feb 25 2017 23:16:31
compiler: gcc 4.9.1
arch: 1248 48 88 L
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe: SoX v14.4.2
time: Feb 25 2017 23:16:31
compiler: gcc 4.9.1
arch: 1248 48 88 L

Input File : '-' (raw)
Channels : 2
Sample Rate : 44100
Precision : 16-bit
Sample Encoding: 16-bit Signed Integer PCM
Endian Type : little
Reverse Nibbles: no
Reverse Bits : no


Output File : '-' (sox)
Channels : 1
Sample Rate : 44100
Precision : 32-bit
Sample Encoding: 32-bit Signed Integer PCM
Endian Type : little
Reverse Nibbles: no
Reverse Bits : no
Comment : 'Processed by SoX'

F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe DBUG effects: sox_add_effect: extending effects table, new size = 8
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe DBUG remix: 0:
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe DBUG remix: 1 0.5
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe DBUG remix: 0 0.5
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe INFO fir: 16384 coefficients
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe INFO sox: effects chain: input 44100Hz 2 channels (multi) 16 bits unknown length
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe INFO sox: effects chain: remix 44100Hz 1 channels (multi) 32 bits unknown length
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe INFO sox: effects chain: fir 44100Hz 1 channels 32 bits unknown length
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe INFO sox: effects chain: output 44100Hz 1 channels (multi) 32 bits unknown length
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe DBUG sox: start-up time = 0.022938
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe INFO formats: detected file format type `sox'
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe: SoX v14.4.2
time: Feb 25 2017 23:16:31
compiler: gcc 4.9.1
arch: 1248 48 88 L

Input File : '-' (raw)
Channels : 2
Sample Rate : 44100
Precision : 16-bit
Sample Encoding: 16-bit Signed Integer PCM
Endian Type : little
Reverse Nibbles: no
Reverse Bits : no


Output File : '-' (sox)
Channels : 1
Sample Rate : 44100
Precision : 32-bit
Sample Encoding: 32-bit Signed Integer PCM
Endian Type : little
Reverse Nibbles: no
Reverse Bits : no
Comment : 'Processed by SoX'

F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe DBUG effects: sox_add_effect: extending effects table, new size = 8
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe DBUG remix: 0:
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe DBUG remix: 0 0.5
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe DBUG remix: 0 0.5
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe INFO fir: 16384 coefficients
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe INFO sox: effects chain: input 44100Hz 2 channels (multi) 16 bits unknown length
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe INFO sox: effects chain: remix 44100Hz 1 channels (multi) 32 bits unknown length
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe INFO sox: effects chain: fir 44100Hz 1 channels 32 bits unknown length
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe INFO sox: effects chain: output 44100Hz 1 channels (multi) 32 bits unknown length
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe DBUG sox: start-up time = 0.021942

Input File : '|F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe -V6 -t raw -c 2 -b 16 -r 44100 -e signed-integer -L - -p remix 1,1 fir shelvingHigh.txt' (sox)
Channels : 1
Sample Rate : 44100
Precision : 32-bit
Sample Encoding: 32-bit Signed Integer PCM
Endian Type : little
Reverse Nibbles: no
Reverse Bits : no
Comment : 'Processed by SoX'


Input File : '|F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe -V6 -t raw -c 2 -b 16 -r 44100 -e signed-integer -L - -p remix 2,1 fir shelvingLow.txt' (sox)
Channels : 1
Sample Rate : 44100
Precision : 32-bit
Sample Encoding: 32-bit Signed Integer PCM
Endian Type : little
Reverse Nibbles: no
Reverse Bits : no
Comment : 'Processed by SoX'

F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe INFO sox: Overwriting `out.wav'
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe DBUG wav: Writing Wave file: Microsoft PCM format, 2 channels, 44100 samp/sec
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe DBUG wav: 352800 byte/sec, 8 block align, 32 bits/samp

Output File : 'out.wav'
Channels : 2
Sample Rate : 44100
Precision : 32-bit
Sample Encoding: 32-bit Signed Integer PCM
Endian Type : little
Reverse Nibbles: no
Reverse Bits : no
Comment : 'Processed by SoX'

F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe DBUG effects: sox_add_effect: extending effects table, new size = 8
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe INFO sox: effects chain: input 44100Hz 2 channels (multi) 32 bits unknown length
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe INFO sox: effects chain: output 44100Hz 2 channels (multi) 32 bits unknown length
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe DBUG sox: start-up time = 0.095743
F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe DBUG wav: Finished writing Wave file, 3670016 data bytes 458752 samples​


I'm stuck.
 
Last edited:
Just in case someone would like to have a look to the result (out.wav) Here the link to the zip containing it, the source and filter files I used in the command:


"F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\flac.exe" -dcs --force-raw-format --endian=little --sign=signed -- "F:\SVILUPPO\01 - SqueezeboxServer Plugins\musica campione\20_Sec\wav_16_044100.flac" | "F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe" -V6 -M -t sox "|"F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe" -V6 -t raw -c 2 -b 16 -r 44100 -e signed-integer -L - -p remix 1,1 fir shelvingHigh.txt" "|"F:\Sviluppo\slimserver\Bin\MSWin32-x86-multi-thread\sox.exe" -V6 -t raw -c 2 -b 16 -r 44100 -e signed-integer -L - -p remix 2,1 fir shelvingLow.txt" -t wav -c 2 -b 32 -r 44100 out.wav


sox.zip - Google Drive

Many thanks.
 
It occurred to me, that the issue you are having is exactly the same as those trying to use SOX to make an XO. The channel(s) are split, processed, then remixed. Hopefully they are still in sync at the end. The XO just uses a different sox effect and sometimes includes the fir effect.

This thread has a nice explanation of the issues and some script examples : SoX - implementing loudspeaker crossovers using Sox + LADSPA plugins .
 
Last edited:
It occurred to me, that the issue you are having is exactly the same as those trying to use SOX to make an XO. The channel(s) are split, processed, then remixed. Hopefully they are still in sync at the end. The XO just uses a different sox effect and sometimes includes the fir effect.

This thread has a nice explanation of the issues and some script examples : SoX - implementing loudspeaker crossovers using Sox + LADSPA plugins .

THX I'll have a look to that page. I'm giving a try to Brutrfir at moment and it works just fine in Linux.
 
multi channel realtime FIR in SoX - not

Sox can do multichannel FIR if the I/O is a file. The problem in real time is piping data into multiple processing paths and actually have it remain in sync for output mixing. I used a 2 way stereo cross over to generate 4 channels (LL, LH, RL, RH) to check if sync was possible.

There is a wonderful discussion on SoX for crossovers and pipes at SoX - implementing loudspeaker crossovers using Sox + LADSPA plugins and I had a chance to try some of those ideas. It's only to aid my understanding of sox as it seemed like a solution was so close. Spoiler alert.

The input pipe is progressively split into multiple pipes using the TEE command. The link above explains the issues with pipes and blocking and requires this structure. A "pipe-zilla" tree is created that could be fed from a single file or a soundcard (dsnoop) or another process and each process chain gets its own fifo.

Code:
mkfifo llo.sox 
mkfifo rlo.sox 
mkfifo lhi.sox 
mkfifo rhi.sox 
mkfifo split1.fifo 
mkfifo split2.fifo 
mkfifo split3.fifo 


# Notice a pattern?  You could add more FIFOs into this chain :) 
(tee <split1.fifo split2.fifo | cat >lhi.sox) & 
(tee <split2.fifo split3.fifo | cat >rlo.sox) & 

# "sox -M" actually reads from this, first 
(cat < split3.fifo > rhi.sox) & 

# Onesubscriber feeds multiple pipes: 
(sox $INPUT -p | tee split1.fifo | cat > llo.sox) &

I tried 3 different processing chains to create the output and compare the outputs.

The first (8a) uses R. Taylor's LADSPA plugins and it works well.

Code:
 sox -S -V3 -M -t sox \
	"|sox llo.sox -p remix 1,1 ladspa RTlr4hipass $XOfc  " \
	"|sox lhi.sox -p remix 1,1 ladspa RTlr4lowpass $XOfc " \
	"|sox rlo.sox -p remix 2,1 ladspa RTlr4hipass $XOfc  " \
	"|sox rhi.sox -p remix 2,1 ladspa RTlr4lowpass $XOfc " \
	-t wav stereoout8a.wav \

The second (8b) uses the sox built in filters and it works well.

Code:
 sox --multi-threaded -S -V3 -M -t sox \
	"|sox llo.sox -p remix 1v1 lowpass  -$XOorder $XOfc $XOq"q" lowpass  -$XOorder $XOfc $XOq"q" " \
	"|sox lhi.sox -p remix 1v1 highpass -$XOorder $XOfc $XOq"q" highpass -$XOorder $XOfc $XOq"q" " \
	"|sox rlo.sox -p remix 2v1 lowpass  -$XOorder $XOfc $XOq"q" lowpass  -$XOorder $XOfc $XOq"q" " \
	"|sox rhi.sox -p remix 2v1 highpass -$XOorder $XOfc $XOq"q" highpass -$XOorder $XOfc $XOq"q" " \
	 -t wav stereoout8b.wav

The third (8c) uses the FIR effect in sox and 16K tap FIR filters. It does not work and appears to be blocked. It seems the fir effect requires a file and does not work with pipes (error - can't open pipe error on ctrl-C to interrupt blockage). [Sigh] I think this is a dead end that requires code changes to the SoX fir effect. I'm going back to Ecasound and Brutefir.

Code:
 sox -S -V3 -M -t sox \
	"|sox llo.sox -p remix 1,1 fir $LowPassFir " \
	"|sox lhi.sox -p remix 1,1 fir $HighPassFir " \
	"|sox rlo.sox -p remix 2,1 fir $LowPassFir " \
	"|sox rhi.sox -p remix 2,1 fir $HighPassFir " \
	-t wav stereoout8c.wav \

The scripts used are attached and the FIR filters and music file are in a previous post BruteFIR setup for windows?
 

Attachments

  • soxfilter8a.sh.txt
    1.9 KB · Views: 35
  • soxfilter8b.sh.txt
    2 KB · Views: 32
  • soxfilter8c.sh.txt
    1.9 KB · Views: 26
  • fox in sox.jpg
    fox in sox.jpg
    25.5 KB · Views: 70
Last edited:
I thought that different processing for each channel (eq or fir) was the goal here. I was assuming that the DRC or EQ solution was unique to each channel hence the split, process and merge. Otherwise the same SoX effect or LADSPA plugin would be applied uniformly to an entire set of channels by default.

I did look for a LADSPA convolution engine that permits an external FIR file and could not find one. There are some impulse response based plugins that have internal presets for amp or room modelling but you can't specify your own file. Mostly I was just interested in what sox could do.

I still find Linux audio confusing because there are many places and ways to do the same function and often the documentation is lacking.
 
Last edited:
Status
This old topic is closed. If you want to reopen this topic, contact a moderator using the "Report Post" button.