Dynamic transcoding and convolution in Logitechmediaserver

Status
Not open for further replies.
Jump to Latest
S

Simofil

Member
Joined 2017
#1
YCLD - Dynamic transcoding and convolution in Logitechmediaserver

I'v always liked the idea to have the option to transcode AND apply convolution in LMS without many headaches. ATM there is C-3PO as a plugin for transcoding (but not optimal for convolution) and the hacked squeezelite SuperPlayer (I love it but it breaks the client-server paradigm where all the load should be on the server).

So I decided to make my own wrapping all the SoX + CamillaDSP logic.
I based it on a previous software which I made for batch audio file conversion (Recursive Audio Converter GUI) and I basically add to it the option to read and write from/to pipes and to read wav headers .

It is ugly, it is in beta and it has bugs.

Installation
Download and untar (Link at the end of the post)
Make sure to have OpenJDK >= 11 installed

Usage
Bad news! Fot the moment you'll have to edit your custom-convert.conf file. I know it is a pain in the *** and I'll try to help anyone but even for me this lms file is dark magic. Later I'll made a GUI (Desktop/web/mobile or whatever).
Before anyone asks me : no, I won't write an official LMS plugin, I'll start writing PERL code in my life only for at least double the hourly wage I take now as a dev. (what an ugly language)

First Example:
Code: 
flc flc * *
	# IFT:{START=--skip=%t}U:{END=--until=%v}
	[flac] -dcs $START$ $END$ -- $FILE$ | /usr/bin/java -jar /usr/local/bin/YCLD.jar --debug --input-dir stdin --output-dir stdout --output-bit 24 --output-rate NextSync --output-format flac --convolution /mnt/music/0000-EQ-FILES/LCD2
Explanation:
For whatever input flac file (also from Tidal or Qobuz), upsample it at double its current sample rate (ex 44100->88200) and then search in the convolution folder for an impulse named impulse{target_sample_rate}.bin/dbl/txt. In this case with a 44.1 input it will pick /mnt/music/0000-EQ-FILES/LCD2/impulse88200.dbl

Second example:
Code: 
flc dsf * *
	# IFT:{START=--skip=%t}U:{END=--until=%v}
	[flac] -dcs $START$ $END$ -- $FILE$ | /usr/bin/java -jar /usr/local/bin/YCLD.jar --debug --input-dir stdin --output-dir stdout --output-bit 1 --output-rate 5644800 --output-format dsf --convolution /mnt/music/0000-EQ-FILES/LCD2
Explanation:
It will take the input file, apply convolution (with pcm->dsd the impulse MUST be at 352.8kHz) and then convert to DSD128

The possibilities are quite infinite, you can even do DSD64 -> convolution -> DSD256 (internally it will do DSD64->64/352k->HF fir filter->your custom fir filter->DSD256)

Config
In the tar.gz you'll find the main jar and config/config.properties

Example with comment:
Code: 
[COLOR="SeaGreen"]#useless option in this case, useful when used as a batch converter. Dont' write 0 please, any value >= 1 is ok and won't make any difference[/COLOR]
Threads=12
[COLOR="SeaGreen"]#SoX path[/COLOR] 
SoXDir=/usr/local/bin/sox
[COLOR="SeaGreen"]#CamillaDSP path[/COLOR]
CamillaDSPDir=/usr/local/bin/camilladsp   
[COLOR="SeaGreen"]#where to write the debug file when --debug option is active[/COLOR]
DebugFile=/usr/local/bin/debug.log 
[COLOR="SeaGreen"]#Pre gain before apply convolution[/COLOR]   
PreProcessGain=-2  
[COLOR="SeaGreen"]#Useful only for batch conversion, keep it at 0 with LMS[/COLOR] 
Normalize=0
[COLOR="SeaGreen"]#when false you must pick a single impulse to apply, the sample rate of the filter MUST match the output sample rate. When true you can instead specify a directory where multiple convolution files with different sample rates to be applied. The impulses MUST be named with the following syntax "impulse"<samplerate>.<extension> . Example: impulse44100.txt, impulse192000.dbl. When you convert TO DSD the sample rate of the impulse must be 352.8 kHz.[/COLOR]
UseMultipleConvolutionFiles=true
[COLOR="SeaGreen"]#check CamillaDSP wiki, FLOAT64LE = *.dbl, FLOAT32LE = *.bin, TEXT = *.txt[/COLOR]
ConvolutionFileFormat=FLOAT64LE
[COLOR="SeaGreen"]#compression level when output to flac[/COLOR]
CompressionLevel=8
[COLOR="SeaGreen"]Maximum PCM sample rate supported by your dac[/COLOR]
MaxSampleRate=384000
[COLOR="SeaGreen"]Maximum DSD sampel rate supported by your dac[/COLOR]
MaxDSDSampleRate=DSD128

Currently tested on Debian and FreeBSD. It should work on Win. Don't know about MacOS.

Downloads
YCLD download -> YCLD

Pre-compiled sox binaries with dsd support -> SoX

Pre-compiled CamillaDSP binaries (only with options needed by YCLD)-> CamillaDSP

If downloading from unix* command line --> curl -A "audiodigitale" -o /path/YCLD.tar.gz https://etc.etc.etc

Don't thank me, thank CamillaDSP and SoX devs and maintainers, and offer them a coffee.

Why YCLD ?
Easy! YourCpuLooksDelicious! Try to run it with both convolution and DSD upsample enabled and you'll understand
 
Last edited:
#4
HenrikEnquist said:
This doesn't look that great and I would like to take a look:

How would I reproduce it? Preferrably without setting up LMS.
Click to expand...



Original file : https://i.imgur.com/gFxzpUg.png
After upsampling 44100 -> 88200 with synchronous resampling : https://i.imgur.com/0cChi8t.png

To reproduce:

Input file 24/44 flac

Yaml file:
Code: 
devices:
  samplerate: 88200
  chunksize: 4096
  enable_resampling: true
  resampler_type: Synchronous
  queuelimit: 128
  capture_samplerate: 44100
  capture:
    type: Stdin
    channels: 2
    format: S24LE3
  playback:
    type: Stdout
    channels: 2
    format: S24LE3

Command:
Code: 
sox.exe 01.flac --type raw - | camilladsp.exe temp.yml | sox.exe --multi-threaded --type raw --bits 24 --rate 88200 --channels 2 --encoding signed-integer -L - --type flac --compression 8 02.flac

When using the same command and configuration but with Asynchronous resampling the result is perfect without high frequency noise
 
Last edited:
#5
Do pipes on windows behave the same way as on unix/posix, in terms of timing, blocking, etc?

Perhaps using a generated single/multitone instead of complex music would be convenient because a spectrogram of the result (sox spectrogram effect) shows at first sight any imperfections in the chain, even a single missed/incorrect sample in many minutes.
 
#6
phofman said:
Do pipes on windows behave the same way as on unix/posix, in terms of timing, blocking, etc?

Perhaps using a generated single/multitone instead of complex music would be convenient because a spectrogram of the result (sox spectrogram effect) shows at first sight any imperfections in the chain, even a single missed/incorrect sample in many minutes.
Click to expand...

This problem with synchronous resampling happens to me on linux too so I don't think it is OS dependent.


BTW
UPDATE 0.2
Some minor fixes and tested the compatibility with windows (Win 10 at least)
Now I need to deeply understand the syntax of the custom-convert file (some help could be needed), and then write an app (even mobile) which creates resampling profiles for every player or customized for each player (you can do that with MAC addresses of the players), auto-generate a correct custom-convert file and automatically reboot lms to apply the rules. But before that I need some testers for YCLD
 
Last edited:
#7
#8
Awesome, I haven't opened an issue on GitHub before because I needed more testing with Camilla and I thought It was an implementation problem on my side. This Is indeed very good because AccurateAsync, while It provides awesome results, it's quite CPU intensive. Thanks again!
 
#9
UPDATE 0.3
Now in config file there are two new line:
Code: 
MaxSampleRate=384000
MaxDSDSampleRate=DSD128

In this way you can specify the maximum sample rates of your DAC.
When you upsample using the option "MaxSync" or "NextSync" YCLD will never go over the maximum sample rates set in the config file.

I'm starting to write a backend web service app to automate all the editing of the custom-convert file. It will expose some WebMethods that can be used with normal http GET/POST request. I will then write an Android (and maybe iOs) app to configure all the resampling/convolution profiles for each player and for each format. Since the backend app is agnostic about the client anyone can then write a Desktop or Web app too.
 
#10
Update 0.4
The old "ResamplerQuality=" option is now replace by
Code: 
UseMaxResamplerQuality=true

In this way when the output sample rate is a multiple of the input sample rate CamillaDSP will be set to use Synchonous resampling.
If input and output sample rates are not multiple (like 44.1->96):
- If UseMaxResamplerQuality = true AccurateAsync will be used
- If UseMaxResamplerQuality = false BalancedAsync will be used.

Now the code is backward compatible since Java 8. No more Java 11 required.
 
#11
Simofil said:
In this way when the output sample rate is a multiple of the input sample rate CamillaDSP will be set to use Synchonous resampling.
If input and output sample rates are not multiple (like 44.1->96):
- If UseMaxResamplerQuality = true AccurateAsync will be used
- If UseMaxResamplerQuality = false BalancedAsync will be used.
.
Click to expand...
You should be able to use synchronous resampling also between the 44.1 and 48 kHz groups. For example the ratio for 44.1 -> 96 kHz is 320/147. It becomes a bit slower than for the simple factors of 2, but should still be faster than asynchronous.
 
#13
Simofil said:
Mmmmmm but then, if synchronous is ok also between 44khz group and 48khz group. Where's the advantage of the asynchronous resampling?
Click to expand...
Synchronous resampling works when the ratio can be expressed as a fraction, and once the resampler is created the ratio can't be changed. Asynchronous doesn't have those limitations. The ratio can be pi if you want, and you can adjust it while running. This is useful for stuff like matching the rates of a playback device and a capture device with different clock sources, meaning the ratio between them can drift a little.
 
#14
UPDATE 0.5
Thanks to the clarification of Henrik now the camilla resample is always default at Synchronous. Config sample in first post edited accordingly. Minimum Java version required 11.

Currently working on webui for integration with lms.
 
#15
Source 24/48 file -> FIR filter at 524288 TAPS -> DSD128 in a VM with 8 E5-2697v3 cores

QWm00o3.png


That CPU is really delicious.
The project has delayed a bit due to personal commitments. 0.5 is already perfectly working and pretty stable, ETA for WebUI ~ end of February
 
Last edited:
#17
Ciao Simofil,


I have tried Superplayer but I cannot make it works. Please could you advise if there is a guide for "dummies" to install your solution? I would like to apply my DSP to LMS. I have a RPI 4 with 2gb of ram.


Grazie mille e tante belle cose!!
 
Status
Not open for further replies.
Top Bottom