using a Raspberry Pi 4 as a USB DSP-DAC

[phofman]

Charlie, we need lsusb -v of the USB audio device, i.e. on the host PC. Thanks :-)

[CharlieLaub]

Got it. Attached.

[CharlieLaub]

Below is the output from sudo modinfo g_audio:

ON HOST:

charlie@ZBOX-CI329-1:~$ sudo modinfo g_audio
filename:       /lib/modules/4.15.0-58-generic/kernel/drivers/usb/gadget/legacy/g_audio.ko
license:        GPL
author:         Bryan Wu <cooloney@kernel.org>
description:    Linux USB Audio Gadget
srcversion:     CAC36978C11B925AFA752EF
depends:        libcomposite
retpoline:      Y
intree:         Y
name:           g_audio
vermagic:       4.15.0-58-generic SMP mod_unload 
signat:         PKCS#7
signer:         
sig_key:        
sig_hashalgo:   md4
parm:           idVendor:USB Vendor ID (ushort)
parm:           idProduct:USB Product ID (ushort)
parm:           bcdDevice:USB Device version (BCD) (ushort)
parm:           iSerialNumber:SerialNumber string (charp)
parm:           iManufacturer:USB Manufacturer string (charp)
parm:           iProduct:USB Product string (charp)
parm:           p_chmask:Playback Channel Mask (uint)
parm:           p_srate:Playback Sampling Rate (uint)
parm:           p_ssize:Playback Sample Size(bytes) (uint)
parm:           c_chmask:Capture Channel Mask (uint)
parm:           c_srate:Capture Sampling Rate (uint)
parm:           c_ssize:Capture Sample Size(bytes) (uint)

ON PI:

pi@Pi4gadget:~ $ sudo modinfo g_audio
filename:       /lib/modules/4.19.67-v7l+/kernel/drivers/usb/gadget/legacy/g_audio.ko
license:        GPL
author:         Bryan Wu <cooloney@kernel.org>
description:    Linux USB Audio Gadget
srcversion:     CAC36978C11B925AFA752EF
depends:        libcomposite
intree:         Y
name:           g_audio
vermagic:       4.19.67-v7l+ SMP mod_unload modversions ARMv7 p2v8 
parm:           idVendor:USB Vendor ID (ushort)
parm:           idProduct:USB Product ID (ushort)
parm:           bcdDevice:USB Device version (BCD) (ushort)
parm:           iSerialNumber:SerialNumber string (charp)
parm:           iManufacturer:USB Manufacturer string (charp)
parm:           iProduct:USB Product string (charp)
parm:           p_chmask:Playback Channel Mask (uint)
parm:           p_srate:Playback Sampling Rate (uint)
parm:           p_ssize:Playback Sample Size(bytes) (uint)
parm:           c_chmask:Capture Channel Mask (uint)
parm:           c_srate:Capture Sampling Rate (uint)
parm:           c_ssize:Capture Sample Size(bytes) (uint)

[flexus]

nice aproach, lots of work put into it.
here is an alternative aproach:

USB/IP/

it generates a kernel sided VHCI (virtual host controler interface) and forwards it over ethernet . the code is upstream, userspace tools are in most of the repos. it would be nice to hear your opinions about it.

i wanted to do a setup later because i wanted to wait for pipewire to implement A/V delay function and use it this way. it might work with audio.
the homepage is pretty much outdated but its being actively developed: last update is a month ago. cheers.

[CharlieLaub]

Originally Posted by flexus

nice aproach, lots of work put into it.
here is an alternative aproach:

USB/IP/

it generates a kernel sided VHCI (virtual host controler interface) and forwards it over ethernet . the code is upstream, userspace tools are in most of the repos. it would be nice to hear your opinions about it.

i wanted to do a setup later because i wanted to wait for pipewire to implement A/V delay function and use it this way. it might work with audio.
the homepage is pretty much outdated but its being actively developed: last update is a month ago. cheers.

That project seems to have a different goal. According to the text in the About section:

The USB/IP Project aims to develop a general USB device sharing system over IP network.

Here we are doing the reverse: using the USB bus to send IP packets over a local connection, so no "sharing". We happen to be using that system to send audio over the IP link (USB ethernet gadget) to another computer that is directly connected to the host. There is no need for a true, routable ethernet connection. Instead we set up the USB gadget to act as a private, non-routable local network and use that for audio. The motivation for that is to use the connected computer (e.g. a Raspberyr Pi 4) to do DSP on the incoming (stereo) audio and then output it over a multichannel DAC. We can create an audio peripheral that makes use of the audio processing capabilities of Linux and offload it from the host (e.g. Windows) platform.

[phofman]

Charlie, attached is the f_uac2.c with the commit usb: gadget: f_uac2: disable IN/OUT ep if unused * torvalds/linux@3fa4eaa * GitHub cherry-picked to 4.19 version of the file + ASYNC -> ADAPTIVE change. Please replace it in your 4.19 RPI kernel and post lsusb -v. Thanks.

[CharlieLaub]

While I work with phofman to try to get the USB audio gadget mode working correctly under Windows, I am working on the batch/script files that are needed to send audio over the USB ethernet gadget mode. The latter works under Windows, and has the added benefit that the same scripts can be used to send PCM (uncompressed) audio over your LAN from a Windows machine to a Pi or other linux machine where your endpoint (DAC) is located and where DSP can be performed.

[phofman]

Keeping the async mode for the audio gadget would be technically best, but quite difficult to implement [alsa-devel] Options for ASYNC feedback source in USB-audio gadget (USB OTG)?

[Tfive]

I read your posts on the alsa-devel mailing list. IMO all the stuff you describe is pretty complicated. My opinion:
a) adaptive in this case is perfectly fine if you have proper adaptive resampling, i.e. a pll implemented digitally with smooth, long term averaged adjustement of the resampling factor.
b) if you want to go async and at the same time avoid all the userspace feedback hassle and the daemon keeping an eye on this, why not use an existing alsa device as clocking master and slave the gadget to it. And then base the clock/timing feedback channel off of the master alsa device's clock/timing? The "master" device could be specified as module parameter.

Sorry for being lazy and not signing up to the alsa devel mailing list

[CharlieLaub]

At this point phofman has split off the effort on the USB audio gadget, while I have been continuing to work on sending audio using the USB ethernet gadget.

I used the RTP protocol to send over UDP for streaming audio over IP. To do this I have been using the Gstreamer platform, since it has good clock management under this scheme. Previously I was sending audio data over my LAN from where my audio player was located (computer #1) to where my DACs, amps, and speakers were located (computer #2, e.g. a Raspberry Pi). Since our "local" connection between the host computer and the Raspberry Pi 4 is ethernet over USB, the same kind of audio streaming is taking place, just over a local and non-routable connection.

The following approach is used:
On the host (e.g. Windows) computer, we can tap into the audio stream for the currently active audio device using WASAPI as input to Gstreamer. The audio will be in whatever playback format the user has selected in the sound device on the Windows machine. Gstreamer then converts this to 16 or 24 bit RTP frames and sends it to an IP address (the IP address of local USB ethernet gadget). On the Pi 4, another running instance of Gstreamer receives the RTP stream, puts the packets in the correct order, calculates the clock from the RTP timestamps, buffers the audio, and then sends it to the Pi's DAC.

There is some initial one-time setup that includes configuring the Pi 4 to present the USB-C port as an ethernet gadget, editing a couple of config files on the Pi, and installing Gstreamer on both the Pi and the Windows machine (a prebuilt executable is available for Windows). Please keep in mind this can only work on the Pi 4 or Pi zero (not tested on the zero), and on the Pi 4 you MUST supply power to the Pi via the GPIO pins and not via the USB-C (power) connector.

Once the initial setup is complete, the user should install two scripts, one for Windows and the other for the Pi. These automate all the audio streaming tasks for the user. On the Pi, the script should be running all the time - it could be called at startup. On Windows, a batch script can be placed on the Desktop where it appears as an icon. Clicking on the icon starts the audio data flowing to the Pi. I am doing the final polishing of these scripts now, and I should be able to post them here in the next couple of days.

The scripts check whether the USB ethernet connection has been established, and then run the relevant Gstreamer pipeline. On the Pi, the script checks to see if any audio data is flowing as well, because after the first gadget connection is made it always appears to be "up" even when the cable has been disconnected. Checking the RX buffer to see if data is arriving is a better way to trigger the launch of the Gstreamer pipeline. At boot up of the Pi, the IP address is not yet assigned, so that still has to be checked as well.

The scripts are configured to stream stereo audio. The user does not need to do anything except make sure the IP address, bit depth, and sample rate for the stream are correctly configured in the script files by editing a couple of lines at the top of each one. It's then set and forget, as long as the user maintains that audio format in the Windows audio device that is being used as the source for WASAPI.

This application requires resampling except when the source audio is already at the same rate as the stream. Windows audio, or the application playing the source, will perform the resampling.

This setup is also capable of performing multichannel DSP within Gstreamer on the Pi using LADSPA plugins. To do that requires editing of the Gstreamer pipeline in the shell script on the Pi, installing LADSPA on the Pi, and choosing/building one or more plugins. I will cover how to do that at a later date. If anyone wants more info they are welcome to post in the thread to encourage that discussion.

For now, the scripts will demonstrate what can be expected when sending audio over the USB ethernet gadget, and that is a good first step.