5.4 is still pretty old, all patches would have to be backported. RPi team pushes RPi-specific patches to the latest released 5.12, which makes it simple to create and test patches for inclusion in kernel upstream. Every serious deployment ends up with some patches, best if accepted by the upstream.
I need the fully working gadget mode for my project anyway. Fortunately I can use my own kernel and do not have to wait for upstream acceptance.
I need the fully working gadget mode for my project anyway. Fortunately I can use my own kernel and do not have to wait for upstream acceptance.
I've been looking to do something with a Pi that would be able to capture the audio stream and create output emulating a USB Mass Storage Device. The reader would then either be able to start playing the "file" that is created as an endless file or have the Pi fill a number of files that are pre-formatted containers.
Why you ask? Well my car does not have an Aux in jack, nor does it act as a USB DAC. What it can do is use Bluetooth to talk to my phone or other device, or it can read music stored on a USB mass-storage stick or drive. Only the USB stored music is truly hi-res, all other is just not.
I'm not a Linux, USB or audio programmer, I wouldn't even know where to start to learn these days, but it does sound like some of you posting here are, and are doing something similar. There are more than a few people that I've seen on the car forums looking for something like this. With all the high tech built into cars these days I think the designers thought they were doing us a favor with bluetooth but the audiophiles know better.
I'm not sure that doing what I propose fits into the EULA of most of the streaming services since we pay to stream not to store, but I'm not proposing storage as much as using a storage medium to stream. Kind of borderline, because once written to the MSD it could be permanently stored even if that isn't my intent.
Why you ask? Well my car does not have an Aux in jack, nor does it act as a USB DAC. What it can do is use Bluetooth to talk to my phone or other device, or it can read music stored on a USB mass-storage stick or drive. Only the USB stored music is truly hi-res, all other is just not.
I'm not a Linux, USB or audio programmer, I wouldn't even know where to start to learn these days, but it does sound like some of you posting here are, and are doing something similar. There are more than a few people that I've seen on the car forums looking for something like this. With all the high tech built into cars these days I think the designers thought they were doing us a favor with bluetooth but the audiophiles know better.
I'm not sure that doing what I propose fits into the EULA of most of the streaming services since we pay to stream not to store, but I'm not proposing storage as much as using a storage medium to stream. Kind of borderline, because once written to the MSD it could be permanently stored even if that isn't my intent.
Better crashing than getting stuck by stopped stream. The last patch usb: u_audio: Stopping u_audio PCM substream when capture/playback is… * pavhofman/linux-rpi@9e59143 * GitHub closes the gadget-side stream when it's stopped on the USB host side. Every rate change requires stopping the stream first.
You can read the current rate from alsa control Capture Rate/Playback Rate GADGET: Multiple samplerate support - quick rebase of Julian Scheel's… * pavhofman/linux-rpi@fb5afce * GitHub
Unfortunately the code does not inform about opening the stream on USB host side yet, no one has replied to my enquiry usb: u_audio: Notifying gadget audio device about starting/stopping capture/playback on the host - Patchwork .
I guess the easiest would be trying to open the device with ffmpeg in some loop. I am not sure if the gadget device will get stuck if the USB host side is not open/is not sending any data, not at my computer now. There are still several pieces missing to make the audio gadget fully working. But it's slowly coming, the async feedback patches were accepted to mainline kernel recently, today I sent another patchset of Ruslan's changes rebased to the latest kernel. It will take a few more months though...
I am not sure how the USB host side would handle the "endless file" on the mounted MSD filesystem. You could report a file with 4GB size (max FAT32 filesize) But since the data would come from a stream, the MSD could not pre-store the data into the file. Players may fetch a large portion of the file first - but there would be no data for that available on the gadget side yet. IMO this is a complicated issue. Maybe feasible, maybe not.
Testing the audio gadget with multiple-rates patch in win10 now. Foobar with wasapi-exclusive plugin switches the rates correctly with various samplerate files, even plays at non-standard rates such as 512kHz - attached spectrum of 200kHz tone at 512kHz/32bits. As Foobar internally works at float32, it looses full 32bit precision - hence the spectrum artefacts at -150dB.
Attachments
Using WASAPI exclusive mode in example code of Henrik's wasapi crate wasapi-rs/playsine_events.rs at master * HEnquist/wasapi-rs * GitHub - bitperfect 32bit playback of a 240kHz tone at 768kHz samplerate. Period time tested from reported minimum of 3ms up to 3s (which makes a VERY safe time margin for the application). The windows UAC2 driver seems to work quite good.
Attachments
Last edited:
Really cool! How fast can the RPi I2S interface go? And how fast are the fastest I2S dacs/adcs today?
I tested RPi4 I2S loopback at 1024/2/24 STICKY: The I2S sound thread. [I2S works] - Page 41 - Raspberry Pi Forums and 1536/2/16 STICKY: The I2S sound thread. [I2S works] - Page 41 - Raspberry Pi Forums , in the latter case the bit clock about 50MHz. At higher bit clock the loopback was already failing. Of course a practical implementation would require the codecs very close to the RPi, better using the Compute Module 4 with shorter traces.
As of real-world chips I tested ES9038Q2M at 768/24/2 - impulse response https://www.diyaudio.com/forums/equ...-oversampling-filter-issue-8.html#post6661524 , sines https://www.diyaudio.com/forums/equ...-oversampling-filter-issue-9.html#post6662799 . ESS chips support MCLK up to 100MHz which corresponds to 780kHz (128x FS). I do not know of faster I2S chips.
But CPLD/FPGA can be used to convert from fast-speed SPI to I2S, there are projects doing this here on diyaudio. Also, other ARM SBCs (e.g. with the Rockchip64 SOCs) offer several multichannel I2S interfaces and have the same dwc2 USB OTG. They are reported to support up to 16 channels, some channels up to 768kHz. Certainly enough bandwidth to fill up whole isochronous USB 2.0 bandwidth of 1024 bytes x 8000 packets per second.
In addition, multipacket support could be added to the gadget driver, allowing 3 1024 bytes packets per USB2 microframe (perfectly explained in How to transfer data to USB isochronous endpoints - Windows drivers | Microsoft Docs , the overall Microsoft USB documentation is the best I was able to find). I do not know if the windows UAC2 driver supports this, I think the linux one does. That would theoretically extend the available bandwidth to 24.5Mbytes/s, e.g. 42 channels at 192/24. I kind of doubt anyone has ever tested that because the linux gadget driver does not have the required support yet and I would be surprised if the commercial USB IPs for FPGAs implemented that.
As of real-world chips I tested ES9038Q2M at 768/24/2 - impulse response https://www.diyaudio.com/forums/equ...-oversampling-filter-issue-8.html#post6661524 , sines https://www.diyaudio.com/forums/equ...-oversampling-filter-issue-9.html#post6662799 . ESS chips support MCLK up to 100MHz which corresponds to 780kHz (128x FS). I do not know of faster I2S chips.
But CPLD/FPGA can be used to convert from fast-speed SPI to I2S, there are projects doing this here on diyaudio. Also, other ARM SBCs (e.g. with the Rockchip64 SOCs) offer several multichannel I2S interfaces and have the same dwc2 USB OTG. They are reported to support up to 16 channels, some channels up to 768kHz. Certainly enough bandwidth to fill up whole isochronous USB 2.0 bandwidth of 1024 bytes x 8000 packets per second.
In addition, multipacket support could be added to the gadget driver, allowing 3 1024 bytes packets per USB2 microframe (perfectly explained in How to transfer data to USB isochronous endpoints - Windows drivers | Microsoft Docs , the overall Microsoft USB documentation is the best I was able to find). I do not know if the windows UAC2 driver supports this, I think the linux one does. That would theoretically extend the available bandwidth to 24.5Mbytes/s, e.g. 42 channels at 192/24. I kind of doubt anyone has ever tested that because the linux gadget driver does not have the required support yet and I would be surprised if the commercial USB IPs for FPGAs implemented that.
Last edited:
Thanks for your great work on this!
I tested usb audio gadget with the RPi4 following those instructions: https://www.hackster.io/masonrf/rpi-zero-2-w-audio-gadget-448a6a
(Kernel rpi-5.16.y, which seems to include several changes from you.)
Are there still changes missing to get to this high channel count you mentioned?
(changes to the device tree ?)
For me it seems to max out at 10 channels with 16bit and 48kHz.
Thanks!
In the stock kernel the EP-IN and EP-OUT bIntervals are still fixed to 4 (i.e. data every 1ms frame) which limits available bandwidth to 1ms * 1024 bytes per packet. You can change the value to 1 for max bandwidth in lines https://github.com/raspberrypi/linux/blob/rpi-5.16.y/drivers/usb/gadget/function/f_uac2.c#L347 and https://github.com/raspberrypi/linux/blob/rpi-5.16.y/drivers/usb/gadget/function/f_uac2.c#L481
There are still a number of patches to be delivered to kernel 5.18, including multiple samplerates and automated bInterval configuration based on the maximum required bandwidth for the configured params. Patch window for 5.17 is already closed and there were a number of modification requests by other developers.
There are still a number of patches to be delivered to kernel 5.18, including multiple samplerates and automated bInterval configuration based on the maximum required bandwidth for the configured params. Patch window for 5.17 is already closed and there were a number of modification requests by other developers.
Thanks, this works fine!
Looking forward to the upcoming patches.
I was also testing connecting a Windows host. I had to apply following modification to get it to work:
https://github.com/raspberrypi/linux/issues/4587#issuecomment-926567213
Looking forward to the upcoming patches.
I was also testing connecting a Windows host. I had to apply following modification to get it to work:
https://github.com/raspberrypi/linux/issues/4587#issuecomment-926567213
Maybe this is something that should be included upstream?
@phofman, I just discovered this thread and am following with fascination. All manner of intelligent DAC projects become possible! Thank you so much for your continued effort into this topic.
Forgive me, but I'm unable to piece this together: If I wanted to build a kernel myself with all these changes today, is there a succinct list of patches somewhere? And, are any significant configuration changes needed, compared to "ordinary" gadget mode?
Thanks!
Forgive me, but I'm unable to piece this together: If I wanted to build a kernel myself with all these changes today, is there a succinct list of patches somewhere? And, are any significant configuration changes needed, compared to "ordinary" gadget mode?
Thanks!
I would suggest to wait for 5.18 where all important patches should be included by default. RPi devs usually release a new branch at https://github.com/raspberrypi/linux when RC1 gets out, with all their off-tree patches. For linking the gadget to some output sound device I would recommend camilladsp with patches in https://github.com/HEnquist/camilladsp/pull/179 - so far tested just by me, most likely will receive some brush ups, configuration etc.
I built a kernel with all the relevant patches over a week ago. Now it's just sitting there waiting for some free time so I can try phofmans changes. The plan is to include them in camilladsp v1.1.0. I was thinking to include them in 1.0.0, but considering that 5.18 isn't ready yet there is no rush, better to focus on finally finishing that 1.0.0 release.