PeppyMeter

[SergeG]

Hi,

>I'll check if linking redrawing/updating to the frame rate can improve the animation and reduce CPU consumption.


Can be much lower even below the frame rate I guess


>2% consumption, is it for linear meters? If possible it would be great to see the video with meter animation.


Dropbox - VID_20200506_130605.mp4 - Simplify your life


Python peppymeter takes 2%. all the rest is VNC session


Yours,
Serge

 

[rpi]

Hi Serge,

It looks pretty good! I had not so good result with your smooth function. The hand was jumping.

Best regards

 

[SergeG]

Hi,


Actually with my datasource.py I sent you recently it was it was 3-4%.
After I started using audioop instead of my ugly signed/unsigned/peak-detection processing the CPU consumption dropped to 2%. It's Pi4 though, not Pi3

Would like to get my latest code?


Thank you,
Serge

 

[rpi]

Hi,
...Would like to get my latest code?

Sure, it can be helpful when I'll start working on the new version.


Thank you!

 

[SergeG]

Hi,


The final patches in the use


Yours,
Serge


PS Sorry, a lot of comments are still there

 

[rpi]

New version (Turner Edition) of the Peppy Player was released today.
The disk images with new release are available here:
Disk Images * project-owner/PeppyPlayers.doc Wiki * GitHub

New features:

- Added screen backgrounds.
Background * project-owner/Peppy.doc Wiki * GitHub

- Introduced new icon types: bi-color and gradient.
Icons * project-owner/Peppy.doc Wiki * GitHub

- Implemented ability to change player screen layout and UI components dimensions.
Player Screen * project-owner/Peppy.doc Wiki * GitHub

- Improved Lyrics Screensaver functionality.
Lyrics * project-owner/Peppy.doc Wiki * GitHub

- Added condensed and extra-condensed fonts.
https://github.com/project-owner/Peppy.doc/wiki/Fonts

- Implemented DSI display backlight control.
https://github.com/project-owner/Peppy.doc/wiki/Display-Backlight

- Added 'amixer' volume control.
https://github.com/project-owner/Peppy.doc/wiki/Volume-Control

- Audio Files Collection can be updated now.
https://github.com/project-owner/Peppy.doc/wiki/Collection-Updating

- Removed mplayer support.
- Improved slider and equalizer functionality.
- Code refactoring and bug fixing.

 

[rpi]

​If you use HiFiBerry Amp/Amp2 you need to change the default setting 'Headphone' to the 'Digital' in the configuration file /home/pi/Peppy/config.txt. So it should look like this:

[volume.control]
type = amixer
amixer.scale = linear
amixer.control = Digital

Try to cleanup the browser's cache if you will experience the issues with Web UI.

Best regards

 

[SergeG]

Hi Rpi,

I’ve made some progress on below, If you are curious – welcome

1 Support for symmetrical circular meters (as on the screenshot). Not sure I’ve understood all the details of your source, so the fix may be ugly, it works though.

2 You were right, connecting mpd’s fifo to PeppyMeter directly (without alsa plugin) was a bad idea. As appeared when mpd sends data to the pipe, it starts producing hiccups for 500-800 ms (or so) once or twice an hour. I didn’t try to fix the mpd code, I just can confirm that if Peppymeter is running the problem occurs, stopping Peppymeter solves the problem.
Nearly sure the problem is not in Peppymeter, its source is pretty straightforward at that point.
All in all I had to switch to the alsa-plugin, albeit some changes to datasource.py were required to make the needle moving more adequate to the sound.

 

[rpi]

Hi Serge,

Thank you for sharing your progress.

The PeppyMeter code is straightforward but it's running on top of the Python interpreter which runs Garbage Collector from time to time:
Python Garbage Collection: What It Is and How It Works

When that happens the whole app will be frozen for the period required by the Garbage Collector (GC) to cleanup objects/release memory. Maybe this is what you are facing. Usually it's not recommended to call gc.collect() in the code but probably in this case it would make sense. Maybe do that periodically in a separate thread. Though I'm not sure that the meter code creates any objects which should be collected by the GC. Probably GC runs periodically in any case.

I'm planning to work on PeppyMeter for the next Peppy player release (add buffer flush etc). I would appreciate any ideas in this area.

Thanks in advance!

 

[SergeG]

Hi Rpi,


The hypothesis is very good and can explain the behavior.
The explanation is not full, but very logical though.
Even if the python based the pipe receiver is not responsible for a period, it should not be a reason
for mpd to “hiccup” on other “audio_outputs”. So mpd and/or alsa (that is involved into the signal routing even without the alsa plugin) are not “asynchronous" enough.
So the problem could easily start from the garbage collection, but where it stops.
The kernel level pipes processing may affect the problem as well.


I think developing a python level program (meter) should not go out of the python language scope.
Otherwise the process might be not convergent, especially if other programs and kernel modules are involved .
In this case using a different approach (e.g. the alsa-plugin) looks like a more convergent approach.


Needless to say I have no confirmations that the problem is not a unique to my hardware/kernel version etc,etc…. I removed nearly everything from cron and did control/log the CPU usage but still not sure the “hiccups” are not brought by the system itself. There is a lot of background threads in linux kernel and existing CPU consumption meters are not ideal for measuring short peaks.




Thanks

 

[rpi]

Hi Serge,

It's hard to say what could be the reason of that issue. Theoretically mpd and meter are decoupled by the pipe and mpd should not be affected by the pipe behavior. For example if the pipe is full the playback to another mpd output should not be affected. To find the root cause more information is required - CPU utilization and memory consumption of all top processes at the moment when the issue happened, processes priorities and so on. More likely the root cause is in Python+PeppyMeter combination.

As far as I understand when you use peppyalsa plugin you don't see the issue, do you?

If you don't mind could you share your changes in datasource.py which you mentioned? That should be helpful when I'll start making my own changes.

Thank you!

 

[SergeG]

Hi Rpi,

>Theoretically mpd and meter are decoupled by the pipe

Theoretically you are right.
But I have no idea for what the kernel would do with pipe writer
if the pipe reader is connected but timeouted in the middle of the writer's write operation.
Should it make buffering inside? Forever/endlessly (hard to believe) ?
Or should it block the writer while the reader is not responsible.
If it blocks - voila. The problem is. And (probably) the synchronous multiple mpd's audio_outputs will cause hiccups


> For example if the pipe is full the playback to another mpd output should not be affected

I didn't check the mpd source so far ;-)
Probably some ALSA internals should be checked as well.


> To find the root cause more information is required - CPU utilization and memory consumption of all top processes at the moment when the issue happened, processes priorities and so on.

Entire memory utilization is about 10%, average CPU is about the same 10%.
Unfortunately I was not able to find a "proper" CPU consumption checker.
All the checkers I know just perform (adjustable but) periodical only checks.
So ~500ms hiccup is hardly traceable.
Moreover it happens once (rare twice) an hour without a strict period, It's really hard to trace.


>More likely the root cause is in Python+PeppyMeter combination.

The same seems to me as well

>As far as I understand when you use peppyalsa plugin you don't see the issue, do you?

Using peppyalsa plugin does not cause any problems.
Maybe just because the data is rounded in the plugin and the data amount through the pipe is significantly lower


>If you don't mind could you share your changes in datasource.py which you mentioned?

Sure, attached. Sorry there is a lot of my commented "debug prints" and "results" of the development inside.
The main idea is simple though - read all the available data from the pipe, not just 4 bytes (2left+2right) and find maximum "manually". If you need more comments and reasonings - welcome

 

[rpi]

Hi Serge,

Did you notice a high CPU consumption when you use the peppyalsa plugin? That happened when I changed a track for example and lasted for about 10 seconds. It doesn't happen if you comment out the line 'scopes.0 peppyalsa' in the .asoundrc file:
Configuration * project-owner/peppyalsa.doc Wiki * GitHub
I observed this behavior with different audio players: mpd, mpv and vlc.

 

[SergeG]

>Did you notice a high CPU consumption when you use the peppyalsa plugin?


I didn’t. But I’m not sure I know how to distinguish the alsa related consumption.
What process should I check? Should it be in a kernel/daemon thread?
Or it happens inside the playing process?
Anyway the entire CPU usage I observed is hardly above 10-11%.
I will try to recheck though. Thanks!




>That happened when I changed a track for example and lasted for about 10 seconds.


Actually my configuration is bit different,
All mpd’s audio_output-s are directed to “hw-s” straightly (to avoid a possible resampling)

and for the level measurement I use a separate
audio_output {
type "alsa"
name "peppyalsa"
}
that looks in etc/asound.conf as
pcm.!DummyPCM {
type hw
card Dummy
}

pcm.!peppyalsa {
type meter
slave.pcm "DummyPCM"
scopes.0 peppyalsa
}
where dummy is a snd-dummy driver.


Some strangeness exists though. Sometimes (rarely) the needle movements start to be late from the sound.
Pausing-resuming the playing solves that always.
And probably you are right, I’m starting to recall this happens while changing tracks.
It didn’t draw my attention too much before.


Do you have an explanation/understanding of the "scopes.0..." influence.
Maybe a link for the scopes.0 native explanation? I’m still newbie with alsa unfortunately.

 

[rpi]

On that line 'scopes.0 peppyalsa' the redirection of the data stream to the peppyalsa plugin occurs.
The plugin works in the pipe non-blocking mode:
peppyalsa/meter.c at ddd631ca6f9ba3fc4c5df0f1912ac737b7eddadb * project-owner/peppyalsa * GitHub
It looks like on the Raspberry Pi the pipe buffer size is 1MB (1048576). It can be found in /proc/sys/fs/pipe-max-size.

The only explanation I have at the moment is that the plugin is filling up the buffer every time when change track happens.
That could explain the high CPU utilization by the players (mpd, mpv and vlc). The high CPU usage in turn causes the slowness in all other processes.

I still need to test if having a constantly running process consuming data on the other end of the pipe can help here. So far I tested without pipe reader.

 

[phofman]

Would perhaps setting fixed pipe size for all architectures help? Test get and set pipe/fifo buffer size. * GitHub

 

[rpi]

Yeah, that's what first came to my mind as well. I have tried to set the buffer size manually but the system automatically resets it to the default value. I'll try to do that from the code. Thank you!

 

[rpi]

I've added the following code to the meter.c:

#define PIPE_BUFFER_SIZE 16384

static void open_pipe() {
    fifo_fd = open(mypipe, O_WRONLY | O_NONBLOCK);
    if(fifo_fd != -1) {
        fcntl(fifo_fd, F_SETPIPE_SZ, PIPE_BUFFER_SIZE);
    }
}

Unfortunately that didn't help. I still see spikes in CPU usage when I change radio station or change track. The regular usage is about 12% spike usage is more than 100% meaning that one core cannot handle the load. The usage goes to the normal in about 12 seconds.

 

[phofman]

Is the performance problem only when peppymeter is enabled?

Perhaps some performance profiling (perf) to try to identify the CPU-load culprit?

 

[rpi]

Yes, the issue happens only when the peppyalsa plugin is in use.

Just looking at the code I cannot identify what could be the issue:
peppyalsa/meter.c at master * project-owner/peppyalsa * GitHub
peppyalsa/peppyalsa.c at master * project-owner/peppyalsa * GitHub

I've never used profilers for C code, let me see...