Hi,
Many popular media players (vlc, mpd, mpv etc) are using the 'ffmpeg' libraries (e.g. libavcodec) behind the scenes.
If you are writing your own audio player in Python or any other language then more likely you leverage some library which helps you to communicate with one of those players (mpd, vlc, mpv etc).
The question is how difficult it would be instead of piggy backing on those players (mpd etc) to use ffmpeg libraries through some Python binding?
I found several Python bindings for ffmpeg. Some of them just invoke ffmpeg, ffprobe, ffplay as a subprocess. But some provide real bindings for ffmpeg libraries. For example:
https://github.com/PyAV-Org/PyAV
Does anybody have experience using this kind of ffmpeg bindings for audio playback purposes?
Thanks!
Many popular media players (vlc, mpd, mpv etc) are using the 'ffmpeg' libraries (e.g. libavcodec) behind the scenes.
If you are writing your own audio player in Python or any other language then more likely you leverage some library which helps you to communicate with one of those players (mpd, vlc, mpv etc).
The question is how difficult it would be instead of piggy backing on those players (mpd etc) to use ffmpeg libraries through some Python binding?
I found several Python bindings for ffmpeg. Some of them just invoke ffmpeg, ffprobe, ffplay as a subprocess. But some provide real bindings for ffmpeg libraries. For example:
https://github.com/PyAV-Org/PyAV
Does anybody have experience using this kind of ffmpeg bindings for audio playback purposes?
Thanks!
@rpi
I don't have much experience with binding directly to ffmpeg but more from spawning it as a subprocess for transcoding jobs but here are just some thoughts to consider.
I was exploring that as well when building Randy player just so that I can have less things in between Randy and music playing and less overhead, but the reality is that music playing is more than just decoding the audio file and piping the raw data to alsa.
You need to consider things like reading metadata (from both files and streams), getting cover art, listening to events like - seeking/pause/play/end of file, calculating the listening position in the track, optimising buffers, handling different file type edge cases etc...
Player libraries like mpd, mpv vlc etc.. have done great work on making these things stable and working around different library quirks and fine tuning stuff that I imagine can take a lot of time.
Yeah but as with everything it could be a fun rabbit hole to dive in to.
I don't have much experience with binding directly to ffmpeg but more from spawning it as a subprocess for transcoding jobs but here are just some thoughts to consider.
I was exploring that as well when building Randy player just so that I can have less things in between Randy and music playing and less overhead, but the reality is that music playing is more than just decoding the audio file and piping the raw data to alsa.
You need to consider things like reading metadata (from both files and streams), getting cover art, listening to events like - seeking/pause/play/end of file, calculating the listening position in the track, optimising buffers, handling different file type edge cases etc...
Player libraries like mpd, mpv vlc etc.. have done great work on making these things stable and working around different library quirks and fine tuning stuff that I imagine can take a lot of time.
Yeah but as with everything it could be a fun rabbit hole to dive in to.
@gideon, it looks like the ffplay (player based on ffmpeg libraries) handles all the tasks which you listed:
https://github.com/FFmpeg/FFmpeg/blob/master/fftools/ffplay.c
Though it does that on a very low level by handling frames/packets, managing buffers, synching to a master clock etc. Even if we exclude video/subtitles and leave only audio functionality it's too much stuff to handle directly in Python. That's probably the reason why nobody is doing that and using additional layer - libraries which connect to mpd, vlc, mpv etc. So, instead of the short path: player -> ffmpeg (through binding), it becomes a much longer path: player -> connection library (for mpd etc) -> mpd -> ffmpeg
https://github.com/FFmpeg/FFmpeg/blob/master/fftools/ffplay.c
Though it does that on a very low level by handling frames/packets, managing buffers, synching to a master clock etc. Even if we exclude video/subtitles and leave only audio functionality it's too much stuff to handle directly in Python. That's probably the reason why nobody is doing that and using additional layer - libraries which connect to mpd, vlc, mpv etc. So, instead of the short path: player -> ffmpeg (through binding), it becomes a much longer path: player -> connection library (for mpd etc) -> mpd -> ffmpeg
I did a quick check and it seems it's not just about the encoding, but it's a standard with its quirks as you can check in the C and the Python projects referenced below. Unless it's for homework, learning or embedded work I guess one would use a binding such us GStreamer, VLC, PyGame, ...
C: https://github.com/lieff/minimp3
"Minimalistic, single-header library for decoding MP3. minimp3 is designed to be small, fast (with SSE and NEON support), and accurate (ISO conformant)."
Python: https://portalfire.wordpress.com/2010/08/26/decoder-first-pass-complete/
"I finished the decoder and it can produce wav file output from an mp3. The good news is that is works, the bad news is that its 34 times to slow to play the audio runtime. (I have a core2 duo running at 1.83ghz.)"
And a backup of the code seems to live here: https://github.com/nikox94/kuko/tree/master/pyMp3/pyMp3Orig
I think you're looking for something like this:
https://git.videolan.org/?p=vlc/bindings/python.git;a=blob;f=examples/play_buffer.py
Calling the library and not an external binary. It seems this api has many helpers.
https://git.videolan.org/?p=vlc/bindings/python.git;a=blob;f=examples/play_buffer.py
Calling the library and not an external binary. It seems this api has many helpers.
I'm not sure how that mp3 stuff is related to ffmpeg. I'm using one of the VLC library bindings already : https://github.com/oaubert/python-vlc
The question was if it's possible to use ffmpeg binding instead. Thanks!
The question was if it's possible to use ffmpeg binding instead. Thanks!
Yeah, I misunderstood then the added the second post.
ffmpeg has libraries prefixed "libav" so it seems you might want wrappers for that. This one seems interesting: https://github.com/PyAV-Org/PyAV
ldd /usr/bin/ffmpeg | grep -i libav
libavdevice.so.58 => /lib/x86_64-linux-gnu/libavdevice.so.58 (0x00007fc096101000)
libavfilter.so.7 => /lib/x86_64-linux-gnu/libavfilter.so.7 (0x00007fc095c00000)
libavformat.so.58 => /lib/x86_64-linux-gnu/libavformat.so.58 (0x00007fc095800000)
libavcodec.so.58 => /lib/x86_64-linux-gnu/libavcodec.so.58 (0x00007fc094200000)
libavutil.so.56 => /lib/x86_64-linux-gnu/libavutil.so.56 (0x00007fc093e00000)
libavc1394.so.0 => /lib/x86_64-linux-gnu/libavc1394.so.0 (0x00007fc0960a2000)
I used ffmpeg once in production and the wrapper was great, but it called the binary. Pretty convenient though.
ffmpeg has libraries prefixed "libav" so it seems you might want wrappers for that. This one seems interesting: https://github.com/PyAV-Org/PyAV
ldd /usr/bin/ffmpeg | grep -i libav
libavdevice.so.58 => /lib/x86_64-linux-gnu/libavdevice.so.58 (0x00007fc096101000)
libavfilter.so.7 => /lib/x86_64-linux-gnu/libavfilter.so.7 (0x00007fc095c00000)
libavformat.so.58 => /lib/x86_64-linux-gnu/libavformat.so.58 (0x00007fc095800000)
libavcodec.so.58 => /lib/x86_64-linux-gnu/libavcodec.so.58 (0x00007fc094200000)
libavutil.so.56 => /lib/x86_64-linux-gnu/libavutil.so.56 (0x00007fc093e00000)
libavc1394.so.0 => /lib/x86_64-linux-gnu/libavc1394.so.0 (0x00007fc0960a2000)
I used ffmpeg once in production and the wrapper was great, but it called the binary. Pretty convenient though.