Hacker News

FFmpeg-over-IP – Connect to remote FFmpeg servers

2026-03-1018:2623366github.com

Connect to remote ffmpeg servers. Contribute to steelbrain/ffmpeg-over-ip development by creating an account on GitHub.

Use GPU-accelerated ffmpeg from anywhere — a Docker container, a VM, or a remote machine — without GPU passthrough or shared filesystems.

GPU transcoding is powerful, but getting access to the GPU is painful:

  • Docker containers need --runtime=nvidia, device mounts, and driver version alignment between host and container
  • Virtual machines need PCIe passthrough or SR-IOV — complex setup that locks the GPU to one VM
  • Remote machines need shared filesystems (NFS/SMB) with all the path mapping, mount maintenance, and permission headaches that come with them

You just want your media server to use the GPU for transcoding. You shouldn't need to restructure your infrastructure to make that happen.

Run the ffmpeg-over-ip server on the host (or any machine with a GPU). Point your app at the client binary instead of ffmpeg. Done — your app gets GPU-accelerated transcoding without needing direct GPU access.

The client pretends to be ffmpeg. It forwards arguments to the server, which runs a patched ffmpeg that tunnels all file I/O back through the connection. Files are never stored on the server.

CLIENT (has files, no GPU)              SERVER (has GPU)
========================              ===========================

Media server invokes "ffmpeg"         Daemon listening on :5050
        |                                      |
  ffmpeg-over-ip-client               ffmpeg-over-ip-server
        |                                      |
        +--------- TCP connection ------------>+
        |                                      |
  Local filesystem                      patched ffmpeg
  (real files)                    (file I/O tunneled back to client)

No GPU passthrough. No shared filesystem. No NFS. No SMB. Just one TCP port.

Releases include pre-built ffmpeg and ffprobe binaries with broad hardware acceleration support (NVENC, QSV, VAAPI, AMF, VideoToolbox, and more) — built on the jellyfin-ffmpeg pipeline. No need to install ffmpeg separately on either side.

See docs/quick-start.md to get up and running in a few minutes.

See docs/upgrading.md for migration guide and breaking changes.

See docs/configuration.md for full configuration reference (config file search paths, server/client options, rewrites, logging, address formats).

See docs/docker.md for Docker integration, Unix socket setup, and debugging tips.

  1. Your media server calls ffmpeg-over-ip-client with normal ffmpeg arguments
  2. The client connects to the server and sends the command with HMAC authentication
  3. The server launches its patched ffmpeg, which tunnels all file reads and writes back to the client
  4. stdout/stderr are forwarded in real-time; when ffmpeg exits, the client exits with the same code

Multiple clients can connect to the same server simultaneously — each session gets its own ffmpeg process.

Client Server + ffmpeg
Linux x86_64 ✓ ✓
Linux arm64 ✓ ✓
macOS arm64 ✓ ✓
macOS x86_64 ✓ ✓
Windows x86_64 ✓ ✓

See docs/troubleshooting.md for common issues and debugging tips.

See CONTRIBUTING.md for build instructions, running tests, and project structure.

  • Authentication: HMAC-SHA256 with a shared secret. Every command is signed.
  • Single port: Only the server listens on a port. The client makes outbound connections only.

Split license — see LICENSE.md. The fio layer and ffmpeg patches (fio/, patches/) are GPL v3 (derived from ffmpeg). Everything else is MIT.


Read the original article

steelbrain

Karma: 1094
...
 @Hacker__News
@hacker._news

Comments

  • By steelbrain 2026-03-1018:268 reply

    Hi HN!

    I'm excited to show case an update to a personal project of mine. Its called ffmpeg-over-ip and it allows you connect to remote ffmpeg servers, so if you have one machine with a GPU (could be your windows gaming laptop, gaming PC, a macbook?) and a machine (or VM, docker container etc) without a GPU, you could use the remote GPU to do GPU-accelerated video conversion.

    The way it works is pretty neat, there are two components, a server and a client.

    - The server (has the GPU) comes with a patched up ffmpeg and listens on a specified port - The client (without the GPU) connects to the server, takes the file IO requests from the server and runs them locally.

    ffmpeg doesn't know that its not dealing with a local filesystem, so this approach works with multiple inputs or outputs like HLS, and is perfect for home media servers like Plex or Jellyfin or Emby.

    One server can offer up its GPU to as many clients as needed, all clients offer up their own filesystems for their requests, the server also comes with a static build of ffmpeg bundled (built from jellyfin-ffmpeg scripts) so all you have to do is create a config file, set a password and you're good to go!

    It's been about a year and half since this was last submitted (https://news.ycombinator.com/item?id=41743780). The feedback at the time was around the difficulty of sharing a filesystem between the machines so that should no longer be a problem.

    This has been really useful in my local setup, I hope you find it useful. If you have any further questions, the website has some FAQs (linked in github repo), or you could post them here and I'll answer them for you!

    Thanks!

    • By tda 2026-03-119:21

      Some operations (downsizing already heavily compressed video with the latest and greatest compression techniques) are CPU or GPU bound, but others, like bunching thousands of high res jopg into a lightly compressed timelapse are more likely to be IO bound. So how does this tool make the trade-off? I imagine some things can best be dealt with locally, whilest for ither operations offloading to a external GPU will be beneficial. Also makes a lot of difference if your bandwidth is megabits or gigabits.

    • By alfg 2026-03-110:55

      This is pretty neat. I was experimenting something similar with my ffmpeg frontend to connect to the local machine (and remote) to run arbitrary encoding jobs, thus offloading the encode tasks to another machine, but still with a queuing mechanism locally.

      The project is https://ffmpeg-commander.com for generating ffmpeg commands, but with an experimental backend to offload the tasks.

      Do you support chunked encoding across multiple servers? It would be a great feature to support larger video files.

    • By bsimpson 2026-03-1021:011 reply

      Nice to see you here!

      Was really impressed by your work on Pundle. (It was an amazingly fast HMR dev environment - much like Vite today.) Felt like I was the only one using it, but it was hard to walk away from instant updates.

      • By steelbrain 2026-03-1021:08

        Thanks for putting a smile on my face! I am glad you liked it! :)

    • By themafia 2026-03-111:031 reply

      ffmpeg has great http input and output support. I've been using this quite a bit recently. Wrapping ffmpeg with node.js and using the built in http server and client to interact with it.

      It's even reduced load considerably because most of the time the disk doesn't even need to be touched.

      • By tcbrah 2026-03-117:06

        yep same here, i do the same thing for my video pipeline. spawn ffmpeg as a child process from node, pipe stdin/stdout directly and skip disk entirely for intermediate steps. concat demuxer + xfade filters for stitching scenes together. the only time i touch disk is the final output and even thats optional if youre uploading straight to s3 or whatever

    • By tonymet 2026-03-1019:241 reply

      cool idea. can you elaborate on IO and how the ffmpeg-server reads blocks from the client? that would seem to be a big blocker

      • By steelbrain 2026-03-1019:402 reply

        > cool idea. can you elaborate on IO and how the ffmpeg-server reads blocks from the client? that would seem to be a big blocker

        ffmpeg-server runs a patched version of ffmpeg locally, ffmpeg requests to read some chunks (ie give me video.mp4) through our patched filesystem (https://github.com/steelbrain/ffmpeg-over-ip/blob/main/fio/f...), which gets sent over the same socket that the client established, client receives the request, does the file operations locally and sends the results back over the socket to the server, and server then sends them to ffmpeg.

        ffmpeg has no idea its not interacting with a local file system

        • By matsemann 2026-03-1021:191 reply

          Is video that cpu/gpu bound that streaming it over the interwebs isn't the issue?

          Maybe my use cases for ffmpeg are quite narrow, but I always get a speedup from moving the files off my external hard-drive, suggesting that is my current bottleneck.

          • By steelbrain 2026-03-1021:211 reply

            > streaming it over the interwebs isn't the issue

            The hope is that you stream over LAN not the interwebs!

            > I always get a speedup from moving the files off my external hard-drive

            Based on your description, it does seem like your ffmpeg may be IO limited

            • By matsemann 2026-03-117:16

              Ah, yeah, so this is probably for more professional workflows where you have a workhorse somewhere. Perhaps even in the cloud as long as the files are close by as well? My use case would be more "my computer sucks, so would be nice to do it on a beefy cloud computer", but of course no time is saved when just reading my files is slow, heh.

        • By tonymet 2026-03-1019:49

          very clever and thanks for explaining. for gpu-bound processes, which are common ffmpeg use cases, this is a great approach

    • By throwaway81523 2026-03-1020:221 reply

      Maybe you can submit a patch to ffmpeg.org.

    • By doctorpangloss 2026-03-1022:393 reply

      What's the point of this?

      A single CPU core on a 9500T or a Ryzen V1500B is fast enough to real-time re-encode 60mbps 4K H264 to 1080p 5mbps h264, aka, for a core use case - transcoding for web for Jellyfin over cellular, for example - you haven't needed hardware video engines on PCs for 9 YEARS.

      I have no idea why people are so hung up on hardware video encoding. It's completely wrong. The quality is worse. The efficiency is a red herring - you will still use every CPU core for IO threads in ffmpeg, if you don't configure that away, which you do not. And it requires really annoying setup and premium features on stuff like Plex. It just makes no sense!

      If latency is important to you, well then hardware engines make sense. But you are throwing away the latency sending it over the network. The only use case (basically) is video game streaming, and in that case, you'll have a local GPU.

      I have never read one of these ffmpeg network hardware encode innovations to have an actual benchmark comparison to single thread software transcoding tasks.

      I know you mean well but really. It makes NO sense.

      • By GranPC 2026-03-1022:492 reply

        > The efficiency is a red herring - you will still use every CPU core for IO threads in ffmpeg, if you don't configure that away, which you do not. And it requires really annoying setup and premium features on stuff like Plex. It just makes no sense!

        I would love to learn more about this! What can I do to fully optimize ffmpeg hardware encoding?

        My use case is transcoding a massive media library to AV1 for the space gains. I am aware this comes with a slight drop in quality (which I would also be keen to learn about how to minimize), but so far, in my testing, GPU encoding has been the fastest/most efficient, especially with Nvidia cards.

        • By garblegarble 2026-03-1112:14

          GPU encoding is fast, but usually it produces poorer quality results because it avoids trying paths that are hard to do quickly on the GPU.

          If you want to optimise, try different encoders (sounds like you've already done some of this) and lots of different settings - it'll involve a lot of tuning if you want to figure out the right balance for your particular media between quality/speed/size, while also making sure that your machine hurts as much as possible.

          Driveby 2c as a video industry person: don't retranscode your media unless you've got them in a really space inefficient codec and you're seriously hurting for space. You'll burn a lot of power retranscoding, are you actually saving useful $$$ of storage in exchange for that spend? Storage is cheap, and there's always a better codec coming along you could retranscode into and save some more space. It's a vicious cycle: each generation has to encode the artifacts from the previous generations.

        • By doctorpangloss 2026-03-1022:59

          You would use your full system, saturating the CPU and GPU, including unlocking the number of simultaneous video sessions for consumer NVIDIA GPUs. That said, software AV1 looks a lot better than hardware AV1 per bit.

      • By ranger_danger 2026-03-110:30

        As a rule, strong feelings about issues do not emerge from deep understanding.

      • By steelbrain 2026-03-1022:451 reply

        Thank you for sharing your experience. Seems like this is not relevant to your setup & usecase.

        People who need this know who they are. Not everything is for everybody.

        • By doctorpangloss 2026-03-1022:461 reply

          What is the use case?

          I'd argue this is for nobody haha

          Nobody using jellyfin plex or whatever needs it: they should just use software transcoding, it's better in pretty much every way.

          • By steelbrain 2026-03-1022:501 reply

            I've traveled around a lot in the past couple years so my situation (read: homelab equipment) has been changing and my usecase has been changing with it. It started out as:

            - I dont want to unplug the GPU from my gaming PC and plug it into my linux server

            - Then: I dont want to figure out PCI forwarding, I'll just open a port and nfs to the containers/vms (ffmpeg-over-ip v4 needed shared filesystem)

            - Now: I have a homelab of 4 mini PCs and one of them has an RTX 3090 over Oculink. I need it for local LLMs but also video encoding and I dont want to do both on the same machine.

            But you've asked a more fundamental question, why would people need hardware accelerated video decoding in the first place? I need it because my TV doesn't support all the codecs and I still want to watch my movies at 4K without stuttering.

            • By doctorpangloss 2026-03-1022:533 reply

              You can transcode in realtime in software to your TV. You don't need the GPU at all. Even on ancient USFF PCs.

              • By steelbrain 2026-03-1022:54

                I'll tell my TV you said that and I'll see if it stops buffering during playback :)

              • By selcuka 2026-03-113:241 reply

                > You can transcode in realtime in software

                Sometimes you want faster-than-realtime encoding, such as when backing up your video archive.

                • By doctorpangloss 2026-03-115:03

                  A CPU using all its cores is much faster than realtime.

              • By 8n4vidtmkvmk 2026-03-116:24

                This doesn't appear to be true. My Plex media server is ancient and it really struggles if it has to do any kind of transcoding. Definitely can't handle high bitrate 4k stuff.

    • By steelbrain 2026-03-1019:562 reply

      [flagged]

      • By majorchord 2026-03-110:041 reply

        Why are you using github as your personal proprietary app depot?

      • By synergy20 2026-03-110:352 reply

        nice, problem is that, with hikivision and dahua got banned these days, the majority of ip cameras on the market do not do onvif or rtsp, or neither, what a shame.

        • By steelbrain 2026-03-111:01

          Get a TP Link Tapo! They are like 20-30 bucks and come with ONVIF.

          EZVIZ is another ban-evading arm of HIKVision, easily available in Europe and has RTSP (confirmed) with alleged support of ONVIF as well.

        • By monsecchris 2026-03-117:23

          Nearly all ip cameras support ONVIF and/or RTSP.

  • By naikrovek 2026-03-1019:433 reply

    This is software which basically replicates what Plan9 gives you out of the box.

    Dammit I really wish Plan9 had taken off. It isn’t perfect but it does a much, much better job of helping me run my applications in ways that I want.

    If anyone doesn’t already know, one method of Plan9 remote access is to “cpu” into a remote machine which has the hardware you need. Your local filesystems go with you, and your environment on the remote machine consists of your local filesystems mounted to the remote machine, but only for you, and all applications you run in this context execute on the cpu of the remote machine and have access to the remote machines hardware, but your local filesystems. Imagine SSHing into a remote machine and your entire environment goes with you, development tools and all. That’s what Plan9 does for you.

    So if I “cpu” into a machine without ffmpeg, but with a GPU and I run ffmpeg, not only will it work, but I can tell ffmpeg to use a hardware encoder with a simple command line flag, and it’ll work.

    • By olowe 2026-03-1022:28

      The great thing about Plan 9 is it can make other distributed systems look so complicated. The worst thing about Plan 9 is that it can make other distributed systems look so complicated!

    • By cestith 2026-03-1113:59

      It would be nice to see http://9legacy.org/ pick up a bunch more development.

    • By arboles 2026-03-1120:04

      Or just run sshd on your client computer and mount (sshfs) a directory on the render farm server

  • By deadlyllama 2026-03-1021:071 reply

    I've been meaning to build exactly this for a while, and my waiting has been rewarded by someone else doing it!

    Had you thought about using FUSE on the server side, rather than patching ffmpeg? Like a reverse sshfs? That avoids patching the ffmpeg binary, which allows usage of wierd and wonderful ffmpeg binaries with other people's patches.

    I'd be interested in seeing how well it works with SBC GPUs - many have hardware decoding and encoding, and their vendors love to fork ffmpeg.

    • By steelbrain 2026-03-1021:121 reply

      I've explored sftp since ffmpeg has built-in support for it (-i sftp://...), but the support is quite buggy in code, I hope to submit some patches upstream to be able to change it. FTP in contrast seemed much more stable, at least looking at the code. FTP had some other shortcomings that made it undesirable for my usecase.

      That was the one motivation, the other one was that it would require rewriting arguments going into the server. What you're describing was essentially what ffmpeg-over-ip v4 (and its earlier versions!) was, and the constant feedback I'd heard was that sharing filesystems is too much work, ssh servers on windows and macOS are a bad experience, people want to use a bundled solution.

      Forking ffmpeg was no easy task! Took forever to figure out the build process, eventually caved in and started using jellyfin build scripts, but that has the downside of being a few versions behind of upstream HEAD.

      • By deadlyllama 2026-03-1021:26

        Sharing filesystems is hard when you make users do it in advance.

        I was thinking of the server end of an ffmpeg-over-ip system bringing up a FUSE filesystem backed by something similar to your VFS-served-by-the-client. Combine that either with argument rewriting, or chrooting into the FUSE filesystem.

        As another commenter said, where's plan 9 when you need it? If you go the FUSE route there are existing 9P implementations for both server and FUSE client you can use.

HackerNews

About