I’m building a web app where users upload a video, the app crops the first 60 seconds client-side, extracts audio for server-side AI processing, then muxes the returned audio back into the cropped video. This must work on mobile including iOS.
The problem
iPhones record HEVC (H.265) by default. Every client-side approach I’ve tried fails on iOS Safari for large HEVC files:
WebCodecs:
VideoDecoder.isConfigSupported({ codec: "hev1..." }) → false
ffmpeg.wasm: fetchFile() copies entire file to JS heap → OOM (tab crash)
captureStream: video.captureStream() not supported on iOS Safari
canvas: must decode HEVC frames → ~8MB × 5 frame buffer → OOM
File.slice() works for byte-level cropping with no decode, but I still need to mux a new audio track in afterward — which brings the same memory constraints back.
What works
– iOS + H.264: WebCodecs pipeline works fine (decode → re-encode at 720p via VideoEncoder → mp4box.js mux)
– Android Chrome: WebCodecs or ffmpeg.wasm both work
What I’ve considered
1. Extract audio-only via mp4box.js + AudioDecoder (AAC is always supported even on iOS HEVC) — solves the crop step, not the mux step
2. Server-side mux — works but adds latency and bandwidth
3. mp4box.js container-level audio track replacement — still requires loading moov + mdat into memory
Question
Is there any iOS Safari 16.4+ web API that allows replacing the audio track in an MP4/MOV container without decoding video frames? Or any memory-safe approach to mux new audio into a large HEVC file client-side?
Or is server-side mux the only realistic option here?
