v0.4.11: video frame ring buffer + decoder stats + 0.1s audio buffer
Some checks failed
build / build (push) Has been cancelled

0.4.10 still played at ~2-5 fps even though the decoder buffer was
preallocated. Root cause: the single-slot staging buffer was paced by
SourceDataLine backpressure at the audio buffer's granularity (~0.5 s),
so the decoder burst-produced ~12 video frames into the slot while audio
drained, the consumer saw only the last frame of each burst, then the
decoder stalled until audio drained again. Net visible rate ~ source_fps
/ frames_per_burst.

Fix:
- Replace single staging slot with a 4-slot ring (preallocated, FIFO).
  Decoder writes to ringTail; if full, overwrites oldest and bumps
  droppedFrames so we can see overflow in the log. Render thread drains
  oldest under the same lock — no allocation, no race.
- Shrink audio driver buffer 0.5 s → 0.1 s so the decoder is paced more
  tightly. Burst size collapses from ~12 frames to 2-3, which fits
  inside the ring.
- Log decoder spec on start (WxH @ fps, audio Hz x ch, ring depth) and
  produced/consumed/dropped counters every ~10 s. Lets the user log
  confirm whether the decoder is keeping real-time pace and whether the
  ring is overflowing.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This commit is contained in:
tkrmagid
2026-05-16 02:10:47 +09:00
parent cee01bd448
commit 9b99283b70
3 changed files with 135 additions and 55 deletions

View File

@@ -3,7 +3,7 @@
마인크래프트 안에서 임의의 동영상 URL을 벽·바닥·천장에 평면으로 재생하는 Fabric 모드. 마인크래프트 안에서 임의의 동영상 URL을 벽·바닥·천장에 평면으로 재생하는 Fabric 모드.
- 모드 ID: `video_player` - 모드 ID: `video_player`
- 현재 버전: **0.4.10** - 현재 버전: **0.4.11**
- 마인크래프트 버전: **26.1.2** - 마인크래프트 버전: **26.1.2**
- 필요 Java: **25** (마인크래프트 26.x 가 요구함) - 필요 Java: **25** (마인크래프트 26.x 가 요구함)
@@ -51,23 +51,23 @@ Fabric은 마인크래프트에 모드 기능을 추가해 주는 로더입니
https://cdn.modrinth.com/data/P7dR8mSH/versions/Sy2Bq7Xc/fabric-api-0.149.0%2B26.1.2.jar https://cdn.modrinth.com/data/P7dR8mSH/versions/Sy2Bq7Xc/fabric-api-0.149.0%2B26.1.2.jar
- 더 최신 빌드를 찾을 땐: https://modrinth.com/mod/fabric-api/versions → 페이지에서 게임 버전 필터 `26.1.2` 를 직접 선택. (URL 파라미터 필터가 듣지 않는 경우가 있어서 페이지 안에서 한 번 더 확인하는 게 안전합니다.) - 더 최신 빌드를 찾을 땐: https://modrinth.com/mod/fabric-api/versions → 페이지에서 게임 버전 필터 `26.1.2` 를 직접 선택. (URL 파라미터 필터가 듣지 않는 경우가 있어서 페이지 안에서 한 번 더 확인하는 게 안전합니다.)
- 받은 `fabric-api-0.149.0+26.1.2.jar``mods` 폴더에 넣습니다. - 받은 `fabric-api-0.149.0+26.1.2.jar``mods` 폴더에 넣습니다.
2. **video_player** (이 모드, 0.4.10 부터 JavaCV 가 jar 안에 포함됨) 2. **video_player** (이 모드, 0.4.11 부터 JavaCV 가 jar 안에 포함됨)
- 다운로드: https://git.tkrmagid.kr/tkrmagid/mc_video_player_mod/releases - 다운로드: https://git.tkrmagid.kr/tkrmagid/mc_video_player_mod/releases
- 자신의 OS·CPU 에 맞는 jar **한 개** 만 받아서 `mods` 폴더에 넣으면 됩니다 (별도 JavaCV 설치 불필요): - 자신의 OS·CPU 에 맞는 jar **한 개** 만 받아서 `mods` 폴더에 넣으면 됩니다 (별도 JavaCV 설치 불필요):
- Windows 64bit: `video_player-windows-x86_64-0.4.10.jar` (~32MB) - Windows 64bit: `video_player-windows-x86_64-0.4.11.jar` (~32MB)
- macOS Intel: `video_player-macosx-x86_64-0.4.10.jar` (~24MB) - macOS Intel: `video_player-macosx-x86_64-0.4.11.jar` (~24MB)
- macOS Apple Silicon (M1/M2/M3/M4): `video_player-macosx-arm64-0.4.10.jar` (~21MB) - macOS Apple Silicon (M1/M2/M3/M4): `video_player-macosx-arm64-0.4.11.jar` (~21MB)
- Linux 64bit: `video_player-linux-x86_64-0.4.10.jar` (~27MB) - Linux 64bit: `video_player-linux-x86_64-0.4.11.jar` (~27MB)
- 자기 OS 가 헷갈리면: Windows 는 거의 다 `windows-x86_64`, 인텔맥은 `macosx-x86_64`, 애플 실리콘 맥은 `macosx-arm64`, 리눅스는 `linux-x86_64`. - 자기 OS 가 헷갈리면: Windows 는 거의 다 `windows-x86_64`, 인텔맥은 `macosx-x86_64`, 애플 실리콘 맥은 `macosx-arm64`, 리눅스는 `linux-x86_64`.
이전 버전(`video_player-0.4.0.jar`, `0.4.2.jar`, `0.4.3.jar`, `0.3.x.jar` 등)이 mods 폴더에 남아있다면 **반드시 삭제**하세요. 두 개가 같이 있으면 마인크래프트가 충돌로 켜지지 않습니다. 0.4.7 이하에서 쓰던 JVM 인수(`-Xbootclasspath/a:...javacv...`) 도 0.4.10 부터는 **빼주세요** — 모드 jar 안에 같은 JavaCV 가 들어있어서 부트클래스패스의 것과 충돌해 검은 화면이 날 수 있습니다. 이전 버전(`video_player-0.4.0.jar`, `0.4.2.jar`, `0.4.3.jar`, `0.3.x.jar` 등)이 mods 폴더에 남아있다면 **반드시 삭제**하세요. 두 개가 같이 있으면 마인크래프트가 충돌로 켜지지 않습니다. 0.4.7 이하에서 쓰던 JVM 인수(`-Xbootclasspath/a:...javacv...`) 도 0.4.11 부터는 **빼주세요** — 모드 jar 안에 같은 JavaCV 가 들어있어서 부트클래스패스의 것과 충돌해 검은 화면이 날 수 있습니다.
### STEP 5. 잘 설치됐는지 확인 ### STEP 5. 잘 설치됐는지 확인
게임 안에서 채팅창에 `/videostick` 을 입력하세요. 정상이라면: 게임 안에서 채팅창에 `/videostick` 을 입력하세요. 정상이라면:
- 인벤토리에 **비디오 스틱** 아이템이 들어옵니다 (보라/검정 missing-texture 가 아니라 작대기 모양 아이콘). - 인벤토리에 **비디오 스틱** 아이템이 들어옵니다 (보라/검정 missing-texture 가 아니라 작대기 모양 아이콘).
- 보라/검정 missing texture 가 나오면 **STEP 4** 에서 이전 버전 jar(`video_player-0.4.0.jar` / `0.4.1.jar` 등)가 mods 폴더에 같이 남아있는 경우입니다. 다 지우고 `0.4.10` 만 남기고 다시 시작하세요. (0.4.1 이하는 Fabric 26.1.2 model 로더가 unprefixed `item/generated` parent 를 거부해서 스틱 아이콘이 missing-model 큐브로 보입니다 — 0.4.2 에서 수정됨.) - 보라/검정 missing texture 가 나오면 **STEP 4** 에서 이전 버전 jar(`video_player-0.4.0.jar` / `0.4.1.jar` 등)가 mods 폴더에 같이 남아있는 경우입니다. 다 지우고 `0.4.11` 만 남기고 다시 시작하세요. (0.4.1 이하는 Fabric 26.1.2 model 로더가 unprefixed `item/generated` parent 를 거부해서 스틱 아이콘이 missing-model 큐브로 보입니다 — 0.4.2 에서 수정됨.)
--- ---
@@ -172,7 +172,7 @@ Fabric은 마인크래프트에 모드 기능을 추가해 주는 로더입니
```sh ```sh
JAVA_HOME=/usr/lib/jvm/java-25-openjdk-amd64 ./gradlew build JAVA_HOME=/usr/lib/jvm/java-25-openjdk-amd64 ./gradlew build
``` ```
산출물: `build/libs/video_player-0.4.10.jar` (~85KB) 산출물: `build/libs/video_player-0.4.11.jar` (~85KB)
플랫폼별 fat jar (JavaCV 1.5.13 + ffmpeg 8.0.1 네이티브 nested): 플랫폼별 fat jar (JavaCV 1.5.13 + ffmpeg 8.0.1 네이티브 nested):
```sh ```sh
@@ -181,7 +181,7 @@ JAVA_HOME=/usr/lib/jvm/java-25-openjdk-amd64 ./gradlew clean build -Pplatform=li
JAVA_HOME=/usr/lib/jvm/java-25-openjdk-amd64 ./gradlew clean build -Pplatform=macosx-x86_64 JAVA_HOME=/usr/lib/jvm/java-25-openjdk-amd64 ./gradlew clean build -Pplatform=macosx-x86_64
JAVA_HOME=/usr/lib/jvm/java-25-openjdk-amd64 ./gradlew clean build -Pplatform=macosx-arm64 JAVA_HOME=/usr/lib/jvm/java-25-openjdk-amd64 ./gradlew clean build -Pplatform=macosx-arm64
``` ```
산출물: `build/libs/video_player-<platform>-0.4.10.jar` (~21-32MB, jar 내부에 nested 로 javacv/javacpp/ffmpeg jar 5개 포함, Fabric loader 가 런타임에 classpath 로 풀어서 로딩) 산출물: `build/libs/video_player-<platform>-0.4.11.jar` (~21-32MB, jar 내부에 nested 로 javacv/javacpp/ffmpeg jar 5개 포함, Fabric loader 가 런타임에 classpath 로 풀어서 로딩)
JavaCV를 직접 의존성으로 가져오는 경우의 Maven 좌표: JavaCV를 직접 의존성으로 가져오는 경우의 Maven 좌표:
``` ```

View File

@@ -5,7 +5,7 @@ org.gradle.configuration-cache=false
# Mod # Mod
mod_id=video_player mod_id=video_player
mod_version=0.4.10 mod_version=0.4.11
maven_group=com.ejclaw.videoplayer maven_group=com.ejclaw.videoplayer
archives_base_name=video_player archives_base_name=video_player

View File

@@ -15,6 +15,7 @@ import java.nio.ByteBuffer;
import java.nio.ByteOrder; import java.nio.ByteOrder;
import java.nio.ShortBuffer; import java.nio.ShortBuffer;
import java.util.concurrent.atomic.AtomicBoolean; import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicLong;
/** /**
* SPEC §5.3 — fallback mp4/http(s) backend driven by JavaCV's FFmpegFrameGrabber. * SPEC §5.3 — fallback mp4/http(s) backend driven by JavaCV's FFmpegFrameGrabber.
@@ -40,16 +41,35 @@ public class JavaCvBackend implements VideoBackend {
private final AtomicBoolean running = new AtomicBoolean(false); private final AtomicBoolean running = new AtomicBoolean(false);
private final AtomicBoolean paused = new AtomicBoolean(false); private final AtomicBoolean paused = new AtomicBoolean(false);
/** /**
* Single preallocated RGBA staging buffer. Decoder thread writes into it under * Ring buffer of preallocated RGBA staging slots. Decoder thread writes to {@code ringTail}
* {@link #frameLock}; render thread reads via {@link #consumeFrame(long, long)} under the * under {@link #frameLock}; render thread drains the oldest slot via
* same lock. One allocation for the lifetime of the backend instead of one per frame — * {@link #consumeFrame(long, long)} under the same lock.
* see 0.4.10 changelog for the regression that motivated this. The lock is short-held *
* (one 8MB memcpy ≈ 1ms at 1080p) so contention is negligible. * <p>0.4.10 used a single staging slot and relied on {@link SourceDataLine#write}
* backpressure to pace the decoder. That paced only at audio-buffer granularity (~0.5 s):
* the decoder burst-produced ~12 video frames into the slot while the audio line drained,
* the consumer (60+ Hz polling) saw only the last frame of each burst, then the decoder
* stalled until audio drained again — net effect ~2 fps of visible video despite the
* decoder producing at the source's 24 fps. The ring absorbs the burst; combined with the
* smaller audio buffer (~0.1 s) below the burst collapses to 23 frames which fits in
* {@link #FRAME_RING_SLOTS}.
*
* <p>If the ring still fills, the decoder overwrites the oldest slot and increments
* {@link #droppedFrames}. Memory cost: {@code 4 × w × h × 4} bytes (32 MB at 1080p,
* ~130 MB at 4K).
*/ */
private static final int FRAME_RING_SLOTS = 4;
private final Object frameLock = new Object(); private final Object frameLock = new Object();
private ByteBuffer frameBuf; private final ByteBuffer[] ringBufs = new ByteBuffer[FRAME_RING_SLOTS];
private int frameBufBytes = 0; private final int[] ringBytes = new int[FRAME_RING_SLOTS];
private boolean frameDirty = false; private int ringHead = 0; // next slot to consume
private int ringTail = 0; // next slot to produce into
private int ringCount = 0;
/** Decoder telemetry (cumulative). Logged ~every 10 s from the decode thread. */
private final AtomicLong producedFrames = new AtomicLong();
private final AtomicLong consumedFrames = new AtomicLong();
private final AtomicLong droppedFrames = new AtomicLong();
private volatile int width = 0; private volatile int width = 0;
private volatile int height = 0; private volatile int height = 0;
private volatile float gain = 1.0F; private volatile float gain = 1.0F;
@@ -101,17 +121,22 @@ public class JavaCvBackend implements VideoBackend {
@Override @Override
public boolean consumeFrame(long dstAddr, long maxBytes) { public boolean consumeFrame(long dstAddr, long maxBytes) {
synchronized (frameLock) { synchronized (frameLock) {
if (!frameDirty || frameBuf == null || frameBufBytes <= 0) return false; if (ringCount <= 0) return false;
if (frameBufBytes > maxBytes) { int idx = ringHead;
// Texture not yet resized for this frame's dimensions — drop and wait for the int n = ringBytes[idx];
// caller to ensure capacity next tick. ensureTexture() runs in Entry.upload ByteBuffer buf = ringBufs[idx];
// before consumeFrame, so this is only hit on the exact tick of a resolution // Always advance head regardless of memcpy outcome — otherwise a single oversize
// change. // frame (e.g. mid-resize) would jam the ring forever.
frameDirty = false; ringHead = (idx + 1) % FRAME_RING_SLOTS;
ringCount--;
if (buf == null || n <= 0 || n > maxBytes) {
// Texture not yet sized for this frame, or empty slot — skip. ensureTexture()
// runs in Entry.tryUpload() before consumeFrame, so n > maxBytes only happens
// on the exact tick of a resolution change.
return false; return false;
} }
MemoryUtil.memCopy(MemoryUtil.memAddress(frameBuf), dstAddr, frameBufBytes); MemoryUtil.memCopy(MemoryUtil.memAddress(buf), dstAddr, n);
frameDirty = false; consumedFrames.incrementAndGet();
return true; return true;
} }
} }
@@ -121,9 +146,11 @@ public class JavaCvBackend implements VideoBackend {
closed = true; closed = true;
stopWorker(); stopWorker();
synchronized (frameLock) { synchronized (frameLock) {
frameBuf = null; for (int i = 0; i < FRAME_RING_SLOTS; i++) {
frameBufBytes = 0; ringBufs[i] = null;
frameDirty = false; ringBytes[i] = 0;
}
ringHead = ringTail = ringCount = 0;
} }
} }
@@ -209,6 +236,18 @@ public class JavaCvBackend implements VideoBackend {
localAudioLine = openLine(sampleRate, audioChannels); localAudioLine = openLine(sampleRate, audioChannels);
this.audioLine = localAudioLine; this.audioLine = localAudioLine;
// Decoder spec — printed once per playback so the user log shows what the decoder
// actually sees (resolution / frame rate / sample rate). Used to verify our pacing
// assumptions (e.g. ring depth, audio buffer length) match the source.
double srcFrameRate = 0;
try { srcFrameRate = ((Number) grabberCls.getMethod("getFrameRate").invoke(grabber)).doubleValue(); }
catch (Throwable ignored) {}
VideoPlayerMod.LOG.info(
"[{}] decoder started: {}x{} @ {} fps, audio {} Hz x{}, ring={} slots",
VideoPlayerMod.MOD_ID, width, height,
String.format("%.2f", srcFrameRate),
sampleRate, audioChannels, FRAME_RING_SLOTS);
Class<?> frameCls = Class.forName(FRAME_CLASS); Class<?> frameCls = Class.forName(FRAME_CLASS);
Field imageField = frameCls.getField("image"); Field imageField = frameCls.getField("image");
Field samplesField = frameCls.getField("samples"); Field samplesField = frameCls.getField("samples");
@@ -216,6 +255,14 @@ public class JavaCvBackend implements VideoBackend {
// but we still resolve its class so a future code path could fall back to it if a // but we still resolve its class so a future code path could fall back to it if a
// grabber refuses setPixelFormat. Keep the lookup defensive. // grabber refuses setPixelFormat. Keep the lookup defensive.
// Stats sampling: every 10 s of wall-clock we log produced/consumed/dropped deltas
// and the implied fps. Lets us tell from the log whether the decoder is keeping
// real-time pace (produced≈source fps) and whether the ring is overflowing
// (dropped>0). All counters are cumulative; we keep the previous sample to compute
// deltas.
long statsLastNs = System.nanoTime();
long lastProd = 0, lastCons = 0, lastDrop = 0;
while (running.get() && !closed) { while (running.get() && !closed) {
if (paused.get()) { Thread.sleep(20); continue; } if (paused.get()) { Thread.sleep(20); continue; }
Object frame; Object frame;
@@ -243,37 +290,63 @@ public class JavaCvBackend implements VideoBackend {
Object[] images = (Object[]) imageField.get(frame); Object[] images = (Object[]) imageField.get(frame);
if (images != null && images.length > 0 && images[0] instanceof ByteBuffer src) { if (images != null && images.length > 0 && images[0] instanceof ByteBuffer src) {
// frame.image[0] is the swscale-converted RGBA plane, reused by the grabber // frame.image[0] is the swscale-converted RGBA plane, reused by the grabber
// across grab() calls. Copy into our preallocated staging buffer under // across grab() calls. Copy into the next ring slot under frameLock so the
// frameLock so the render thread's consumeFrame() sees a coherent image. // render thread's consumeFrame() sees coherent frames in FIFO order.
// //
// 0.4.9 used `ByteBuffer.allocateDirect(w*h*4)` on every grab — at 1080p × // Allocation is one-time per slot, lazily on first use (or on a resolution
// 24fps that's ~192 MB/s of direct memory churn (each allocation zero-fills // upgrade) — never per frame. 0.4.9's per-frame allocateDirect was the
// the page, plus the Cleaner enqueues the old buffer for finalization). // primary memory-churn problem; 0.4.10 fixed that; 0.4.11 adds the ring on
// The decoder thread spent so much time on memory bookkeeping that grab() // top to absorb the burst-then-stall caused by SourceDataLine backpressure
// fell behind real time, the single-slot `latest` AtomicReference was // pacing only at audio-buffer granularity.
// refilled in bursts, and the user saw ~5fps playback even though the
// game/render thread was fine.
//
// Preallocating once eliminates both the zero-fill cost and the Cleaner
// pressure. The decoder thread now spends its budget on the actual decode +
// swscale + a single 8MB memcpy — well within 42ms at 1080p × 24fps.
int need = src.remaining(); int need = src.remaining();
if (need > 0) { if (need > 0) {
int srcPos = src.position();
long srcAddr = MemoryUtil.memAddress(src) + srcPos;
synchronized (frameLock) { synchronized (frameLock) {
if (frameBuf == null || frameBuf.capacity() < need) { int idx = ringTail;
frameBuf = ByteBuffer.allocateDirect(need).order(ByteOrder.nativeOrder()); if (ringBufs[idx] == null || ringBufs[idx].capacity() < need) {
ringBufs[idx] = ByteBuffer.allocateDirect(need).order(ByteOrder.nativeOrder());
} }
int srcPos = src.position(); long dstAddr = MemoryUtil.memAddress(ringBufs[idx]);
long dstAddr = MemoryUtil.memAddress(frameBuf);
long srcAddr = MemoryUtil.memAddress(src) + srcPos;
MemoryUtil.memCopy(srcAddr, dstAddr, need); MemoryUtil.memCopy(srcAddr, dstAddr, need);
src.position(srcPos); // unchanged, but explicit — JavaCV reads it too ringBytes[idx] = need;
frameBufBytes = need; ringTail = (idx + 1) % FRAME_RING_SLOTS;
frameDirty = true; if (ringCount < FRAME_RING_SLOTS) {
ringCount++;
} else {
// Ring was full — we overwrote the oldest frame. Advance head
// to point at the next-oldest so consume order stays FIFO.
ringHead = (ringHead + 1) % FRAME_RING_SLOTS;
droppedFrames.incrementAndGet();
}
producedFrames.incrementAndGet();
} }
src.position(srcPos); // restore — JavaCV reads it on subsequent grabs
} }
} }
// Periodic stats — once per ~10 s of wall-clock. Includes ring depth so we can
// see whether the consumer is keeping up.
long now = System.nanoTime();
if (now - statsLastNs > 10_000_000_000L) {
long prod = producedFrames.get();
long cons = consumedFrames.get();
long drop = droppedFrames.get();
double elapsedS = (now - statsLastNs) / 1e9;
int depth;
synchronized (frameLock) { depth = ringCount; }
VideoPlayerMod.LOG.info(
"[{}] decoder stats: produced={} ({} fps), consumed={} ({} fps), dropped={} (+{}) over {}s, ring={}/{}",
VideoPlayerMod.MOD_ID,
prod, String.format("%.1f", (prod - lastProd) / elapsedS),
cons, String.format("%.1f", (cons - lastCons) / elapsedS),
drop, (drop - lastDrop),
String.format("%.1f", elapsedS),
depth, FRAME_RING_SLOTS);
statsLastNs = now;
lastProd = prod; lastCons = cons; lastDrop = drop;
}
// If we have an open audio line, SourceDataLine.write() blocks for backpressure // If we have an open audio line, SourceDataLine.write() blocks for backpressure
// and provides natural A/V pacing; otherwise tick ~60fps so we don't busy-loop. // and provides natural A/V pacing; otherwise tick ~60fps so we don't busy-loop.
if (localAudioLine == null) Thread.sleep(15); if (localAudioLine == null) Thread.sleep(15);
@@ -308,10 +381,17 @@ public class JavaCvBackend implements VideoBackend {
try { try {
AudioFormat fmt = new AudioFormat(sampleRate, 16, channels, true, false); // signed 16-bit LE AudioFormat fmt = new AudioFormat(sampleRate, 16, channels, true, false); // signed 16-bit LE
SourceDataLine line = AudioSystem.getSourceDataLine(fmt); SourceDataLine line = AudioSystem.getSourceDataLine(fmt);
// ~0.5 s of audio buffered in the driver. Smooths over upstream hiccups without // ~0.1 s of audio buffered in the driver. 0.4.10 used 0.5 s, which let the decoder
// delaying close() — stopWorker() calls line.stop() / line.flush() to dump it. // burst ~12 video frames between backpressure stalls — way past the video ring's
// capacity and the visible cause of the "2-5 fps" stutter the user saw. With 0.1 s
// the audio line refills more often, so the decoder is paced more tightly and
// bursts collapse to 2-3 frames (well inside FRAME_RING_SLOTS).
//
// Floor at frameSizeBytes * 256 keeps the buffer above the typical OS / driver
// minimum so we don't get UnsupportedOperationException at line.open() on
// exotic sample rates.
int frameSizeBytes = 2 * channels; int frameSizeBytes = 2 * channels;
int bufferBytes = Math.max(sampleRate * frameSizeBytes / 2, frameSizeBytes * 1024); int bufferBytes = Math.max(sampleRate * frameSizeBytes / 10, frameSizeBytes * 256);
line.open(fmt, bufferBytes); line.open(fmt, bufferBytes);
line.start(); line.start();
return line; return line;