見出し画像

Fixed SeedVR2&VideoHelperSuite to generate a large size video clip.



范逸臣 - 秀麗江山

Complete Explanation of Fix Details

Root Cause: Differences Between v2.5.15 and v2.5.17

v2.5.15 Behavior (Before Fix)

File Path: `ComfyUI/custom_nodes/seedvr2_videoupscaler/src/optimization/compatibility.py`

def get_supported_compute_dtype(debug=None) -> torch.dtype:
    """
    Compute capability-based detection method
    """
    if torch.cuda.is_available():
        major, minor = torch.cuda.get_device_capability()
        if major < 8:  # Compute capability below 8.0
            return torch.float16
        return torch.bfloat16  # Compute capability 8.0 or higher
    return torch.float16

File Path: `ComfyUI/custom_nodes/seedvr2_videoupscaler/src/core/generation_utils.py`

'compute_dtype': get_supported_compute_dtype(debug),  # Dynamically determined by function call

Behavior:

  • RTX 5060 Ti has high compute capability, so `get_supported_compute_dtype()` returns `torch.bfloat16`

  • However, actual CUBLAS support status is not considered

  • As a result, BFloat16 may be enabled, but not necessarily guaranteed

v2.5.17 Behavior (Before Fix)

File Path: `ComfyUI/custom_nodes/seedvr2_videoupscaler/src/optimization/compatibility.py`

def _probe_bfloat16_support() -> bool:
    """
    Runtime probe method: Actually attempts computation to verify CUBLAS support
    """
    if not torch.cuda.is_available():
        return True
    try:
        a = torch.randn(8, 8, dtype=torch.bfloat16, device='cuda:0')
        _ = torch.matmul(a, a)  # Actually attempts computation
        del a
        return True
    except RuntimeError as e:
        if "CUBLAS_STATUS_NOT_SUPPORTED" in str(e):
            return False
        raise

BFLOAT16_SUPPORTED = _probe_bfloat16_support()  # Executed once at module import
COMPUTE_DTYPE = torch.bfloat16 if BFLOAT16_SUPPORTED else torch.float16  # Stored as constant

File Path: `ComfyUI/custom_nodes/seedvr2_videoupscaler/src/core/generation_utils.py`

from ..optimization.compatibility import COMPUTE_DTYPE, BFLOAT16_SUPPORTED
# ...
'compute_dtype': COMPUTE_DTYPE,  # Uses constant

Behavior:

  • Actually attempts BFloat16 computation at module import time

  • On RTX 5060 Ti, CUBLAS supports BFloat16, so `BFLOAT16_SUPPORTED = True`

  • `COMPUTE_DTYPE = torch.bfloat16` is determined

  • BFloat16 is used for all subsequent processing

Problem Occurrence Mechanism

Behavior Flow in v2.5.17 (Before Fix):


Detailed Explanation of Fix Details

Fix File 1: generation_phases.py

File Path: `ComfyUI/custom_nodes/seedvr2_videoupscaler/src/core/generation_phases.py`

Fix Location 1: final_video Allocation (lines 868-871)

Before Fix:

ctx['final_video'] = torch.empty((total_frames, true_h, true_w, C), 
                                 dtype=ctx['compute_dtype'],  # BFloat16
                                 device=target_device)

After Fix:

# Force float16 if compute type is bfloat16 to ensure NumPy compatibility in downstream nodes
# while maintaining memory efficiency (float32 would double memory usage)
storage_dtype = torch.float16 if ctx['compute_dtype'] == torch.bfloat16 else ctx['compute_dtype']
ctx['final_video'] = torch.empty((total_frames, true_h, true_w, C), dtype=storage_dtype, device=target_device)

Fix Meaning:

  • If `compute_dtype` is BFloat16, allocate `final_video` as Float16

  • Reason 1: NumPy does not directly support BFloat16, so use Float16 for compatibility

  • Reason 2: Float32 uses 2x memory, so keep Float16 for efficiency

  • Effect: `final_video` is always allocated in a NumPy-compatible type (Float16 or Float32)

Fix Location 2: sample Write in decode_all_batches (lines 994-1006)

Before Fix:

sample = manage_tensor(
    tensor=sample,
    target_device=target_device,
    tensor_name=f"sample_{decode_idx+1}",
    dtype=ctx['compute_dtype'],  # Could be BFloat16
    debug=debug,
    reason="writing to final_video",
    indent_level=1
)
ctx['final_video'][write_start:write_end] = sample

After Fix:

# Move sample to target device and write directly to final_video
# Use final_video's dtype (storage_dtype) instead of compute_dtype to ensure consistency
# This prevents BFloat16 from being written to final_video when storage_dtype is Float16
target_dtype = ctx['final_video'].dtype
sample = manage_tensor(
    tensor=sample,
    target_device=target_device,
    tensor_name=f"sample_{decode_idx+1}",
    dtype=target_dtype,  # Match final_video's dtype (Float16)
    debug=debug,
    reason="writing to final_video",
    indent_level=1
)

Fix Meaning:

  • Convert `sample` to match `final_video`'s dtype (Float16)

  • Reason: When `final_video` is Float16, prevent writing BFloat16

  • Effect: Maintains type consistency and prevents type conversion errors in downstream processing

  • Note: PyTorch slice assignment performs automatic type conversion, but explicit conversion ensures reliability

Fix Location 3: sample Write in postprocess_all_batches (lines 1353-1362)

Before Fix:

sample = manage_tensor(
    tensor=sample,
    target_device=ctx['final_video'].device,
    tensor_name=f"sample_{info_idx+1}_final",
    dtype=ctx['compute_dtype'],  # Could be BFloat16
    debug=debug,
    reason="writing processed result to final_video",
    indent_level=1
)

After Fix:

# Use final_video's dtype (storage_dtype) instead of compute_dtype to ensure consistency
# This prevents BFloat16 from being written to final_video when storage_dtype is Float16
target_dtype = ctx['final_video'].dtype
sample = manage_tensor(
    tensor=sample,
    target_device=ctx['final_video'].device,
    tensor_name=f"sample_{info_idx+1}_final",
    dtype=target_dtype,  # Match final_video's dtype (Float16)
    debug=debug,
    reason="writing processed result to final_video",
    indent_level=1
)

Fix Meaning:

  • Same as Fix Location 2: convert `sample` to match `final_video`'s dtype

  • Reason: Maintain type consistency in post-processing writes

  • Effect: Ensures type consistency at all write locations


Fix File 2: video_upscaler.py

File Path: `ComfyUI/custom_nodes/seedvr2_videoupscaler/src/interfaces/video_upscaler.py`

Fix Location: Type Conversion Order in Output Processing (lines 514-527)

Before Fix:

# Ensure CPU tensor in float32 for maximum ComfyUI compatibility
if torch.is_tensor(sample):
    if sample.is_cuda or sample.is_mps:
        sample = sample.cpu()  # First move to CPU (still BFloat16)
    if sample.dtype != torch.float32:
        src_dtype = sample.dtype
        try:
            sample = sample.to(torch.float32)  # Then convert to Float32 (on CPU)
            debug.log(f"Converted output from {src_dtype} to float32", category="precision")
        except Exception as e:
            debug.log(f"Could not convert to float32: {e}. Output is {src_dtype}, compatibility with other nodes not guaranteed", 
                      level="WARNING", category="precision", force=True)

After Fix:

# Ensure CPU tensor in float32 for maximum ComfyUI compatibility
if torch.is_tensor(sample):
    # Convert to float32 on GPU first (more memory efficient) before moving to CPU
    if sample.dtype != torch.float32:
        src_dtype = sample.dtype
        try:
            sample = sample.to(torch.float32)  # First convert to Float32 (on GPU)
            debug.log(f"Converted output from {src_dtype} to float32", category="precision")
        except Exception as e:
            debug.log(f"Could not convert to float32: {e}. Output is {src_dtype}, compatibility with other nodes not guaranteed", 
                      level="WARNING", category="precision", force=True)
    # Move to CPU after conversion (avoids temporary 2x memory on CPU)
    if sample.is_cuda or sample.is_mps:
        sample = sample.cpu()  # Then move to CPU

Fix Meaning:

  • Changed conversion order: first convert to Float32 (on GPU), then move to CPU

  • Reason 1: Conversion on CPU temporarily requires 2x memory, causing OOM

  • Reason 2: Converting on GPU avoids temporary 2x CPU memory usage

  • Effect: Reduces OOM risk during Float32 conversion

  • Note: After the fix, SeedVR2 passes `sample` as Float16, so Float16→Float32 conversion occurs on GPU


Fix File 3: nodes.py

File Path: `ComfyUI/custom_nodes/comfyui-videohelpersuite/videohelpersuite/nodes.py`

Fix Location: BFloat16 Handling in tensor_to_int Function (lines 124-130)

Before Fix:

def tensor_to_int(tensor, bits):
    tensor = tensor.cpu().numpy() * (2**bits-1) + 0.5  # Error if BFloat16
    return np.clip(tensor, 0, (2**bits-1))

After Fix:

def tensor_to_int(tensor, bits):
    # BFloat16 is not supported by NumPy, convert to float32 if needed
    # (SeedVR2 already converts to Float16, so this path should rarely be taken)
    if tensor.dtype == torch.bfloat16:
        tensor = tensor.float()  # Convert BFloat16 to Float32
    tensor = tensor.cpu().numpy() * (2**bits-1) + 0.5
    return np.clip(tensor, 0, (2**bits-1))

Fix Meaning:

  • Convert BFloat16 to Float32 before calling `.numpy()`

  • Reason: NumPy does not directly support BFloat16

  • Effect: Fallback when Float32 conversion fails in SeedVR2 output processing

  • Note: After the fix, SeedVR2 passes Float16, so this path should not be taken

This is the complete explanation of the fix details with all file paths specified.


いいなと思ったら応援しよう!