VideoHelp Forum

+ Reply to Thread
Results 1 to 8 of 8
Thread
  1. Is there any way to make running out of memory not happen?

    Image
    [Attachment 61076 - Click to enlarge]
    Quote Quote  
  2. You're running out of GPU ram, since you can't add more ram you need to swicth to using the CPU for this workload. i.e. no CUDA.

    Or spend the money and buy a GPU with more RAM.
    Quote Quote  
  3. Originally Posted by sophisticles View Post
    You're running out of GPU ram, since you can't add more ram you need to swicth to using the CPU for this workload. i.e. no CUDA.

    Or spend the money and buy a GPU with more RAM.
    What about the 600MB free it is reporting. Or I am just wrong
    Quote Quote  
  4. GPU memory allocation and usage is a is a very complicated thing, you have 4gb video ram, PyTorch has reserved 2gb ram, about 600mb is already allocated and the "free" ram is most likely reserved by the OS.

    https://developer.nvidia.com/blog/unified-memory-cuda-beginners/

    https://developer.nvidia.com/blog/enhancing-memory-allocation-with-new-cuda-11-2-features/

    https://en.wikipedia.org/wiki/Swap_chain

    https://en.wikipedia.org/wiki/Multiple_b

    There's a lot that goes into how GPU ram is used and allocated, but the long and short of it is that under normal circumstances it is not possible for an application to use all the video ram available.

    I have seen instances where using multiple GPU filters simultaneously coupled with using a CUDA powered encoder maxed out all the video cards onboard ram and when that happened the GUI froze up and stopped responding, just like happens when all of the system ram and swap are used up.

    You don't want to allocate all the available vram.
    Quote Quote  
  5. Only way I see that might work is to split the video into tiles apply vsgan on the tiles separately and then join the tiles back together.
    So something along the lines of:
    Code:
    # split video into 4 tiles
    tile1 = core.std.CropRel(clip=clip, left=0, right = clip.width%2*1, top=0, bottom=clip.height%2*1)
    tile2 = core.std.CropRel(clip=clip, left=clip.width%2*1, right = 0, top=0, bottom=clip.height%2*1)
    tile3 = core.std.CropRel(clip=clip, left=0, right = clip.width%2*1, top=clip.height%2*1, bottom=0)
    tile4 = core.std.CropRel(clip=clip, left=clip.width%2*1, right = 0, top=clip.height%2*1, bottom=0)
    # load VSGAN  
    from vsgan import VSGAN
    vsgan = VSGAN("cuda")
    vsgan.load_model(model)
    # apply VSGAN  
    tile1 = vsgan.run(clip=tile1)
    tile2 = vsgan.run(clip=tile2)
    tile3 = vsgan.run(clip=tile3)
    tile4 = vsgan.run(clip=tile4)
    # join tiles
    clip = core.std.StackVertical([core.std.StackHorizontal([tile1, tile2]),core.std.StackHorizontal([tile3, tile4])])
    Cu Selur
    users currently on my ignore list: deadrats, Stears555
    Quote Quote  
  6. btw. you can get some lower memory usage by enabling 'chunk' (it splits the frame in 4 parts):
    Code:
    clip = vsgan.run(clip=clip, chunk=True)
    problem is since there is no padding between the chunking you might get visible borderlines between the parts.
    see: https://github.com/rlaphoenix/VSGAN/issues/9

    Cu Selur
    users currently on my ignore list: deadrats, Stears555
    Quote Quote  
  7. Thank you Selur
    Quote Quote  
  8. Member
    Join Date
    Oct 2021
    Location
    South Africa
    Search PM
    I think I know how to solve your problem.
    The exact same thing happens when AI upscaling in ESRGAN. If you don't have enough memory it just flat out fails.

    However the reason it fails is not because you don't have enough memory. But because your GPU has a process timeout config. IE if it can't process what its
    being asked in the time you are asking it it will flat out deny you.

    Good news, Nvidia allows you to disable this if you'd like.

    https://docs.nvidia.com/gameworks/content/developertools/desktop/timeout_detection_recovery.htm

    You find it in NVIDIA Nsight tool:
    https://developer.nvidia.com/tools-overview

    Idk if it will actually work for you in this specific program but worth trying.
    Quote Quote