VRAM (Video RAM)
< GlossaryVideo memory of a graphics processor. Determines the maximum model size that can be loaded for inference. Llama 70B in FP16 requires ~140 GB VRAM.
Video memory of a graphics processor. Determines the maximum model size that can be loaded for inference. Llama 70B in FP16 requires ~140 GB VRAM.