V100-PCIE-32GB has ~12GB VRAM headroom after model weights. ornith:35b Q4_K_M KV cache at 64K ≈ 11.8GB — fits within budget. Model natively supports 256K but hardware caps safe limit at 64K.
V100-PCIE-32GB has ~12GB VRAM headroom after model weights. ornith:35b Q4_K_M KV cache at 64K ≈ 11.8GB — fits within budget. Model natively supports 256K but hardware caps safe limit at 64K.