LLM VRAM calculator.

Roughly the parameter count multiplied by the bytes per parameter, plus the context. A 7-billion-parameter model at 4-bit needs about 4 GB, at 8-bit about 7 GB, and at full 16-bit precision about 14 GB. A 70B model at 4-bit needs around 40 GB, which is beyond any single consumer card.

It stores each weight in fewer bits, so the model shrinks roughly in proportion. Going from 16-bit to 8-bit is close to free in quality terms; 4-bit is the usual sweet spot and the loss is small for most uses; below 4-bit the degradation becomes noticeable.

Yes — the layers are split across cards and the totals add up, so two 16 GB cards can hold a model that needs 30 GB. Expect some loss to the traffic between them, and note that the slower card sets the pace.

Whatever does not fit spills into system memory, and the parts held there run at a fraction of the speed, because the processor's memory is far slower than the card's. A model that mostly fits is usable; one that mostly does not is not.

Yes, and people routinely forget it. The attention cache grows with the number of tokens in play, and at long contexts it can take more memory than the model weights themselves. Size for the context you intend to use, not the smallest one.