For inference on 7B to 13B class models, fine-tuning with LoRA, and classic FP64 compute, the A100 40GB delivers most of the capability of far newer cards at a fraction of the price. It is the wrong card for unquantized 70B-class work. Know which side of that line your workload sits on before you spend.
The NVIDIA A100 40GB launched in 2020 as the flagship of the Ampere generation. Two hardware generations later, it has quietly become one of the best price-to-performance buys in enterprise AI, because the market prices GPUs on novelty while workloads only care about throughput and memory.
What you actually get
The A100 40GB brings third-generation Tensor Cores with TF32 and BF16 support, NVLink for multi-GPU scaling, and Multi-Instance GPU (MIG), which splits one card into up to seven isolated instances. MIG matters more than most spec sheets suggest: for inference fleets serving many small models, one A100 can do the work of several smaller cards with cleaner isolation.
The market prices GPUs on novelty. Workloads only care about throughput and memory.
How it stacks up
| Spec | A100 40GB | A100 80GB | H100 80GB |
|---|---|---|---|
| Memory | 40GB HBM2 | 80GB HBM2e | 80GB HBM3 |
| Bandwidth | ~1.6TB/s | ~2.0TB/s | ~3.35TB/s |
| Tensor Cores | 3rd gen | 3rd gen | 4th gen, FP8 |
| MIG | Up to 7 | Up to 7 | Up to 7 |
| Sweet spot | Inference, fine-tuning, HPC | Mid-size training | Frontier training |
| Relative cost | Lowest | Mid | Highest |
Where it fits
- Inference at scale. Models in the 7B to 13B class, quantized, run comfortably in 40GB with room for batching.
- Fine-tuning and experimentation. LoRA and QLoRA workflows on mid-size models fit well inside 40GB, and interconnected pairs cover most training runs a growing team actually does.
- Classic HPC and data processing. FP64 throughput remains strong, and plenty of scientific and rendering workloads never needed more memory than this.
Where it does not
Honest caveat: if your workload is training or serving models that need 70B-class weights unquantized, the 40GB card is the wrong tool. That is what the A100 80GB and H100 are for, and we sell those too. Buy for the workload you have, not the one on the conference slide.
What to check before buying secondary-market A100s
- Form factor. SXM4 modules need a compatible baseboard; PCIe cards drop into standard servers.
- Thermal history and testing. Every card should pass a full functional and stress test before it ships. Ours do, and we say so per unit.
- Warranty. A card without one is a different asset class. Core 4 hardware carries an advanced replacement warranty.
Pricing on 40GB cards moves with the market, so we quote current numbers rather than publishing stale ones. Talk to a rep for today's pricing, or browse what is on the shelf right now.