AWQ, or Activation-aware Weight Quantization, is a technique for compressing and accelerating large language models (LLMs) by quantizing the model's weights based on the observation that not all weights are equally important. This method takes into account the activations of the model during the quantization process, searching for optimal per-channel scaling to protect salient weights by observing the activation, not just the weights. AWQ does not rely on backpropagation or reconstruction, preserving LLMs' generalization ability on different domains and modalities. It has been shown to outperform existing work on various language modeling and domain-specific benchmarks, achieving excellent quantization performance for instruction-tuned LMs and multi-modal LMs. AWQ can significantly reduce memory requirements and latency, making it feasible to deploy large models on a wider range of devices.