What's Quantization Parameter (QP) in Video Encoder?

Quantization Parameter (QP) plays a crucial role in video encoding, affecting both compression efficiency and video quality. Let's explore what QP is and how it works,

1. Definition:
   - QP is a dimensionless value used during video compression.
   - It determines the quantization step size applied to transform coefficients within video frames.
   - Quantization reduces the precision of coefficients, leading to compression.
   - Larger QP values result in more aggressive quantization, yielding smaller file sizes but potentially lower quality.
   - Smaller QP values preserve more detail but increase file size.

2. How It Works:
   - During video encoding, each frame is divided into smaller blocks called macroblocks.
   - A quantization matrix is applied to these blocks based on their spatial frequency content.
   - The resulting values are rounded off based on the selected QP.
   - High QP values lead to coarser quantization, emphasizing compression.
   - Low QP values maintain more detail but may result in larger files.
   - QP ranges from 0 to 51 in H.264/H265, where 0 represents lossless encoding and 51 represents strong compression.
   - Note that libvpx does not have a fixed QP mode.

In summary, QP allows video encoders to strike a balance between file size and quality by adjusting quantization levels. 

For Oupree Video Encoder, Min QP range from 1-35, preset value is 5, and 42 for Max QP.

(1) Understanding Rate Control Modes (x264, x265, vpx). https://slhck.info/video/2017/03/01/rate-control.html.