This title slide presents "Int8 Quantization and Training Optimization Practices" as the main topic. The subtitle describes it as "Deep Learning Model Optimization: Sharing from Principles to Practices."

Int8量化与训练优化实践

深度学习模型优化：从原理到实践分享

Source: 技术分享PPT，面向深度学习工程师

This agenda slide outlines a presentation on quantization in deep learning, starting with its background and significance. It then covers Int8 principles, PTQ vs. QAT comparison, framework support with trade-offs, and MobileNet experiments with summary and outlook.

分享大纲

1. 1. 量化的背景与意义

介绍量化技术的背景、必要性及其在深度学习中的意义。

2. 2. Int8量化原理

详解Int8量化的核心原理和数学基础。

3. 3. PTQ与QAT对比

对比训练后量化(PTQ)与量化感知训练(QAT)的差异。

4. 4. 框架支持与权衡

主流框架支持及精度、性能的权衡分析。

5. 5. 实验、总结与展望

MobileNet量化实践案例、总结及未来展望。 Source: Int8量化与训练优化实践

This section header slide, titled "Int8 Quantization and Training Optimization Practices," introduces Section 1: "Background and Significance of Quantization." The subtitle outlines model deployment pain points, edge computing demands, and quantization benefits like reduced memory/computation and faster inference speed.

Int8量化与训练优化实践

1

量化的背景与意义

模型部署痛点、边缘计算需求与量化益处：降低内存计算，提升推理速度

The slide outlines challenges of deploying floating-point models, including high memory and power consumption, especially on resource-limited mobile and edge devices. It explains quantization's goal of 8-bit integer representation for 2-4x acceleration, with INT8 as the emerging industry mainstream.

量化背景详解

• 浮点模型部署挑战：高内存、高功耗
• 移动/边缘设备：资源严格限制
• 量化目标：8bit整数表示，加速2-4x
• 行业趋势：INT8量化成主流

This slide diagram illustrates the benefits of INT8 quantization compared to FP32. It shows model size reduced to 1/4, 2-4x faster inference speed, significantly lower memory usage, and efficient deployment on edge devices.

量化意义示意图

!Image

• 模型大小：INT8 缩小至 FP32 的 1/4
• 推理速度：提升 2-4 倍性能
• 内存占用：显著降低，节省资源
• 部署优势：边缘设备高效运行

Source: Image from Wikipedia article "Large language model"

This slide is a section header for Section 2: Int8 Quantization Principles. It subtitles the mapping of weights and activations from FP32 to INT8 via dynamic/static range calibration and zero point/scale calculations.

2. Int8 量化原理

2

Int8 量化原理

权重/激活值从 FP32 映射到 INT8：动态/静态范围校准与零点/尺度计算

Source: Int8量化与训练优化实践 PPT

The slide details the INT8 quantization workflow in four steps: collecting activation statistics via model forward propagation, computing min/max values, calculating scale as (max - min)/255, and applying the quantization formula q = round(x / scale + zp). This process converts floating-point activations to INT8 format using per-tensor scaling and zero-point.

INT8量化流程

{ "headers": [ "步骤", "描述", "公式/细节" ], "rows": [ [ "1. 收集激活统计", "通过模型前向传播收集激活值的统计信息", "统计激活分布" ], [ "2. 计算min/max", "从收集的统计中提取激活的最小值和最大值", "min = min(activations) max = max(activations)" ], [ "3. 计算尺度", "基于范围计算量化尺度", "scale = (max - min) / 255" ], [ "4. 量化公式", "将浮点值转换为INT8量化值", "q = round(x / scale + zp)" ] ] }

The slide outlines the PyTorch code and formula for asymmetric quantization, mapping dynamic input ranges to [0,255]. It calculates scale as (max - min)/255, applies quant = torch.round(x / scale + zeropoint).clamp(0, 255), and implements the non-symmetric mapping.

量化公式与代码

• scale = (max - min) / 255 // 缩放因子
• quant = torch.round(x / scale + zeropoint) // 量化计算
• .clamp(0, 255) // 范围裁剪
• 实现动态范围到[0,255]的非对称映射

Source: scale = (max - min) / 255 quant = torch.round(x / scale + zero_point).clamp(0, 255)

This section header slide marks Section 3, titled "PTQ vs QAT Comparison." The subtitle contrasts Post-Training Quantization (PTQ) with Quantization-Aware Training (QAT).

3

PTQ vs QAT对比

训练后量化(PTQ) vs 量化感知训练(QAT)

The slide compares PTQ (Post-Training Quantization) and QAT (Quantization-Aware Training) in a two-column format. PTQ offers simple, fast quantization without retraining but with significant accuracy loss, while QAT simulates quantization during training for higher precision at the cost of increased training effort.

PTQ 与 QAT 对比

The slide "Mainstream Framework Support" compares quantization capabilities across PyTorch, TensorFlow, and ONNX Runtime. It lists PTQ (e.g., ✓ for PyTorch, tf.lite for TensorFlow), QAT (e.g., torch.ao.quantization, Model Optimization), and optimizers (e.g., FXGraph, TPU, CUDA/CPU).

主流框架支持

{ "headers": [ "框架", "PTQ", "QAT", "优化器" ], "rows": [ [ "PyTorch", "✓", "torch.ao.quantization", "FXGraph" ], [ "TensorFlow", "tf.lite", "Model Optimization", "TPU" ], [ "ONNX Runtime", "ORTQuant", "QOperator", "CUDA/CPU" ] ] }

The slide on precision-performance trade-off reports a minimal Top1 accuracy drop of 0.5-2%. It delivers a 3x inference speed gain and 4x memory usage reduction.

5. 精度与性能权衡

• 0.5-2%: Top1 Acc Drop

Minimal precision degradation

• 3x: Inference Speed Gain

Significant acceleration boost

• 4x: Memory Usage Cut

Drastic resource reduction

The slide on MobileNet quantization experiments shows MobileNetV2 INT8 achieving 71.8% ImageNet Top-1 accuracy (vs. 72.0% for FP32). It highlights a 2.8x inference speed improvement using PyTorch's torch.quantization.quantizedynamic(model).

MobileNet量化实验

!Image

• MobileNetV2 INT8: ImageNet Top1 71.8% (FP32: 72.0%)
• Inference speed improved by 2.8x
• PyTorch: torch.quantization.quantize_dynamic(model)

Source: Wikipedia

This slide's practical key points recommend using COCO/ImageNet datasets for model calibration and torch.quantization tools for implementation. It addresses accuracy drops from abnormal activation distributions via fusion operators and KMeans clustering optimizations.

实践要点

• 选择COCO/ImageNet数据集进行模型校准
• 使用torch.quantization工具实现量化
• 应对激活分布异常精度下降挑战
• 采用融合算子和KMeans聚类优化方案

INT8 quantization significantly boosts deployment efficiency, with QAT ensuring accuracy preservation. Future work includes INT4/mixed quantization and NAS automation, ending with thanks and Q&A.

总结与未来展望

INT8量化显著提升部署效率，QAT为精度保障。 未来：INT4/混合量化、NAS自动化。

感谢聆听！Q&A？