Abstract:

Although deep learning has achieved remarkable results in the industry, to achieve the ultimate prediction accuracy improvement, many existing models with poor performance will be added with other modules, which will become more and more bloated. To achieve the deployment of the lightweight network in an industrial system, knowledge distillation (KD) is currently an effective way to reduce model size. This study focused on the research of knowledge transfer logic and decoupled the interactive information into 3-probability spaces, namely the target space (t), the intraclass class space (o\t), and the out-of-class class space (c\o). The 3-probability spaces method is named TOCKD (t-o\t-c\o spaces knowledge distillation), which introduces three components corresponding to 3-probability spaces, that is target knowledge distillation (TKD), intraclass knowledge distillation (OKD), and out-of-class knowledge distillation (CKD). Based on the support of the flip-chip vibration signal with defect information obtained by ultrasonic excitation experiment, the research confirms the primacy of TKD in the information transmission process and the auxiliary properties of the other two components by qualitative analysis. To realize the quantitative analysis of the corresponding coefficients of three components in TOCKD, Tree-structured Parzen Estimator (TPE) is used to search the coefficients. On the basis of it, combined with subjective and objective weight analysis methods, the guide value of coefficients is defined. This research verifies the excellent performance and robustness of TOCKD by analyzing the flip-chip vibration signals, and hopes that TOCKD will be helpful for future research and the application of lightweight technology in the engineering field.

This article is available online here and to CALCE Consortium Members for personal review.