基於Transformer的化合物-蛋白質交互作用預測方法之改進

Abstract

近年來，化合物-蛋白質交互作用 (Compound-Protein Interaction, CPI) 預測已經成為計算化學領域的研究熱點之一。隨著深度學習技術的興起，越來越多的基於神經網路的CPI預測方法得到了開發和應用。其中，Transformer模型是採用自注意力機制 (Self-attention) 的深度學習模型，具有強大的建模能力，因此有越來越多模型使用了此方法。不過，基於此方法的模型在預測CPI的任務上存在著一些問題，例如訓練的成本太大、對於3D空間相互作用的捕捉能力較弱等，而這些問題也影響到預測的準確率。為了找到比傳統Transformer還更能準確預測的方法，我們從模型架構、輸入特徵的選擇以及損失函數等面向尋找改進的方法，期望能找出可以提升準確率，甚至降低運算成本的方法。本論文以CAT-CPI (Ying et al., 2022) 的模型架構為基礎，結合TransformerCPI (Chen et al., 2020) 對於化合物特徵的提取方式，提出了基於Transformer的CPI預測之改進方法。TransformerCPI針對一維的SMILES序列產生了對應的原子特徵，而CAT-CPI則是使用二維的化合物圖像作為輸入，利用CNN學習化合物圖像的局部細節特徵，並且取得了優秀的結果。因此本模型結合兩者的特色，同時以一維的原子特徵和二維的分子圖像作為輸入，利用不同的化學結構資訊互補來提高模型的預測能力。此外我們也嘗試以Performer、Conformer等不同的架構取代傳統的Transformer來提升預測的準確率與運算的速度，並觀察不同的損失函數 (Loss Functions) 對於訓練結果的影響。我們使用Human、Celegans以及Davis資料集對所有改進方法進行實驗，發現與只使用分子圖像的方法相比，原子特徵與分子圖像結合的輸入能有效提升預測的準確率，且以Performer和Conformer等模型取代Transformer也可些微提升預測的能力。

In recent years, Compound-Protein Interaction (CPI) prediction has emerged as one of the major research focuses within the field of computational chemistry. With the rapid development of deep learning technologies, an increasing number of neural network-based CPI prediction methods have been proposed and adopted. Among them, Transformer models—which leverage the self-attention mechanism—offer strong modeling capabilities and have thus become widely utilized. However, Transformer-based approaches still face several challenges in CPI prediction tasks, such as high computational costs and limited ability to capture three-dimensional spatial interactions, which in turn affect predictive accuracy. To explore more effective methods beyond the conventional Transformer architecture, we investigate possible improvements in model design, input representation, and loss function strategy, aiming to enhance prediction performance while potentially reducing computational burden. This thesis presents an improved Transformer-based approach for CPI prediction by building upon the architecture of CAT-CPI (Ying et al., 2022) and incorporating the compound feature extraction technique used in TransformerCPI (Chen et al., 2020). TransformerCPI generates atomic-level features from one-dimensional SMILES sequences, while CAT-CPI utilizes two-dimensional compound images as input and applies convolutional neural networks (CNNs) to learn local structural details, achieving promising results. Our proposed model integrates both approaches by simultaneously using one-dimensional atomic features and two-dimensional molecular images as inputs, allowing complementary chemical structure information to enhance prediction capability. Additionally, we explore alternative model architectures such as Performer and Conformer to replace the standard Transformer, aiming to improve prediction accuracy and computational efficiency. We also examine the impact of different loss functions on training outcomes. Experiments conducted on the Human, Celegans, and Davis datasets demonstrate that incorporating both atomic features and molecular images yields better predictive performance compared to using molecular images alone. Moreover, replacing the Transformer with Performer or Conformer models results in moderate improvements in accuracy.