Since Human leukocyte antigen (HLA) can bind foreign peptides to present them to specialized immune cells andinitiate an immune response, accurate prediction of binding between HLA and neoepitope is critical for targetidentification in immunotherapy . However, current algorithms for predicting neoantigens are resulting in high falsepositives and most of them are limited by fixing model input length. In this study, we proposed an allele-specificand transformer-based model to predict antigen presentation in the context of HLA class I alleles. This modelbenefits from ProtBERT which is a pre-trained transformer on proteins to encode peptides and does not needto fix peptide length. Then, we use random forest and multilayer perceptron networks as a classifier on encodedpeptides. The dataset we used was obtained from the immune epitope database (IEDB) as in previous works andincludes peptide and HLA pairs of 20 high-frequency HLA-A and HLA-B allotypes. Results show that our proposedmodel outperforms the former methods in terms of positive predictive value (PPV) (0.58 vs.0.38 on average). Ourbest result was obtained on HLA-A*01:01 with a PPV value of 0.872, which is 0.286 and 0.267 in APPM andnetMHCpan-4.0 models, respectively. Also, using a pre-trained encoder allows the model to predict more quicklyand with less computational e↵ort. On the other hand, these results confirm that transformers can be used as anembedding that extracts structural properties from the sequence. Since our model only requires the peptide sequenceof HLA-peptide binding pairs, it can be applied to other binding problems without the need for structure data.