This contains the Jupyter notebooks (Tensorflow and Pytorch) used to pretrain the SinBERT models and evaluate monolingual and multilingual models for Sinhala text classification. The process and results are presented in "BERTifying Sinhala - A Comprehensive Analysis of Pre-trained Language Models for Sinhala Text Classification - Vinura Dhananjaya, Piyumal Demotte, Surangika Ranathunga, Sanath Jayasena, LREC, 2022". The datasets and pre-trained SinBERT models are available at https://huggingface.co/NLPC-UOM.