In the healthcare research community, Internet of Medical Things (IoMT) is transforming the healthcare system into the world of the future internet. In IoMT enabled Computer aided diagnosis (CAD) system, the Health-related information is stored via the internet, and supportive data is provided to the patients. The development of various smart devices is interconnected via the internet, which helps the patient to communicate with a medical expert using IoMT based remote healthcare system for various life threatening diseases, e.g., brain tumors. Often, the tumors are predecessors to cancers, and the survival rates are very low. So, early detection and classification of tumors can save a lot of lives. IoMT enabled CAD system plays a vital role in solving these problems. Deep learning, a new domain in Machine Learning, has attracted a lot of attention in the last few years. The concept of Convolutional Neural Networks (CNNs) has been widely used in this field. In this paper, we have classified brain tumors into three classes, namely glioma, meningioma and pituitary, using transfer learning model. The features of the brain MRI images are extracted using a pre-trained CNN, i.e. GoogLeNet. The features are then classified using classifiers such as softmax, Support Vector Machine (SVM), and K-Nearest Neighbor (K-NN). The proposed model is trained and tested on CE-MRI Figshare and Harvard medical repository datasets. The experimental results are superior to the other existing models. Performance measures such as accuracy, specificity, and F1 score are examined to evaluate the performances of the proposed model.