Feedback and assessment of expert surgeons in basic robotic-assisted minimally invasive surgery (RMIS) training is very time consuming. It is very favorable to automatically feed- back all the surgeons in different skill levels during the training process. Various methods have been proposed to assist the training and evaluation process for novice surgeons. In this project, we focused on this subject and have used the surgical robot kinematic dataset named JIGSAWS, to extract 43 features from its suturing data profile. Furthermore, by extracting critical features of tool movements, it has been attempted to categorize surgeons with high accuracy into three levels, novice, intermediate and expert surgeons. The features were extracted from time-, motion-, entropy-, and frequency-based indices. With the aid of feature selection methods, the features that make the most distinction between surgeons have been identified, and skill assessments have been based on them. To find out the most relevant features in skill assessment, we perform and compare two feature selection/reduction methods, namely principal component analysis (PCA) and relief algorithm. We separately reduce the features based on these two methods, while using a combined method (PCA+relief). Although resulting in an acceptable accuracy in skill level detection of 84% when using each of them separately, the combined method results in 92% particularly noteworthy accuracy. Moreover,  after presenting the proposed method and its validation on the  JIGSAWS  data set,  we apply it  to  the  dataset extracted from the ARASH:ASiST, our developed eye surgery training system. This experiments, which was on vitrectomy surgery, are performed by surgeons in three skill levels, a specialist surgeon, a surgical resident, and a person without any medical background. The experiments were performed on human eyes bank at Farabi Hospital. While, ARASH:ASiST recorded the position and speed data, the data set is pre-examined and the outliers data are removed. Then the acceleration and jerk indices are obtained by deriving the velocity and acceleration data, respectively. After collecting the data set  and  normalizing them, we apply the proposed method and classify the surgeons into three skill levels (novice, intermediate, expert) with an accuracy of 86%, which is remarkable given the real and unstructured environment.  Briefly, the main contribution of the thesis is to introduce a skill assessment method and its development based on different feature extraction/selection methods. Additionally, the  experimental  results  on  JIGSAWS  and  ARASH:ASiST data sets are presented to verify the proposed methodology, and to provide a promising skill assessment method for facilitating vitrectomy surgery training.

Surgery training is very delicate and of utmost importance for a novice, surgeon to become an expert. Surgery experience can significantly help novice surgeons in her/his surgery skills, However, with the cost of operating on real patients. Therefore surgery simulation may come as a rescue and by collecting surgeons’ operation data during surgery it is possible to enhance the training process and make it more efficient if collected data could be efficiently processed and means for improvements could be provided. In this project, different machine learning methods have been examined in surgical fields. For this means, ARASH:ASiST the ARAS haptic system for eye surgery training is used to extract data from surgeries and the surgery training procedure. Furthermore, the model for this system is used to test and implement the proposed machine learning methods. For the proof of concept, the JIGSAWS dataset and the JHU-ISI Gesture and Skill Assessment Working data Set is used, which is a surgical activity dataset for human motion modeling. JIGSAWS was captured using the da Vinci Surgical System from eight surgeons with different levels of skill performing five repetitions of three elementary surgical tasks on a bench-top model: suturing, knot-tying, and needle-passing, which are standard components of most surgical skills training curricula. The kinematics data of this dataset is used to train our machine learning models. First, every single surgery data is considered as a time series and recurrent neural networks (RNN), especially LSTM (long-short term memory) networks are trained on them. In another method, all our surgery kinematics data are converted to image data type and by use of convolutional neural networks (CNN) to classify our data and train our model. By Using expert surgeon’s data our machine learning model is trained to learn the best surgery pattern and use that pattern to evaluate any new data, extracted from novice surgeons. By this means, it is possible to use these methods to evaluate surgeons’ skills both online and offline also enables us to help them during surgery. In the final stage of this research, a haptic system simulator is used to train a reinforcement learning agent. By Using unity software, ARASH:ASiST model is developed into a labscale simulator to extract exhaustive data, and furthermore, to experience more surgery scenarios in the simulator to train our reinforcement learning agent, and implement our machine learning methods more freely on the simulator platform. It is possible to have a model to evaluate CNN or LSTM methods but different situations in surgery may cause problems in the assessment criteria, therefore in this stage of the research reinforcement learning is used as a complementary amendment. In this method, the simulator is used to experience almost all situations that may happen in surgery as our agent’s episodes and during these episodes, our reinforcement learning agent can learn different expert surgeon’s reactions in different situations. By this means, before any assessment, our reinforcement learning agent can make an optimal pattern for that surgery, and then that surgery pattern is used to evaluate novice or intermediate surgeons through previously developed methods.