People with epilepsy need assistance and are at risk of sudden death (SUDEP) when having convulsive seizures. Automated real-time seizure detection systems can help alert caregivers, but current automated methods for detection of seizures rely on Electroencephalography (EEG) and wearable sensors, which are not always tolerated by patients. Visual motion clues such as facial expression, hand joint stiffness, and body convolutions (pose) are natural semiology features which an epileptologist observes to detect epileptic seizures. Our goal is to breach this gap by developing a robust deep learning framework for detecting and classifying seizures based on the video data.

In collaboration with the Alfred Hospital, we will collect some pilot video dataset of patients experiencing different epileptic seizures. This project comprises two parts. The first milestone is to extract privacy-preserving relevant features from the videos such as body pose, face landmark joints, hand joints, and optical flow information using a local hardware device - e.g., Nvidia Jetson Xavier. This part will involve setting up some open-source available networks e.g. Openpose, RetinaFace, etc. on the Jetson board. The second and next goal is to develop deep learning methods for detecting and classifying seizures based on the privacy-preserved extracted data from the first step and also detecting SUDEP. This part will involve experimenting with CNNs, LSTMs, Transformers, etc.  A way of enhancing LSTMs while also improving explainability is given in Fang, Dowe, Peiris and Rosadi (2021).