MSc Projects 2018-19

Please email me at This email address is being protected from spambots. You need JavaScript enabled to view it. if you are interested in any of these topics and would like to have a discussion.

Keywords: Classification, Clustering, Computer Programming (Matlab, Python, TensorFlow), Computational Intelligence, Convotional Neural Network, Deep Learning,  Deep Q Network, Deep Reinforecement Learning, Ensemble Classification, Fuzzy Logic, Neural Network, Machine Learning, Population-Based Search Algorithms (Genetic Algorithm, Particle Swarm Optimisation, Q-learning, etc.), Reinforcement Learning, Support Vector Machine, Self-Organizing Map, Supervised Learning, Unsupervised Learning

Requirements: Hardworking, self-motivated, creative, good programming skills, willing to learn.

Remarks: 

• All projects are open-ended. They are basically at MSc level but can be taken into research level dpeending on problem formulation and the way approach the problem.
• High Performence Computing (HPC) service can be used to spped up the learning process
  • To creat an HPC account, raise a ticket in Rosalind ticket system (https://helpdesk.rosalind.kcl.ac.uk/ )(accessible from university network only) and complete the form after choosing “Request new account” in help topics.
  • Further information about HPC can be found in https://user-wiki.rosalind.kcl.ac.uk/doku.php?id=hpc_quick_start_guide
    
• Students need to gain some knowledge which is new to them.
• Basically I categorise the project outcome into the following three levels (basic, merit and distinction) according to 1) the challenging level of the problem, 2) scientific and academic contributions (insights, ideas, knowledge, novelity), 3) the breadth and depth (coverage, quality, level of sophistication) of design, methodology, and results
• Encourage to publish the contributions and achievements in a journal paper.
• The project descriptions below are just to give a brief idea. Details project aim and objectives will be discussed when projects start.

[HKL1]: Grasping objects with reinforcement learning 

[HKL02]: Using reinforcement learning algorithms to control UAV

The dynamic system of UAV has high nonlinearity and instability which makes generating control policy for this system a challenging issue. However, reinforcement learning algorithm as one of the machine learning algorithms can learn control policy by simply interacting with environment without knowing the dynamic model of the environment. Thus, this algorithm proposes a prospecting approach to controlling UAV. This project encourages to apply various reinforcement learning algorithms such as Q-learning algorithm, SARSA, actor-critic algorithm, etc, to control the movement of UAV either in simulation environment or on a real-world platform.

Here are some potential simulation environments.

• GymFC: This environment requires installation of Gazebo 8 on Ubuntu operating system.
• ROS Development Studio: There are several UAV models available and can be launched in browser with several tools such as OpenAI libraries, Gazebo Simulator ready to use.

There are also other different versions of simulation environments available online and you are encouraged to discover more.

Here are some papers and tutorials recommended:

• An Application of Reinforcement Learning to Aerobatic Helicopter Flight
• Autonomous Helicopter Aerobatics through Apprenticeship Learning
• Autonomous Quadrotor Landing using Deep Reinforcement Learning
• Control of a Quadrotor with Reinforcement Learning
• Reinforcement Learning: An Introduction (2nd Edition)

Some similar applications can be found as follows:

• Control of a Quadrotor with Reinforcement Learning
• UAV to Perform a Perched Landing
• Deep reinforcement learning for UAV in Gazebo simulation environment
• Autonomous Quadrotor Landing using Deep Reinforcement Learning
• Reinforcement Learning for Autonomous Navigation of UAV's

[HKL03]: Skill learning via reinforcement learning and imitation learning

One of the advantages of reinforcement learning is the ability to grasp skills via interacting with environment and figuring out the optimal policy by itself. This project let robot learn certain skill when demonstrations are given first and improved using reinforcement learning algorithms afterwards or make the robot explore the optimal policy by itself (like the application linked below named 'Reinforcement learning for a robotic soccer goalkeeper' which learns to stop the ball coming from random directions before hitting the goal). Since the control signals applied to robotics are usually continuous, policy based reinforcement learning algorithms with continuous action space are encouraged to be implemented such as actor-critic method, Deep Deterministic Policy Gradient (DDPG) methods as well as the model-based reinforcement learning algorithms.

The project can either be done in simulation environment or on a real platform. One possible training method is to use OpenAI Gym and MuJoCo to implement a few tasks such as hopper, ant, HalfCheetah, Humanoid, Reacher, etc., with source code available from Github. Other simulation environments are also encouraged to discover and test.

Here are some papers and tutorials recommended:

• Continuous control with deep reinforcement learning
• A Survey of Actor-Critic Reinforcement Learning: Standard and Natural Policy Gradients
• Actor-Critic Algorithms
• Natural Actor-Critic
• Deep Reinforcement Learning and Imitation Learning Courses From UC Berkeley

Some similar applications can be found as follows:

• Robot Learns to Flip Pancakes
• Towards Learning Robot Table Tennis
• Learning Motor Primitives for Robotics
• Reinforcement learning for a robotic soccer goalkeeper

Installation of OpenAI Gym and MuJoCo are as follows:

• OpenAI Gym
• mujuco-py

[HKL04]: Traffic management with deep reinforcement learning algorithms

Traffic issues in real world usually come with enormous information but less inefficiency. These problems cause the road intersection and energy waste. Deep reinforcement learning algorithms combine the deep learning algorithms with reinforcement learning algorithm can dynamically generate control policy for the real-time traffic situation. This project aims to apply various deep reinforcement learning algorithms to recognise different traffic cases and generate optimal control policy to make the whole traffic environment more efficient.

Some open-source simulation environments are available for training:

• SUMO (Simulation of Urban MObility): This is an open source traffic simulation package which provides microscopic and continuous road traffic simulation of large road networks. Python is needed for running this simulation. The tutorial of SUMO is available here.
• MATSim (Multi-Agent Transport Simulation): MATSim provides a framework to implement large-scale agent-based transport simulations.
• Transims (TRansportation ANalysis SIMulation System): Transims is under NASA open source and integrates tools which conducts transportation system analyses.

Here are some papers and tutorials recommended:

• Deep Reinforcement Learning for Traffic Light Control in Vehicular Networks
• Reinforcement Learning With Function Approximation for Traffic Signal Control

Some similar applications can be found as follows:

• Reinforcement Learning For Traffic Signal Control
• Traffic Signal Management and Control System
• Reinforcement Learning based Traffic Signal Control Validated in Real-Time Real World Traffic

[HKL05]  Auto parking with reinforcement learning

This project aims to let car or robots learn to park autonomously using reinforcement learning algorithms either in a simulation environment or in on a real platform. Q-learning with epsilon greedy method or policy gradient methods can be considered as to apply control to the move the car or robot to the desired parking slot.

Here are some papers and tutorials recommended as references:

• Automatic Car Parking: A Reinforcement Learning Approach: https://link.springer.com/content/pdf/10.1007%2F3-540-44868-3_28.pdf
• Deep Reinforcement Learning framework for Autonomous Driving: https://arxiv.org/pdf/1704.02532.pdf
• Reinforcement Learning: An Introduction (2nd Edition): http://incompleteideas.net/book/bookdraft2017nov5.pdf

Here are some applications on Youtube:

• Automatic Parking with Reinforcement Learning:

• Automatic Parking with Reinforcement Learning in Matplotlib

• Learning to Park | A collaboration between NNAISENSE and Audi Electronics Venture

[HKL06]  Auto driving combining with reinforcement learning

Auto driving is a concept that has risen widely attention in recent years. An auto driving vehicle would be able to grasp the driving skills and learn the traffic rules using tons of data from different sensors. As to make vehicles make proper decisions during driving, the training process turns out to be crucial. Reinforcement learning as a branch of machine learning algorithms could figure out optimal policie by interacting with environment by maximising the expectation of accumulated reward in long term has been widely used and proved to be an effective technique.

This project aims to apply reinforcement learning algorithms such as Q-learning algorithm, DQN (Deep Q-Network), DDPG (Deep Deterministic Policy Gradients) etc., to train vehicles to learn drive autonomously.

Following are some papers and tutorials as reference:

• Learning to Drive in a Day: https://arxiv.org/pdf/1807.00412.pdf
• Deep Reinforcement Learning for Driving Policy:

• Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation: https://arxiv.org/abs/1709.10489
• MIT 6.S094: Deep Reinforcement Learning for Motion Planning: https://www.youtube.com/watch?v=QDzM8r3WgBw

Here are some applications on Youtube:

• Deep Learning Cars:

• Reinforcement Learning for Autonomous Driving Obstacle Avoidance using LIDAR

• Self Driving Digital Car using Python CNN Reinforcement learning

• Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation

[HKL07]: Classification of Medical Images of Intraepithelial Papillary Capillary Loop using Deep Learning Techniques

'Nonerosive reflux disease (NERD) is defined as the presence of typical symptoms of gastroesophageal reflux disease caused by intraesophageal acid in the absence of visible oesophageal mucosal injury at endoscopy.' (Fass, Fennerty and Vakil, 2001)

Niwa et al (2009) have done much research on feature analysis and diagnosis of NERD. They summarised that the images from magnifying endoscopy show the Intraepithelial Papillary Capillary Loops (IPCLs) at the lower side of the oesophagus in NERD patients dilated and elongated with the regular arrangement.

In order to analyse the morphology of IPCL (whether the IPCLs dilate and elongate with the regular arrangement), the regions containing the clear shape of IPCL in the medical images should be extracted firstly. The lower image in Figure 1 shows the segmentation for the medical image in the upper one. It has been shown experimentally that using the segmented images to train a deep convolutional neural network to do the diagnosis of NERD can get a higher diagnosis accuracy compared with using the original images.

This project aims to develop an algorithm based on deep learning techniques to automatically segment the useful regions in the medical images of IPCL and perform classification. A set of the original images and segmented images will be provided for demonstrating which regions should be remained.

In order to achieve the project aim, the following objectives are defined which have to be polished when project starts:

1. Explore feature extraction methods to extract significant characteristic from datasets to support classification.
2. Develop algorithms to segment the images which is to remove unuseful information from the data.
3. Develop classifiers which make use of the segmented image as information to judge (or predict) if the input image is suffering from the disease or not.
4. Comparing and evaluate the results with different techniques, say, deep structured classifiers, auto encoders, traditional classifiers (KNN, support vector machines, probability models, statistical techniques, neural networks, etc), ranging from supervised to unsupervised learning techniques. Unsupervised learning techniques such as self-organising maps, k-means clustering, fuzzy k-means clustering.
5. Improve and evaluate the recognition performance (robustness) of classifiers when the input is subject to noise, say, using ensemble methods.

This project should be implemented with Python and TensorFlow.

Figure 1. Upper image: An original medical image (from King's hospital). Lower image: The extracted image.

References:

1. Albelwi, S., & Mahmood, A. (2017). A framework for designing the architectures of deep convolutional neural networks. Entropy, 19(6), 242.
2. Chollet, F. (2015). Keras: Deep learning library for theano and tensorflow. URL: https://keras.io/.
3. Fass, R., Fennerty, M. B., & Vakil, N. (2001). Nonerosive reflux disease—current concepts and dilemmas. The American journal of gastroenterology, 96(2), 303.
4. Hajjar, C., & Hamdan, H. (2012). Clustering of Interval Data Using Self-Organizing Maps–Application to Meteorological Data. In Applied Computational Intelligence in Engineering and Information Technology (pp. 135-146). Springer, Berlin, Heidelberg.
5. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems (pp. 1097-1105).
6. Niwa, H. (2009). New challenges in gastrointestinal endoscopy. Springer Science & Business Media.

Useful Links:

• TensorFlow - An open source machine learning framework for everyone: https://www.tensorflow.org/
• Unsupervised Feature Learning and Deep Learning Tutorial: http://ufldl.stanford.edu/wiki/index.php/UFLDL_Tutorial
• CS231n Convolutional Neural Networks for Visual Recognition: https://cs231n.github.io/convolutional-networks/
• Self-organizing Maps - An Example Analysis and Script: http://www.pymvpa.org/examples/som.html
• Self-Organizing Maps with Google’s TensorFlow: https://codesachin.wordpress.com/2015/11/28/self-organizing-maps-with-googles-tensorflow/

[HKL08]: EMG-Based Classification of Hand Movements with Deep Learning Techniques

Electromyography (EMG) is an electrodiagnostic medicine technique for evaluating and recording the electrical activity produced by skeletal muscles. EMG is performed using an instrument called an electromyograph to produce a record called an electromyogram. An electromyograph detects the electric potential generated by muscle cells when these cells are electrically or neurologically activated.

An artificial arm could be controlled according to the EMG signals if we can design an algorithm based on the EMG signals that can recognise the hand movements a human would like to move. In this case, a high classification accuracy should be required to avoid mis-controlling the artificial arm.

This project aims to develop classifiers based on deep learning techniques to automatically recognise 9 types of hand movements according to the EMG signals and compare the results with traditional classification techniques. In order to achieve this aim, a machine learning model should be designed and trained with a set of training samples. Then, the model should be tested and analysed with a test set. A set of EMG signals with labels will be provided.

Zia et al (2018) have done some similar works. They have evaluated the performance of stacked sparse autoencoders (SSAE), an unsupervised deep learning technique, with both handcrafted features (SSAE-f) and raw EMG samples (SSAE-r) extracted from varying lengths (1–15 days) or recorded EMG. Intrasession, intersession, and inter-days analyses were performed, and the performance of both the convolutional neural network (CNN) and SSAE were compared with state-of-the-art linear discriminant analysis (LDA). They demonstrated that CNN achieved higher accuracies in most of the analyses, and proved that CNN was more robust and computationally efficient than SSAE-f. The details of their work can be found in their paper (Zia et al, 2018). In order to improve the classification accuracy, some novel structures of deep CNN could be applied, such as variable weight neural networks (Lam et al, 2015). This project should be implemented with Python and TensorFlow.

References:

1. Lam, H. K., Ekong, U., Xiao, B., Ouyang, G., Liu, H., Chan, K. Y., & Ling, S. H. (2015). Variable weight neural networks and their applications on material surface and epilepsy seizure phase classifications. Neurocomputing, 149, 1177-1187.
2. Ling, S. H., Leung, F. H. F., Lam, H. K., & Iu, H. H. C. (2009). Variable-structure neural network with real-coded genetic algorithm and its application on short-term load forecasting. International journal of information and systems sciences, 5(1), 23-40.
3. Zia ur Rehman, M., Waris, A., Gilani, S., Jochumsen, M., Niazi, I., Jamil, M., ... & Kamavuako, E. (2018). Multiday EMG-based classification of hand motions with deep learning techniques. Sensors, 18(8), 2497.

Useful Links:

• TensorFlow - An open source machine learning framework for everyone: https://www.tensorflow.org/
• Unsupervised Feature Learning and Deep Learning Tutorial: http://ufldl.stanford.edu/wiki/index.php/UFLDL_Tutorial
• CS231n Convolutional Neural Networks for Visual Recognition: https://cs231n.github.io/convolutional-networks/ 

 The following projects are for Data Science Project Students Only (7CCSMDPJ)

[HKL09]:Lung Cancer Detection and Diagnosis

Lung cancer is a malignant lung tumor characterized by uncontrolled cell growth in tissues of the lung [1]. This growth can spread beyond the lung by the process of metastasis into nearby tissue or other parts of the body. Most cancers that start in the lung, known as primary lung cancers, are carcinomas [1]. To provide accurate diagnosis of the lung cancer, the common methods include performing a surgical biopsy which might expose the patient to risks and increases the healthcare costs [2]. Besides, computer aided diagnosis systems (CAD) which could assist radiologists and increase their diagnostic accuracy have been conducted.

In this research, we aim to use deep learning techniques to detect and recognize the tissue abnormalities in the lung and aid to suggest a probable differential diagnosis. The dataset which will be used in this research is LIDC-IDRI, which is the Lung Image Database Consortium Image Collection (LIDC-IDRI) consists of diagnostic and lung cancer screening thoracic computed tomography (CT) scans with marked-up annotated lesions. The methods that can be considered in this research are the convolutional neural networks (CNN), traditional neural networks (NN), k-nearest neighbors algorithm (KNN), support vector machine (SVM), Self-Organizing Map (SOM), fuzzy logic system, clustering method, etc.

[1]. Wikipedia contributors. "Lung cancer." Wikipedia, The Free Encyclopedia. Wikipedia, The Free Encyclopedia, 6 Nov. 2018. Web. 6 Nov. 2018.

[2]. Anthimopoulos M, Christodoulidis S, Ebner L, et al. Lung pattern classification for interstitial lung diseases using a deep convolutional neural network[J]. IEEE transactions on medical imaging, 2016, 35(5): 1207-1216.

[HKL10]: Classification of Electrocardiogram signal using LSTM Recurrent Neural Networks

ECGs record the electrical activity of a person's heart over a period of time. Physicians use ECGs to detect visually if a patient's heartbeat is normal or irregular. Long short-term memory (LSTM) networks, a type of recurrent neural network (RNN) well-suited to study sequence and time-series data [1], can be adopted to analyze and process the ECG signals. An LSTM network can learn long-term dependencies between time steps of a sequence as the LSTM layer (lstmLayer) looks at the time sequence in the forward direction, while the bidirectional LSTM layer (bilstmLayer) looks at the time sequence in both forward and backward directions [1]. LSTMs are designed to avoid the long-term dependency problem, so they can remember the information for long periods of time [2].

In this research, the dataset of ECG signal is from the PhysioNet 2017 Challenge [3], [4], [5], which is available at PhysioNet 2017 Challenge. This research aims to use Long Short-Term Memory (LSTM) networks and time-frequency analysis to classify the ECG data, which will assist the specialists in diagnosing the cardiac arrhythmia, lessening the workload of cardiologists and reducing the cost of data processing in the hospitals.

[1]. https://www.mathworks.com/help/signal/examples/classify-ecg-signals-using-long-short-term-memory-networks.html

[2]. http://colah.github.io/posts/2015-08-Understanding-LSTMs/

[3] AF Classification from a Short Single Lead ECG Recording: the PhysioNet/Computing in Cardiology Challenge, 2017. https://physionet.org/challenge/2017/

[4] Clifford, Gari, Chengyu Liu, Benjamin Moody, Li-wei H. Lehman, Ikaro Silva, Qiao Li, Alistair Johnson, and Roger G. Mark. "AF Classification from a Short Single Lead ECG Recording: The PhysioNet Computing in Cardiology Challenge 2017." Computing in Cardiology (Rennes: IEEE). Vol. 44, 2017 (in press).

[5] Goldberger, A. L., L. A. N. Amaral, L. Glass, J. M. Hausdorff, P. Ch. Ivanov, R. G. Mark, J. E. Mietus, G. B. Moody, C.-K. Peng, and H. E. Stanley. "PhysioBank, PhysioToolkit, and PhysioNet: Components of a New Research Resource for Complex Physiologic Signals". Circulation. Vol. 101, No. 23, 13 June 2000, pp. e215e220. http://circ.ahajournals.org/content/101/23/e215.full

[HKL11]: Classification of Epilepsy Seizure Phases and Electromyographic Hand Gesture Signal Classification using Ensemble Learning

Epilepsy is a common neurology disorder that is a chronic disease of the brain causing sudden paradoxical discharge of cortical neurons, which will cause significant impact on the living quality of sufferers and their carers [1]. The classification of seizure phases of epilepsy (seizure-free, pre-seizure and seizure phases) is very necessary in terms of providing the information of pre-seizure early detection, which helps sufferers and their carers have ample time to take precautions.

Many modern prostheses use electromyogram (EMG) signals generated by the human muscles to derive control commands for powered upper-limb prostheses. To accurately control the different finger postures of a prosthetic hand, the discriminating between individual and combined figures movements using surface EMG signals is of great significance [2]. The electromyogram (EMG) reflects the electrical activity of muscles. Usually, there is proportional relevance between the level of the muscle movements and current intensity.

The EEG and EMG data are the time series signal which are shown as follows.

This research aims to classify the epilepsy seizure phases and electromyographic hand gesture signal using ensemble learning method. Our research groups have developed effective classfication models as well as the data processing methods to deal with these two kinds of signal. Ensemble learning aims to improve the machine learning results by combining several models, which is actually the meta-algorithm that combine several machine learning techniques into one predictive model in order to decrease variance (bagging), bias (boosting), or improve predictions (stacking). This approach generally provides better predictive performance compared to a single model. In this research, the ensemble learning will be adopted to take advange of the existed models and further improve the classification accuracy.

Available datasets:

1. EMG Datasets

2. EEG Motor Movement/Imagery Dataset

[1]. Lam H K, Ekong U, Xiao B, et al. Variable weight neural networks and their applications on material surface and epilepsy seizure phase classifications[J]. Neurocomputing, 2015, 149: 1177-1187.

[2]. R. N. Khushaba, M. Takruri, S. Kodagoda, and G. Dissanayake, Toward Improved Control of Prosthetic Fingers Using Surface Electromyogram (EMG) Signals, in: Expert Systems with Applications 2012, 39: 10731–10738.

[HKL12]: Fault Diagnosis with Machine Learning methods

Intelligent fault diagnosis of machinery is an important pattern recognition problem. The effective diagnosis of the healthy state of machinary based on the many types of signals plays a significant role in increasing the safety and reliability of machinary and reducing operation and maintenance costs [2].

A well designed fault diagnosis strategy can save operating costs by minimizing service downtime and component replacement costs. The strategy benefits from a good knowledge about the operating machine's dynamics which is used in combination with sensor measurements to detect and isolate different kinds of faults. There are several ways to apply machine-learning techniques to the problem of fault detection and diagnosis. Classification is a type of supervised machine learning in which an algorithm “learns” to classify new observations from examples of labeled data. In the context of fault detection and diagnosis, you can pass condition indicators derived from an ensemble and their corresponding fault labels to an algorithm-fitting function that trains the classifier. For more information, please refer to Detect and Diagnose Faults in Matlab.

The traditional methods like support vector machine (SVM) and artificial neural networks (ANN) are heavily depend on the quality of the manually extracted features from the collected signals [1]. The deep learning methods can overcome the shortcomings of traditional intelligent methods by adaptively learn the useful features from the raw data without manually feature engineering. In this research, machine learning and deep learning methods can be considered to do fault diagnosis which may include CNN based auto-encoders, convolutional neural networks (CNNs), Variable weight CNNs, etc.. The datasets used in this project are MFPT dataset and Case Western dataset.

[1]. Shao H, Jiang H, Wang F, et al. An enhancement deep feature fusion method for rotating machinery fault diagnosis[J]. Knowledge-Based Systems, 2017, 119: 200-220.

[2]. Lei Y G, He Z J. Advances in applications of hybrid intelligent fault diagnosis and prognosis technique[J]. Zhendong yu Chongji(Journal of Vibration and Shock), 2011, 30(9): 129-135.

[HKL13]:Parameters Optimization of Convolutional Neural Networks

Convolutional Neural Networks (CNNs) have been widely applied in many classification tasks. CNNs have a lot of parameters (learning rate, input window size, filter size, number of layers, number of feature maps, etc.) which can provide different classification performance for the same tasks but based on diverse parameters. Now many researchers adjust these parameters manually when using CNNs, which is just based on previous experience and might take a lot time and cause large computation load in finding the optimal parameters.

This research aims to optimize the parameters in CNNs so that the parameters can be determined automatically during training rather than adjusting manually. In implementation, the papameters can be variable and determined by the outside neural networks which are fed by the input data. Or the parameters can be determined by the optimization algorithms. The results obtained from this research can be applied to biomedical areas which include cancer diagnosis, EEG/EMG/ECG signal's processing and image/video classification.

Available datasets: 1. MIT-BIH Arrhythmia Database; 2. EMG Datasets; 3. LIDC-IDRI.

Useful links:

1. An Intuitive Explanation of Convolutional Neural Networks

2. A Beginner's Guide To Understanding Convolutional Neural Networks

3. Variable weight neural networks and their applications on material surface and epilepsy seizure phase classifications

4. Neural Network Toolbox Examples

[HKL14]: Parkinson’s Disease Diagnosis with Deep Learning Techniques

Parkinson's disease is a long-term degenerative disorder of the central nervous system that mainly affects the motor system [1]. In the figures below, we can see that 'A–C, Images in a patient with Parkinson disease, a 52-year-old woman. Oblique coronal fast STIR images through the level of the posterior commissure (A) and superior colliculus (B) and axial T2-weighted image (C). D–F, Corresponding images in a control subject, a 55-year-old woman [2].'

The dataset used in this research is The Parkinson’s Progression Markers Initiative (PPMI). To classify the Magnetic Resonance Imaging (MRI) images of healthy control and Parkinson’s disease subjects, deep learning techniques including the convolutional neural networks (CNNs), variable weight CNNs, tranfer learning, model ensembling, etc. are concidered to be used, which aims to assist the clinicians to diagnose the Parkinson’s disease with higher accuracy in the near future.

[1]. Wikipedia contributors. "Parkinson's disease." Wikipedia, The Free Encyclopedia. Wikipedia, The Free Encyclopedia, 2 Nov. 2018. Web. 8 Nov. 2018.

[2]. Oikawa H, Sasaki M, Tamakawa Y, et al. The substantia nigra in Parkinson disease: proton density-weighted spin-echo and fast short inversion time inversion-recovery MR findings[J]. American Journal of Neuroradiology, 2002, 23(10): 1747-1756.

[HKL15]: Video Analysis Using Computer Vision and Deep Learning methods

This research aims to use the computer vision techniques and deep learning methods to process the live video, perform feature detection, extraction, matching, as well as object detection and tracking. The different masks, edge detection methods and multi-scale representations which are the important computer vision techniques can be used in feature detection.

Faster R-CNN [1] is an extension of the R-CNN [2] and Fast R-CNN [3] object detection techniques. All three of these techniques use convolutional neural networks (CNN). The difference between them is how they select regions to process and how those regions are classified. R-CNN and Fast R-CNN use a region proposal algorithm as a pre-processing step before running the CNN. The proposal algorithms are typically techniques such as EdgeBoxes [4] or Selective Search [5], which are independent of the CNN. In the case of Fast R-CNN, the use of these techniques becomes the processing bottleneck compared to running the CNN. Faster R-CNN addresses this issue by implementing the region proposal mechanism using the CNN and thereby making region proposal a part of the CNN training and prediction steps. In this project, the method faster R-CNN will be adopted to analyze the videos and you will learn how to train a type of detectors using deep learning methods.

The datasets can be used in this project are as follows.

1. YouTube Faces DB

2. HMDB: a large human motion database

3. Yahoo Flickr Creative Commons 100M (14G) (Hosted on AWS)

4. YouTube-BoundingBoxes Dataset

[1] Ren, S., K. He, R. Gershick, and J. Sun. "Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks." IEEE Transactions of Pattern Analysis and Machine Intelligence. Vol. 39, Issue 6, June 2017, pp. 1137-1149.

[2] Girshick, R., J. Donahue, T. Darrell, and J. Malik. "Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation." Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH, June 2014, pp. 580-587.

[3] Girshick, R. "Fast R-CNN." Proceedings of the 2015 IEEE International Conference on Computer Vision. Santiago, Chile, Dec. 2015, pp. 1440-1448.

[4] Zitnick, C. L., and P. Dollar. "Edge Boxes: Locating Object Proposals from Edges." European Conference on Computer Vision. Zurich, Switzerland, Sept. 2014, pp. 391-405.

[5] Uijlings, J. R. R., K. E. A. van de Sande, T. Gevers, and A. W. M. Smeulders. "Selective Search for Object Recognition." International Journal of Computer Vision. Vol. 104, Number 2, Sept. 2013, pp. 154-171.

[HKL16]: Stock Analysis and Prediction Using Deep Learning Techniques

The stock price data (shown in the above figure) is a type of nonlinear, uncertain and non-stationary time-series data. Traditional stock price prediction methods are always the econometric or statistical methods based on the analysis of past market movements [2]. Deep learning methods has the ability to extract abstract features from the raw data without feature engineering and without relying on econometric assumptions. In this research, we are going to use the deep learning methods and take stock return as the input data (or the representation of the data obtained through PCA method,CNN autoencoder,etc.), to predict the future market behavior. This research aims to offer practical insights and potentially useful directions for stock market analysis and prediction. Historical stock quotes datasets which can be used in this research include: 1. Yahoo Finance; 2. Nasdaq; 3. Wind.

[1]. https://www.macrotrends.net/1319/dow-jones-100-year-historical-chart

[2]. Chong E, Han C, Park F C. Deep learning networks for stock market analysis and prediction: Methodology, data representations, and case studies[J]. Expert Systems with Applications, 2017, 83: 187-205.

MSc Projects 2019-20

The following project descriptions and references should provide most details. The tools (say, programming languages, simulation platforms) for project realisation depend on what are available.  Please email me at This email address is being protected from spambots. You need JavaScript enabled to view it. if you would like to have a discussion.

Keywords: Classification, Computer Programming (Matlab, Python, TensorFlow), Computational Intelligence, Convotional Neural Network, Deep Learning,  Fuzzy Logic, Neural Network, Machine Learning, Population-Based Search Algorithms (Genetic Algorithm, Particle Swarm Optimisation, Q-learning, etc.), Reinforcement Learning, Support Vector Machine, Self-Organizing Map

Requirements: Hardworking, self-motivated, creative, good programming skills, willing to learn.

Remarks: 

• All projects are open-ended. They are basically at MSc level but can be taken into research level depending on problem formulation and the way approaches the problem.
• High Performence Computing (HPC) service can be used to speed up the learning process
  • To creat an HPC account, raise a ticket in Rosalind ticket system https://helpdesk.rosalind.kcl.ac.uk/ (accessible from university network only) and complete the form after choosing “Request new account” in help topics.
  • Further information about HPC can be found in https://user-wiki.rosalind.kcl.ac.uk/doku.php?id=hpc_quick_start_guide
    
• Students need to gain some knowledge which is new to them.
• Basically I categorise the project outcomes into the following three levels (basic, merit and distinction) according to 1) the challenging level of the problem, 2) scientific and academic contributions (insights, ideas, knowledge, novelty), 3) the breadth and depth (coverage, quality, level of sophistication) of design, methodology, and results
• Encourage to publish the contributions and achievements in a journal paper.

[HKL01] Unpaired Face Expression Translation using Generative Adversarial Networks

References

1. Ian Goodfellow. Nips 2016 tutorial: Generative adversarial networks. arXiv preprint arXiv:1701.00160, 2016.

2. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde- Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages 2672–2680, 2014.

3. Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. Image-to- image translation with conditional adversarial networks. 2017 IEEE Con- ference on Computer Vision and Pattern Recognition (CVPR), Jul 2017.

4. Karol Kurach, Mario Lucic, Xiaohua Zhai, Marcin Michalski, and Sylvain Gelly. The GAN landscape: Losses, architectures, regularization, and nor- malization, 2019.

5. Youssef A. Mejjati, Christian Richardt, James Tompkin, Darren Cosker, and Kwang In Kim. Unsupervised attention-guided image to image trans- lation, 2018.

6. Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets, 2014.

7. Guim Perarnau, Joost van de Weijer, Bogdan Raducanu, and Jose M. A ́lvarez. Invertible conditional gans for image editing, 2016.

8. Hao Tang, Wei Wang, Songsong Wu, Xinya Chen, Dan Xu, Nicu Sebe, and Yan Yan. Expression conditional gan for facial expression-to-expression translation. 2019 IEEE International Conference on Image Processing (ICIP), Sep 2019.

9. Gino Tesei. Generating realistic facial expressions through conditional cycle-consistent generative adversarial networks (ccyclegan). 2019.

10. Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. Un- paired image-to-image translation using cycle-consistent adversarial net- works. 2017 IEEE International Conference on Computer Vision (ICCV), Oct 2017.

Datasets

1. https://ai.google/tools/datasets/google-facial-expression/ 
2. http://www2.ece.ohio-state.edu/~aleix/ARdatabase.html

[HKL02] Music Composition using Generative Adversarial Networks

 References

[HKL03] Fuzzy Controller based on Reinforcement Learning for DC-to-DC Power Converters

Fuzzy control system is a control system based on fuzzy logic, which has advantages in dealing with systems with high non-linearity compared with linear controllers. Reinforcement learning algorithm is one of the machine learning algorithms which generates optimal policy by maximising the expectation of accumulated rewards in a long-term consideration even without knowing the dynamics of the environment. Several RL based fuzzy controllers have been proposed as to improve the controlling performance in challenging situations, such as fuzzy Q-learning system [1], fuzzy actor-critic learning (FACL) system [2]. A DC-to-DC power converter [3] is a device or circuit change direct current (DC) usually from low voltage to high voltage level. This project aims to utilize the state-of-the-art RL algorithms (DQN [4], DDPG [5], A3C [6] etc.) combining with fuzzy controller to achieve the regulation of DC power.

References

1. Fuzzy Q-learning system, https://ieeexplore.ieee.org/abstract/document/622790
2. Fuzzy Inference System Learning by Reinforcement Methods, https://ieeexplore.ieee.org/abstract/document/704563
3. DC-to-DC power converter, https://en.m.wikipedia.org/wiki/DC-to-DC_converter
4. Playing Atari with Deep Reinforcement Learning, https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf
5. Continuous control with deep reinforcement learning, https://arxiv.org/abs/1509.02971
6. Asynchronous Methods for Deep Reinforcement Learning, https://arxiv.org/pdf/1602.01783.pdf

Following are some extra papers and tutorials as reference:

1. Reinforcement Learning: An Introduction (2nd Edition), http://incompleteideas.net/book/bookdraft2017nov5.pdf
2. Adaptive Fuzzy PID Control Strategy for Spacecraft Attitude Control, https://link.springer.com/content/pdf/10.1007/s40815-018-0576-2.pdf
3. A New Adaptive Type-II Fuzzy-Based Deep Reinforcement Learning Control: Fuel Cell Air-Feed Sensors Control, https://ieeexplore.ieee.org/abstract/document/8744606
4. RL Course by David Silver, https://www.youtube.com/watch?v=2pWv7GOvuf0&list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ

[HKL04]  Auto Driving with Reinforcement Learning

Auto driving is a concept that has risen widely attention in recent years. An auto driving vehicle would be able to grasp the driving skills and learn the traffic rules using tons of data from different sensors. As to make vehicles make proper decisions during driving, the training process turns out to be crucial. Reinforcement learning as a branch of machine learning algorithms could figure out optimal policie by interacting with environment by maximising the expectation of accumulated reward in long term has been widely used and proved to be an effective technique.

This project aims to apply reinforcement learning algorithms such as Q-learning algorithm, DQN (Deep Q-Network), DDPG (Deep Deterministic Policy Gradients) etc., to train vehicles to learn drive autonomously.

Following are some papers and tutorials as reference:

• Learning to Drive in a Day: https://arxiv.org/pdf/1807.00412.pdf
• Deep Reinforcement Learning for Driving Policy:

• Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation: https://arxiv.org/abs/1709.10489
• MIT 6.S094: Deep Reinforcement Learning for Motion Planning: https://www.youtube.com/watch?v=QDzM8r3WgBw

Here are some applications on Youtube:

• Deep Learning Cars:

• Reinforcement Learning for Autonomous Driving Obstacle Avoidance using LIDAR

• Self Driving Digital Car using Python CNN Reinforcement learning

• Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation

[HKL05]  Auto Parking with Reinforcement Learning

This project aims to let car or robots learn to park autonomously using reinforcement learning algorithms either in a simulation environment or in on a real platform. Q-learning with epsilon greedy method or policy gradient methods can be considered as to apply control to the move the car or robot to the desired parking slot.

Here are some papers and tutorials recommended as references:

• Automatic Car Parking: A Reinforcement Learning Approach: https://link.springer.com/content/pdf/10.1007%2F3-540-44868-3_28.pdf
• Deep Reinforcement Learning framework for Autonomous Driving: https://arxiv.org/pdf/1704.02532.pdf
• Reinforcement Learning: An Introduction (2nd Edition): http://incompleteideas.net/book/bookdraft2017nov5.pdf

Here are some applications on Youtube:

• Automatic Parking with Reinforcement Learning:

• Automatic Parking with Reinforcement Learning in Matplotlib

• Learning to Park | A collaboration between NNAISENSE and Audi Electronics Venture

[HKL06]: Grasping Objects with Reinforcement Learning 

[HKL07] Machines Learn to Play

This project uses computational intelligence and machine learning techniques to create a machine which learns to play a computer game (of your design).

References: Playing Atari with Deep Reinforcement Learning; Q-learning tutorials: 1, 2, 3; Fuzzy Q-learning 

Some examples from youtube are shown below:

[HKL08] Generative Adversarial Networks with Medical Applications

Fig 1. The architecture of the networks for the unsupervised domain adaptation. The image is from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6241308/

In this research, we aim to apply Generative Adversarial Networks (GANs) to deal with medical image processing application which include but not limited to image denoising, reconstruction and segmentation and classification. GANs have shown great potential in medical image processing due to its advantages in 1). overcoming class imbalance and shortage of labelled medical image data; 2). powerful feature extraction capacity [1].

Different from the conventional GANs, in this research we focuses on making further efforts to improve the performance GANs through modifying their structures so as to make some layers have variable weights, which aims to help GANs adapt to different characteristics of input data and obtain better performance compared with conventional models. Also, by using GANs, we can overcome the shortcomings of conventional feature engineering and establish an automatic representation learning and dimensionality reduction algorithms of the inputs. TCGA dataset (https://www.nature.com/articles/nature12634) can be used in this project to test the performance of proposed method.

1. Kazeminia S, Baur C, Kuijper A, et al. GANs for medical image analysis[J]. arXiv preprint arXiv:1809.06222, 2018.

[HKL09] Classification of Electromyogram (EMG) Signals using Supervised and Unsupervised Machine Learning Methods

Many modern prostheses use electromyogram (EMG) signals generated by the human muscles to derive control commands for powered upper-limb prostheses. To accurately control the different finger postures of a prosthetic hand, the discriminating between individual and combined figures movements using surface EMG signals is of great significance [1]. The electromyogram (EMG) reflects the electrical activity of muscles and usually, there is proportional relevance between the level of the muscle movements and current intensity.

In this research, we mainly use the machine learning methods including supervised learning methods and unsupervised learning methods to classify the EMG signals. The EMG data can be found in the website: https://www.rami-khushaba.com/electromyogram-emg-repository.html . To provide effective classification, supervised learning and unsupervised learning can be combined to overcome the difficulty of limited data size and large number of classes. From this Project, you will be able to learn and implement the state-of-the-art machine learning methods such as convolutional neural networks (CNNs), fuzzy logic classifier, K-means clustering, K-nearest Neighbour, etc. and solve the medical classification problem which have the potential to be combined with upper-limb prostheses control.

Technical details: K-means can be used first to group the data into 3-5 classes, supervised learning like NNs, CNNs and can be used in each group to further classify the data. If multiple classifiers are used, Fuzzy combination rules can be established for classifier ensembles (http://www.scholarpedia.org/article/Fuzzy_classifiers).

1. R. N. Khushaba, M. Takruri, S. Kodagoda, and G. Dissanayake, Toward Improved Control of Prosthetic Fingers Using Surface Electromyogram (EMG) Signals, in: Expert Systems with Applications 2012, 39: 10731–10738.

[HKL10] Hand Gesture Recognition using Deep Learning Techniques

Gestures are the physical actions that convey some meaningful information. Gesture recognition systems as a new input modality in human computer interaction provide a intuitive and more convenient way of interaction. This project aims to recognise hand gestures as commands for controlling somethings like home appliances, robots, etc. For example, using a camera to recognise gestures for controlling a Lego-built robot; or recognising gestures for playing games with computer such as paper-sissorces-rock, etc.

An easy example of implementation including the dataset can be found here. In this dataset, the images are 50  50 gray scaled images. The images can be found in the images_data folder. CNN is used as the classification model.

Useful Links:

Links used to learn CNN:

Convolutional Neural Networks - The Math of Intelligence

CS231n Convolutional Neural Networks for Visual

An Intuitive Explanation of Convolutional Neural Networks

Conv Nets: A Modular Perspective

NRAM: Theano Implementation

Backpropogation:

CS231n Convolutional Neural Networks for Visual Recognition

Convolutional Neural Networks backpropagation: from intuition to derivation

Backpropagation in Convolutional Neural Network

Backpropagation In Convolutional Neural Networks

[HKL11] Position and Tracking Control of Wheeled Mobile Robots using Fuzzy Logic Controller

References: Matlab Fuzzy Logic Toolbox, A Short Fuzzy Logic Tutorial, Type-2 Fuzzy Stes and Systems, General Type-2 Fuzzy Logic Systems Made Simple: A Tutorial, Tutorial on Type-2 Fuzzy Sets and Systems, PID Controller, Fuzzy PID Controller, Linear State-feedback Controller

Open Source toobox: An Open Source Matlab/Simulink Toolbox for Interval Type-2 Fuzzy Logic Systems, toolbox downloaded here

Papers: Fuzzy model reference control of wheeled mobile robots; A practical fuzzy logic controller for the path tracking of wheeled mobile robots; Remote Control of a Mobile Robot for Indoor Patrol; Fuzzy Logic Tracking Control for Unicycle Mobile Robots

The aim of this project is to design a fuzzy controller for wheeled mobile robot for position and tracking control.  For position control, the wheeled mobile robot is driven to reach a target position.  For tracking control, the wheeled mobile robot is driven to move along a pre-defined trajectory.  This project can be broken down into the following tasks. 1) The dynamic model of the wheeled mobile robot is developed.  Based on the model, various types of controllers, such as fuzzy controller, PID controller, linear state-feedback controller, will be employed to perform the control. 2) The system performance will be investigated and optimised by genetic algorithm (GA) or particle swarm optimization (PSO). 3) A path planning algorithm will be developed to find the shortest path to reach the target with the ability of obstacle avoidance. 4) A Matlab software with Graphic User Interface (GUI) will be implemented to facilitate computer simulations and control synthesis with animated output. 5) The system will be implemented physically using a Lego Mindstorm NXT kit.  Hardware results will be recorded and compared.  Student taking this project is required to be hardworking and self-motivated. Good Matlab programming skill and mathematical background are essential to this project.

UG Projects 2019-20

The following project descriptions and references should provide most details. Please email me at This email address is being protected from spambots. You need JavaScript enabled to view it. if you would like to have a discussion.

Keywords: Classification, Computer Programming (Matlab, Python, TensorFlow), Computational Intelligence, Convotional Neural Network, Deep Learning,  Fuzzy Logic, Neural Network, Machine Learning, Population-Based Search Algorithms (Genetic Algorithm, Particle Swarm Optimisation, Q-learning, etc.), Reinforcement Learning, Support Vector Machine, Self-Organizing Map

Requirements: Hardworking, self-motivated, creative, good programming skills, willing to learn.

Remarks: 

• All projects are open-ended. They are basically at UG level but can be taken into research level depending on problem formulation and the way approach the problem.
• High Performence Computing (HPC) service can be used to spped up the learning process
  • To creat an HPC account, raise a ticket in Rosalind ticket system (https://helpdesk.rosalind.kcl.ac.uk/ )(accessible from university network only) and complete the form after choosing “Request new account” in help topics.
  • Further information about HPC can be found in https://user-wiki.rosalind.kcl.ac.uk/doku.php?id=hpc_quick_start_guide
    
• Students need to gain some knowledge which is new to them.
• Basically I categorise the project outcome into the following three levels (basic, merit and distinction) according to 1) the challenging level of the problem, 2) scientific and academic contributions (insights, ideas, knowledge, novelty), 3) the breadth and depth (coverage, quality, level of sophistication) of design, methodology, and results
• Encourage to publish the contributions and achievements in a journal paper.

[HKL01]  Face Creation and Classification using Generative Adversarial Networks and Variational Autoencoders

This project applies generative networks such as generative adversarial networks (GANs) and variational autoencoders (VAEs) to learn and create new samples of faces (or objectives). It follows by performing classification on generated facial samples. The face generator is required to be able to automatically extract the discriminative features from the images in the existing dataset, and create artifi- cial facial images.

Compared with GANs, VAEs are easier to train and perform better in im- age reconstruction. Some noises like reflection may mislead classifiers to false predictions, while this can be addressed through applying VAEs to remove these factors. Meanwhile, GANs are better at generating new samples with high vi- sual fidelity.

Some objectives (and should be expanded) are defined as follows:

• Explore effecient feature representation methods to represent the facial images in a smaller feature space.
• Explore data augmentation methods to generate more samples if existing datasets are insucient.
• Apply VAEs to remove the noises in the existing samples. This step is essential for the classification task later.
• Develop artificial facial image generators with high visual fidelity based on GANs.
• Evaluate and compare the performances between GANs of differerent structures.
• Perform classification tasks on facial characters, such as color of eye and hair, size of lip and nose, etc., according to the labels in dataset.
• Evaluate the performance of classifiers and relevant comparison is necessary.
• Improve the proposed methods in terms of augmentation, VAE-denoising, face creation using GANs and classification, by analysing the potential problems.

References:

• UCLA ACM AI. Tutorial for GAN, https://github.com/uclaacmai/Generative-Adversarial-Network-Tutorial/blob/master/Generative Adversarial Networks Tutorial.ipynb
• Carl Doersch. Tutorial on variational autoencoders. arXiv preprint arXiv:1606.05908, 2016. (https://arxiv.org/abs/1606.05908)
• Forcecore. Keras-gan-animeface-character. https://github.com/forcecore/Keras-GAN-Animeface-Character/
• Ian Goodfellow. Nips 2016 tutorial: Generative adversarial networks. arXiv preprint arXiv:1701.00160, 2016.
• Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Y. Bengio. Generative adversarial nets. ArXiv, 06 2014. (https://papers.nips.cc/paper/5423-generative-adversarial-nets.pdf)
• Y Jin, K Zhang, M Li, Y Tian, H Zhu, and Z Fang. Towards the automatic anime characters creation with generative adversarial networks. 2017. arXiv preprint arXiv:1708.05509. (https://arxiv.org/pdf/1708.05509.pdf)
• Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of GANs for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196, 2017.
• Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.

Possible public dataset (but not limited to):

• Large-scale CelebFaces At- tributes (CelebA) Dataset, http://mmlab.ie.cuhk.edu.hk/projects/CelebA.html

Here are some demonstrations on Youtube:

[HKL02] Classification of Seizure Types/Phases using Deep Learning Techniques

Different types of seizure usually have different causes and accurately diagnosing seizure types often helps identify the cause for the seizures. Hence, an accurate seizure diagnosis is important for clinicians to implement the most appropriate kind of treatment \cite{Hopkinsmedicine2019Types}. This project aims to develop effective classifiers to classify two classes of seizure, seizures with a motor component and seizures without a motor component, and seizure phases based on EMG and/or ECG signals provided by clinicians.

The objectives are defined as follows:

1. Explore feature extraction methods to extract significant characteristic from the dataset to support classification.
2. Employ data augmentation methods to create new training data from existing training data due to the small amounts of data in dataset.
3. Develop classifiers which make use of the extracted features as information to judge (or predict) whether the input signal is relevant to the seizure with a motor component or the seizure without a motor component.
4. Compare and evaluate the results with different techniques, say, deep structured classifiers, autoencoders, traditional classifiers (k-nearest neighbours algorithm, support vector machine, probability models, statistical techniques, neural networks, etc.), ranging from supervised to unsupervised learning techniques. Unsupervised learning techniques such as self-organising map, k-means clustering, fuzzy k-means clustering.
5. Improve and evaluate the recognition performance (robustness) of classifiers when the input is subject to noise, say, using ensemble methods.

References:

• Davies, Keith G., and Edward Ahn. "Epilepsy: surgery perspective." Principles and Practice of Stereotactic Radiosurgery. Springer, New York, NY, 2008. 583-591.
• Sewal, R.K., Modi, M., Saikia, U.N., Chakrabarti, A. and Medhi, B., 2017. Increase in seizure susceptibility in sepsis like condition explained by spiking cytokines and altered adhesion molecules level with impaired blood brain barrier integrity in experimental model of rats treated with lipopolysaccharides. Epilepsy research, 135, pp.176-186.
• Shen, C.P., Liu, S.T., Zhou, W.Z., Lin, F.S., Lam, A.Y.Y., Sung, H.Y., Chen, W., Lin, J.W., Chiu, M.J., Pan, M.K. and Kao, J.H., 2013. A physiology-based seizure detection system for multichannel EEG. PloS one, 8(6), p.e65862.
• https://www.hopkinsmedicine.org/neurology_neurosurgery/centers_clinics/epilepsy/conditions.html
• https://www.hopkinsmedicine.org/health/conditions-and-diseases/epilepsy/types-of-seizures
• https://www.hopkinsmedicine.org/health/conditions-and-diseases/epilepsy/diagnosing-seizures-and-epilepsy
• Lam H. K., Ekong U, Xiao B, et al. Variable weight neural networks and their applications on material surface and epilepsy seizure phase classifications. Neurocomputing, 2015, 149: 1177-1187.
• Ekong, U., Lam, H.K., Xiao, B., Ouyang, G., Liu, H., Chan, K.Y. and Ling, S.H., 2016. Classification of epilepsy seizure phase using interval type-2 fuzzy support vector machines. Neurocomputing, 199, pp.66-76.
• Lam, H.K., Ekong, U., Liu, H., Xiao, B., Araujo, H., Ling, S.H. and Chan, K.Y., 2014. A study of neural-network-based classifiers for material classification. Neurocomputing, 144, pp.367-377.
• Qazi, K.I., Lam, H.K., Xiao, B., Ouyang, G. and Yin, X., 2016. Classification of epilepsy using computational intelligence techniques. CAAI Transactions on Intelligence Technology, 1(2), pp.137-149.
• Lam, H.K. and Prada, J., 2009. Interpretation of handwritten single-stroke graffiti using support vector machines. International Journal of Computational Intelligence and Applications, 8(04), pp.369-393.

[HKL03] Hand Gesture Recognition using Deep Learning Techniques

Gestures are the physical actions that convey some meaningful information. Gesture recognition systems as a new input modality in human computer interaction provide a intuitive and more convenient way of interaction. This project aims to recognise hand gestures as commands for controlling somethings like home appliances, robots, etc. For example, using a camera to recognise gestures for controlling a Lego-built robot; or recognising gestures for playing games with computer such as paper-sissorces-rock, etc.

An easy example of implementation including the dataset can be found here. In this dataset, the images are 50 50 gray scaled images. The images can be found in the images_data folder. CNN is used as the classification model.

Useful Links:

Links used to learn CNN:

Convolutional Neural Networks - The Math of Intelligence

CS231n Convolutional Neural Networks for Visual

An Intuitive Explanation of Convolutional Neural Networks

Conv Nets: A Modular Perspective

NRAM: Theano Implementation

Backpropogation:

CS231n Convolutional Neural Networks for Visual Recognition

Convolutional Neural Networks backpropagation: from intuition to derivation

Backpropagation in Convolutional Neural Network

Backpropagation In Convolutional Neural Networks

[HKL04]  Auto Driving with Reinforcement Learning

Auto driving is a concept that has risen widely attention in recent years. An auto driving vehicle would be able to grasp the driving skills and learn the traffic rules using tons of data from different sensors. As to make vehicles make proper decisions during driving, the training process turns out to be crucial. Reinforcement learning as a branch of machine learning algorithms could figure out optimal policie by interacting with environment by maximising the expectation of accumulated reward in long term has been widely used and proved to be an effective technique.

This project aims to apply reinforcement learning algorithms such as Q-learning algorithm, DQN (Deep Q-Network), DDPG (Deep Deterministic Policy Gradients) etc., to train vehicles to learn drive autonomously.

Following are some papers and tutorials as reference:

• Learning to Drive in a Day: https://arxiv.org/pdf/1807.00412.pdf
• Deep Reinforcement Learning for Driving Policy:

• Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation: https://arxiv.org/abs/1709.10489
• MIT 6.S094: Deep Reinforcement Learning for Motion Planning: https://www.youtube.com/watch?v=QDzM8r3WgBw

Here are some applications on Youtube:

• Deep Learning Cars:

• Reinforcement Learning for Autonomous Driving Obstacle Avoidance using LIDAR

• Self Driving Digital Car using Python CNN Reinforcement learning

• Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation

[HKL05]  Auto Parking with Reinforcement Learning

This project aims to let car or robots learn to park autonomously using reinforcement learning algorithms either in a simulation environment or in on a real platform. Q-learning with epsilon greedy method or policy gradient methods can be considered as to apply control to the move the car or robot to the desired parking slot.

Here are some papers and tutorials recommended as references:

• Automatic Car Parking: A Reinforcement Learning Approach: https://link.springer.com/content/pdf/10.1007%2F3-540-44868-3_28.pdf
• Deep Reinforcement Learning framework for Autonomous Driving: https://arxiv.org/pdf/1704.02532.pdf
• Reinforcement Learning: An Introduction (2nd Edition): http://incompleteideas.net/book/bookdraft2017nov5.pdf

Here are some applications on Youtube:

• Automatic Parking with Reinforcement Learning:

• Automatic Parking with Reinforcement Learning in Matplotlib

• Learning to Park | A collaboration between NNAISENSE and Audi Electronics Venture

MSc Projects 2020-21

The following project descriptions and references should provide most details. The tools (such as programming languages, simulation platforms) for project realisation depend on what are available.

The following topics are dividied into 3 groups of project topics, namely, fuzzy control based, reinforcement learning based and deep-learning based projects. Note that my capacity of project supervision is 15 MSc projects that means I can take at most 15 MSc project students.

For enquiry, please attend the following Q&A Teams meeting sessions.

Keywords: Classification, Computer Programming (Matlab, Python, TensorFlow), Computational Intelligence, Convotional Neural Network, Deep Learning,  Fuzzy Logic, Neural Network, Machine Learning, Population-Based Search Algorithms (Genetic Algorithm, Particle Swarm Optimisation, Q-learning, etc.), Reinforcement Learning, Support Vector Machine, Self-Organizing Map

Requirements: Hardworking, self-motivated, creative, good programming skills, willing to learn.

Remarks: 

• All projects are open-ended. They are basically at MSc level but can be taken into research level depending on problem formulation and the way approaches the problem.
• High Performence Computing (HPC) service can be used to speed up the learning process
  • To creat an HPC account, raise a ticket in Rosalind ticket system https://helpdesk.rosalind.kcl.ac.uk/ (accessible from university network only) and complete the form after choosing “Request new account” in help topics.
  • Further information about HPC can be found in https://user-wiki.rosalind.kcl.ac.uk/doku.php?id=hpc_quick_start_guide
    
• Students need to gain NEW knowledge which is NEW to them.
• I categorise the project outcomes into the following three levels (basic, merit and distinction) according to 1) the challenging level of the problem, 2) scientific and academic contributions (insights, ideas, knowledge, novelty), 3) the breadth and depth (coverage, quality, level of sophistication) of design, methodology, and results
• Encourage to publish the contributions and achievements in a journal paper.
• All the projects below are simulation based.

Fuzzy Control Based Projects

[HKL01] Fuzzy Controller for DC-to-DC Power Converters

Fuzzy control system is a control system based on fuzzy logic, which has advantages in dealing with systems with high non-linearity compared with linear controllers. Reinforcement learning algorithm is one of the machine learning algorithms which generates optimal policy by maximising the expectation of accumulated rewards in a long-term consideration even without knowing the dynamics of the environment. Several RL based type-1/2 fuzzy controllers have been proposed as to improve the controlling performance in challenging situations, such as fuzzy Q-learning system [1], fuzzy actor-critic learning (FACL) system [2]. A DC-to-DC power converter [3] is a device or circuit change direct current (DC) usually from low voltage to high voltage level. This project aims to utilize the state-of-the-art RL algorithms (DQN [4], DDPG [5], A3C [6] etc.) combining with fuzzy controller to achieve the regulation of DC power.

References

1. Fuzzy Q-learning system, https://ieeexplore.ieee.org/abstract/document/622790
2. Fuzzy Inference System Learning by Reinforcement Methods, https://ieeexplore.ieee.org/abstract/document/704563
3. DC-to-DC power converter, https://en.m.wikipedia.org/wiki/DC-to-DC_converter
4. Playing Atari with Deep Reinforcement Learning, https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf
5. Continuous control with deep reinforcement learning, https://arxiv.org/abs/1509.02971
6. Asynchronous Methods for Deep Reinforcement Learning, https://arxiv.org/pdf/1602.01783.pdf

Following are some extra papers and tutorials as reference:

1. Reinforcement Learning: An Introduction (2nd Edition), http://incompleteideas.net/book/bookdraft2017nov5.pdf
2. Adaptive Fuzzy PID Control Strategy for Spacecraft Attitude Control, https://link.springer.com/content/pdf/10.1007/s40815-018-0576-2.pdf
3. A New Adaptive Type-II Fuzzy-Based Deep Reinforcement Learning Control: Fuel Cell Air-Feed Sensors Control, https://ieeexplore.ieee.org/abstract/document/8744606
4. RL Course by David Silver, https://www.youtube.com/watch?v=2pWv7GOvuf0&list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ

[HKL02] Position and Tracking Control of Wheeled Mobile Robots using Fuzzy Logic Controller

References: Matlab Fuzzy Logic Toolbox, A Short Fuzzy Logic Tutorial, Type-2 Fuzzy Stes and Systems, General Type-2 Fuzzy Logic Systems Made Simple: A Tutorial, Tutorial on Type-2 Fuzzy Sets and Systems, PID Controller, Fuzzy PID Controller, Linear State-feedback Controller

Open Source toobox: An Open Source Matlab/Simulink Toolbox for Interval Type-2 Fuzzy Logic Systems, toolbox downloaded here

Papers: Fuzzy model reference control of wheeled mobile robots; A practical fuzzy logic controller for the path tracking of wheeled mobile robots; Remote Control of a Mobile Robot for Indoor Patrol; Fuzzy Logic Tracking Control for Unicycle Mobile Robots

The aim of this project is to design a fuzzy controller for wheeled mobile robot for position and tracking control.  For position control, the wheeled mobile robot is driven to reach a target position.  For tracking control, the wheeled mobile robot is driven to move along a pre-defined trajectory.  This project can be broken down into the following tasks. 1) The dynamic model of the wheeled mobile robot is developed.  Based on the model, various types of controllers, such as fuzzy controller, PID controller, linear state-feedback controller, will be employed to perform the control. 2) The system performance will be investigated and optimised by genetic algorithm (GA) or particle swarm optimization (PSO). 3) A path planning algorithm will be developed to find the shortest path to reach the target with the ability of obstacle avoidance. 4) A Matlab software with Graphic User Interface (GUI) will be implemented to facilitate computer simulations and control synthesis with animated output. Student taking this project is required to be hardworking and self-motivated. Good Matlab programming skill and mathematical background are essential to this project.

[HKL03] Fuzzy Control of Vehicle Active Suspension Systems

References: Matlab LMI toolbox, Active Suspension, Automotive Suspension Systems, Automotive Suspension Systems, Modelling simulation and control of an active suspension systemodelling simulation and control of an active suspension system

The aim of the project is to design a fuzzy controller to control the vehicle  active suspension system, subject to the performance constraints and user preferences. This project can be broken down into the following tasks. 1) Fuzzy model will be constructed describing the system dynamics for analysis and design. 2) Stability analysis and control synthesis will be investigated based on the Lyapunov stability theory.  A set of linear matrix inequalities (LMIs) will be derived to determine the system stability and feedback gains of the fuzzy controller. 3) Computer simulations will be done to verify the analysis result and control performance will be compared with other control schemes. 4) A Graphic User Interface (GUI) will be implemented based on Matlab which involves an animated inverted pendulum. Student taking this project is required to be hardworking and self-motivated. Good Matlab programming skill and mathematical background are essential to this project.

[HKL04] Fuzzy-Model-Based Control of Ball-and-Plate System

    The aim of this project is to control the ball-and-plate system using fuzzy-model-based control method.  A fuzzy model will be developed to facilitate the design of the fuzzy controller.  Stability analysis will then be carried out.  By solving numerically the solution to the stability conditions, a fuzzy controller can be obtained. 

[HKL05] Fuzzy Control of Virtual Inverted Pendulum System

    Please refer to the description of [HKL04].  Instead of ball-and-plate system, an inverted pendulum system or a system of your choice is considered. 

Remarks: The above systems could be replaced by other nonlinear systems such as Magnetic Levitation system, ball inverted pendulum, drone, etc.

Reinforcement-Learning Based Projects

[HKL06] Vehicle Trajectory Control with Fuzzy Reinforcement Learning

Trajectory control is a basic component in the auto-driving technique which is deeply related to the ride comfort and safety concern of the auto-driving system. Although several classical (model-based) control theories have been implemented to solve this trajectory tracking task, the complexity of the car system and the uncertainty caused by the environment make it impossible for us to build the vehicle dynamics accurately.

Fuzzy reinforcement learning is a branch of reinforcement learning (RL) that can obtain (sub-) optimal control policy by directly interact with environment. By combining the fuzzy system with reinforcement learning, the fuzzy reinforcement learning is much more comprehensible and has the ability to handle the uncertainty caused by both the environment and the vehicle itself.

This project aims to apply fuzzy reinforcement learning algorithms such as fuzzy q-learning, fuzzy actor-critic learning, interval type-2 fuzzy q-learning, and interval type-2 actor-critic learning to achieve requested vehicle trajectory tracking under vehicle model uncertainty and environmental disturbance.

Following are some papers and books as reference:

1. A novel structure of actor-critic learning based on an interval type-2 TSK fuzzy neural network, https://ieeexplore.ieee.org/abstract/document/8884245
2. Reinforcement Learning: An Introduction (2nd Edition), http://incompleteideas.net/book/bookdraft2017nov5.pdf
3. Self-Organizing Interval Type-2 Fuzzy Q-learning for reinforcement fuzzy control, https://ieeexplore.ieee.org/abstract/document/6083971
4. A navigation method for mobile robots using interval type-2 fuzzy neural network fitting Q-learning in unknown environments, https://content.iospress.com/articles/journal-of-intelligent-and-fuzzy-systems/ifs182560
5. Fuzzy Inference System Learning by Reinforcement Methods, https://ieeexplore.ieee.org/abstract/document/704563

[HKL07] Reinforcement Learning Based Control for Drones

Drones have been widely used in various fields including aerial inspection and cargo transportation for its low cost and high convenience. However, due to its inherent instability in aerodynamic and highly nonlinearity in its dynamic model, the control issue becomes one of the most important issues that hinder drones from further expanding its application. Several classical control methods like PID control and h-infinity control have been proposed to control the drone, but the model uncertainty and the environmental disturbance prevent these methods from directly applying in practice.

Recently, reinforcement learning (RL) is one of the most promising model-free control methods for solving control issues. By directly interact with environment, the agent can learn the (sub-) optimal control policy without knowing any prior knowledge. The application of deep convolutional neural network also makes it possible for RL to achieve end to end learning, i.e., learning control policy directly from the data of the sensor.

This project aims to apply deep reinforcement learning algorithms such as DDQN, PPO, and DDPG to achieve path tracking/obstacle avoidance/attitude control/path planning task for single quadrotor drone. If the single drone control issue is solved, this project can be extended to the group control issue (e.g., formation control, cooperation control) for multiply drones. 

Following are some papers and books as reference:

1. Reinforcement Learning: An Introduction (2nd Edition), http://incompleteideas.net/book/bookdraft2017nov5.pdf
2. Deep Reinforcement Learning with Double Q-learning, http://arxiv.org/abs/1509.06461
3. Deterministic Policy Gradient Algorithms, http://proceedings.mlr.press/v32/silver14.pdf
4. Quadrotor modelling and control, https://ieeexplore.ieee.org/abstract/document/7754868
5. Multi-Agent Reinforcement Learning: A Selective Overview of Theories and Algorithms, http://arxiv.org/abs/1911.10635

[HKL08]  Auto Driving with Reinforcement Learning

Auto driving is a concept that has risen widely attention in recent years. An auto driving vehicle would be able to grasp the driving skills and learn the traffic rules using tons of data from different sensors. As to make vehicles make proper decisions during driving, the training process turns out to be crucial. Reinforcement learning as a branch of machine learning algorithms could figure out optimal policie by interacting with environment by maximising the expectation of accumulated reward in long term has been widely used and proved to be an effective technique.

This project aims to apply reinforcement learning algorithms such as Q-learning algorithm, DQN (Deep Q-Network), DDPG (Deep Deterministic Policy Gradients) etc., to train vehicles to learn drive autonomously.

Following are some papers and tutorials as reference:

• Learning to Drive in a Day: https://arxiv.org/pdf/1807.00412.pdf
• Deep Reinforcement Learning for Driving Policy:

• Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation: https://arxiv.org/abs/1709.10489
• MIT 6.S094: Deep Reinforcement Learning for Motion Planning: https://www.youtube.com/watch?v=QDzM8r3WgBw

Here are some applications on Youtube:

• Deep Learning Cars:

• Reinforcement Learning for Autonomous Driving Obstacle Avoidance using LIDAR

• Self Driving Digital Car using Python CNN Reinforcement learning

• Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation

 [HKL09]  Auto Parking with Reinforcement Learning

This project aims to let car or robots learn to park autonomously using reinforcement learning algorithms either in a simulation environment or in on a real platform. Q-learning with epsilon greedy method or policy gradient methods can be considered as to apply control to the move the car or robot to the desired parking slot.

Here are some papers and tutorials recommended as references:

• Automatic Car Parking: A Reinforcement Learning Approach: https://link.springer.com/content/pdf/10.1007%2F3-540-44868-3_28.pdf
• Deep Reinforcement Learning framework for Autonomous Driving: https://arxiv.org/pdf/1704.02532.pdf
• Reinforcement Learning: An Introduction (2nd Edition): http://incompleteideas.net/book/bookdraft2017nov5.pdf

Here are some applications on Youtube:

• Automatic Parking with Reinforcement Learning:

• Automatic Parking with Reinforcement Learning in Matplotlib

• Learning to Park | A collaboration between NNAISENSE and Audi Electronics Venture

[HKL10]: Grasping Objects with Reinforcement Learning 

References: 

1. James, Stephen, and Edward Johns. "3D simulation for robot arm control with deep Q-learning." arXiv preprint arXiv:1609.03759 (2016).

2. Breyer, Michel, et al. "Flexible Robotic Grasping with Sim-to-Real Transfer based Reinforcement Learning." arXiv preprint arXiv:1803.04996 (2018).

3. Hierarchical Reinforcement Learning of Multiple Grasping Policies

[HKL11]: Reinforcement Learning Algorithms for  UAV Control

The dynamic system of UAV has high nonlinearity and instability which makes generating control policy for this system a challenging issue. However, reinforcement learning algorithm as one of the machine learning algorithms can learn control policy by simply interacting with environment without knowing the dynamic model of the environment. Thus, this algorithm proposes a prospecting approach to controlling UAV. This project encourages to apply various reinforcement learning algorithms such as Q-learning algorithm, SARSA, actor-critic algorithm, etc, to control the movement of UAV either in simulation environment or on a real-world platform.

Here are some potential simulation environments.

• GymFC: This environment requires installation of Gazebo 8 on Ubuntu operating system.
• ROS Development Studio: There are several UAV models available and can be launched in browser with several tools such as OpenAI libraries, Gazebo Simulator ready to use.

There are also other different versions of simulation environments available online and you are encouraged to discover more.

Here are some papers and tutorials recommended:

• An Application of Reinforcement Learning to Aerobatic Helicopter Flight
• Autonomous Helicopter Aerobatics through Apprenticeship Learning
• Autonomous Quadrotor Landing using Deep Reinforcement Learning
• Control of a Quadrotor with Reinforcement Learning
• Reinforcement Learning: An Introduction (2nd Edition)

Some similar applications can be found as follows:

• Control of a Quadrotor with Reinforcement Learning
• UAV to Perform a Perched Landing
• Deep reinforcement learning for UAV in Gazebo simulation environment
• Autonomous Quadrotor Landing using Deep Reinforcement Learning
• Reinforcement Learning for Autonomous Navigation of UAV's

[HKL12]  Self-Balancing Robot using Reinforcement Learning Algorithms

Reinforcement learning algorithm is one of the machine learning algorithms which generates optimal policy by maximising the expectation of accumulated rewards in a long-term consideration even without knowing the dynamics of the environment. This project aims to apply reinforcement learning algorithms to make a robot learn to balance itself. A two-wheel robot or a unicycle can be considered as a platform. Q-learning algorithm, REINFROCE, actor-critic or any other algorithms as well as fuzzy control algorithms can be used to implement control.

Here are some papers and tutorials recommended:

• Using Reinforcement Learning to Achieve Two Wheeled Self Balancing Control: https://ieeexplore.ieee.org/document/7858451/
• Learning to Control Two-Wheeled Self-Balancing Robot Using Reinforcement Learning Rules and Fuzzy Neural Networks: https://ieeexplore.ieee.org/document/4667312/
• Reinforcement Learning: An Introduction (2nd Edition): http://incompleteideas.net/book/bookdraft2017nov5.pdf

Here are some applications:

• Reinforcement Learning on Real-World Two-Wheeled Robot:

• Machine Learning for Control: Balancing the Robotic Unicycle:

[HKL13] Machines Learn to Play

This project uses computational intelligence and machine learning techniques to create a machine which learns to play a computer game (of your design).

References: Playing Atari with Deep Reinforcement Learning; Q-learning tutorials: 1, 2, 3; Fuzzy Q-learning 

Some examples from youtube are shown below:

Deep-Learning Based Projects

Remarks: If you do not have any programming skills and deep-learning knowledge, do not consider the following projects epsecially the GAN based projects. 

[HKL14] Detection of COVID-19 Cases using Deep Learning Techniques with Chest X-Ray Images

In the fight against COVID-19, the immediate and accurate screening of infected patients is of great significance. As a widely used radiological screening approach, chest X-ray (CXR) images play an important role in the diagnosis of COVID-19 cases, especially in the situation that viral testing is in short supply. The development of deep learning techniques enables the end-to-end data classification without feature engineering. This research aims to design COVID-19 detection system based on deep learning techniques to aid specialists in screening of infected patients. Motivated by the fact that the viral test requires waiting time to obtain the results, this research aims to design a deep-learning based end-to-end CXR images classification system for acceleration of testing process.

Requirements: have coding experience in Python and shell scripting

Suitability: suitable for students who want to acquire image processing and classification related knowledge and experience using deep learning techniques

Available Datasets:

• https://github.com/ieee8023/covid-chestxray-dataset;
• https://github.com/agchung/Figure1-COVID-chestxray-dataset;
• https://www.kaggle.com/tawsifurrahman/covid19-radiography-database;
• https://data.mendeley.com/datasets/rscbjbr9sj/2

Expected outcomes: understand and implement image processing, image classification in terms of distinguishing COVID-19 positive from healthy cases and other lung diseases. Realize SOTA classification performance and desired to have novel contributions in (but not limited to) data processing, model design, reduction of computational complexity, etc. Creative achievements will aim at publishing in AI or biomedical journals. 

[HKL15] Behaviour Detection of Drones

 With the rapid development and popularization of drones, the airspace safety and malicious use of drones are of great concern.  Drone detection and tracking plays an important role in investigating emerging threats of using drones and supporting safe drone operations. In this research, drone detection will be investigated using machine learning techniques. The following datasets can be used for drone detection and classification (other related datasets can also be considered). YOLO model, Faster-RCNN or Mask-RCNN could be considered for this detection task.

Requirements: coding experience in Python and shell scripting

Suitability: suitable for students who have interests in drone detection and want to accumulate knowledge and experience in using machine learning techniques to solve practical problems.

Available datasets:

• Image Dataset for Bird Detection (https://bird.nae-lab.org/dataset/)
• VisDrone-Dataset (https://github.com/VisDrone/VisDrone-Dataset)
• Sound resource databases: https://www.soundsnap.com/tags, https://freesound.org/

Expected outcomes: The developed system should be able to detect drones in different environment and can distinguish drones from birds or other objects. For the detection task, the results are desired to have high mean Average Precision (mAP) and detection speed (FPS). For the classification task, accuracy, precision, recall and robustness should be evaluated and improved.

References:

1. Nassi B, Shabtai A, Masuoka R, et al. Sok-security and privacy in the age of drones: Threats, challenges, solution mechanisms, and scientific gaps. arXiv preprint arXiv:1903.05155, 2019.
2. Guvenc I, Koohifar F, Singh S, et al. Detection, tracking, and interdiction for amateur drones. IEEE Communications Magazine, 2018, 56(4): 75-81.
3. Redmon J, Divvala S, Girshick R, et al. You only look once: Unified, real-time object detection. Proceedings of the IEEE conference on computer vision and pattern recognition. 2016: 779-788.
4. Redmon J, Farhadi A. YOLO9000: better, faster, stronger. Proceedings of the IEEE conference on computer vision and pattern recognition. 2017: 7263-7271.
5. Ghiasi G, Lin T Y, Le Q V. Dropblock: A regularization method for convolutional networks. Advances in Neural Information Processing Systems. 2018: 10727-10737.
6. Ren S, He K, Girshick R, et al. Faster r-cnn: Towards real-time object detection with region proposal networks. Advances in neural information processing systems. 2015: 91-99.
7. He K, Gkioxari G, Dollár P, et al. Mask R-CNN. Proceedings of the IEEE international conference on computer vision. 2017: 2961-2969.

[HKL16] Classification of Electromyogram (EMG) Signals using Supervised and Unsupervised Machine Learning Methods

Many modern prostheses use electromyogram (EMG) signals generated by the human muscles to derive control commands for powered upper-limb prostheses. To accurately control the different finger postures of a prosthetic hand, the discriminating between individual and combined figures movements using surface EMG signals is of great significance [1]. The electromyogram (EMG) reflects the electrical activity of muscles and usually, there is proportional relevance between the level of the muscle movements and current intensity.

In this research, we mainly use the machine learning methods including supervised learning methods and unsupervised learning methods to classify the EMG signals. The EMG data can be found in the website: https://www.rami-khushaba.com/electromyogram-emg-repository.html . To provide effective classification, supervised learning and unsupervised learning can be combined to overcome the difficulty of limited data size and large number of classes. From this Project, you will be able to learn and implement the state-of-the-art machine learning methods such as convolutional neural networks (CNNs), fuzzy logic classifier, K-means clustering, K-nearest Neighbour, etc. and solve the medical classification problem which have the potential to be combined with upper-limb prostheses control.

Technical details: K-means can be used first to group the data into 3-5 classes, supervised learning like NNs, CNNs and can be used in each group to further classify the data. If multiple classifiers are used, Fuzzy combination rules can be established for classifier ensembles (http://www.scholarpedia.org/article/Fuzzy_classifiers).

1. R. N. Khushaba, M. Takruri, S. Kodagoda, and G. Dissanayake, Toward Improved Control of Prosthetic Fingers Using Surface Electromyogram (EMG) Signals, in: Expert Systems with Applications 2012, 39: 10731–10738.

[HKL17] Hand Gesture Recognition using Deep Learning Techniques

Gestures are the physical actions that convey some meaningful information. Gesture recognition systems as a new input modality in human computer interaction provide a intuitive and more convenient way of interaction. This project aims to recognise hand gestures as commands for controlling somethings like home appliances, robots, etc. For example, using a camera to recognise gestures for controlling a Lego-built robot; or recognising gestures for playing games with computer such as paper-sissorces-rock, etc.

An easy example of implementation including the dataset can be found here. In this dataset, the images are 50  50 gray scaled images. The images can be found in the images_data folder. CNN is used as the classification model.

Useful Links:

Links used to learn CNN:

Convolutional Neural Networks - The Math of Intelligence

CS231n Convolutional Neural Networks for Visual

An Intuitive Explanation of Convolutional Neural Networks

Conv Nets: A Modular Perspective

NRAM: Theano Implementation

Backpropogation:

CS231n Convolutional Neural Networks for Visual Recognition

Convolutional Neural Networks backpropagation: from intuition to derivation

Backpropagation in Convolutional Neural Network

Backpropagation In Convolutional Neural Networks

[HKL18]: Classification of Medical Images of Intraepithelial Papillary Capillary Loop using Deep Learning Techniques

'Nonerosive reflux disease (NERD) is defined as the presence of typical symptoms of gastroesophageal reflux disease caused by intraesophageal acid in the absence of visible oesophageal mucosal injury at endoscopy.' (Fass, Fennerty and Vakil, 2001)

Niwa et al (2009) have done much research on feature analysis and diagnosis of NERD. They summarised that the images from magnifying endoscopy show the Intraepithelial Papillary Capillary Loops (IPCLs) at the lower side of the oesophagus in NERD patients dilated and elongated with the regular arrangement.

In order to analyse the morphology of IPCL (whether the IPCLs dilate and elongate with the regular arrangement), the regions containing the clear shape of IPCL in the medical images should be extracted firstly. The lower image in Figure 1 shows the segmentation for the medical image in the upper one. It has been shown experimentally that using the segmented images to train a deep convolutional neural network to do the diagnosis of NERD can get a higher diagnosis accuracy compared with using the original images.

This project aims to develop an algorithm based on deep learning techniques to automatically segment the useful regions in the medical images of IPCL and perform classification. A set of the original images and segmented images will be provided for demonstrating which regions should be remained.

In order to achieve the project aim, the following objectives are defined which have to be polished when project starts:

1. Explore feature extraction methods to extract significant characteristic from datasets to support classification.
2. Develop algorithms to segment the images which is to remove unuseful information from the data.
3. Develop classifiers which make use of the segmented image as information to judge (or predict) if the input image is suffering from the disease or not.
4. Comparing and evaluate the results with different techniques, say, deep structured classifiers, auto encoders, traditional classifiers (KNN, support vector machines, probability models, statistical techniques, neural networks, etc), ranging from supervised to unsupervised learning techniques. Unsupervised learning techniques such as self-organising maps, k-means clustering, fuzzy k-means clustering.
5. Improve and evaluate the recognition performance (robustness) of classifiers when the input is subject to noise, say, using ensemble methods.

This project should be implemented with Python and TensorFlow.

Figure 1. Upper image: An original medical image (from King's hospital). Lower image: The extracted image.

References:

1. Albelwi, S., & Mahmood, A. (2017). A framework for designing the architectures of deep convolutional neural networks. Entropy, 19(6), 242.
2. Chollet, F. (2015). Keras: Deep learning library for theano and tensorflow. URL: https://keras.io/.
3. Fass, R., Fennerty, M. B., & Vakil, N. (2001). Nonerosive reflux disease—current concepts and dilemmas. The American journal of gastroenterology, 96(2), 303.
4. Hajjar, C., & Hamdan, H. (2012). Clustering of Interval Data Using Self-Organizing Maps–Application to Meteorological Data. In Applied Computational Intelligence in Engineering and Information Technology (pp. 135-146). Springer, Berlin, Heidelberg.
5. Krizhevsky, A., Sutskever, I., & Hinton, G. E. (2012). Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems (pp. 1097-1105).
6. Niwa, H. (2009). New challenges in gastrointestinal endoscopy. Springer Science & Business Media.

Useful Links:

• TensorFlow - An open source machine learning framework for everyone: https://www.tensorflow.org/
• Unsupervised Feature Learning and Deep Learning Tutorial: http://ufldl.stanford.edu/wiki/index.php/UFLDL_Tutorial
• CS231n Convolutional Neural Networks for Visual Recognition: https://cs231n.github.io/convolutional-networks/
• Self-organizing Maps - An Example Analysis and Script: http://www.pymvpa.org/examples/som.html
• Self-Organizing Maps with Google’s TensorFlow: https://codesachin.wordpress.com/2015/11/28/self-organizing-maps-with-googles-tensorflow/

[HKL19] Object Recognition Using Deep Learning Method

Object recognition is a computer vision technique to identify objects in images or videos. When humans look at a photograph or watch a video, we can readily spot people, objects, scenes, and visual details. The goal is to teach a computer to do what comes naturally to humans: to gain a level of understanding of what an image contains. Object recognition is a key technology behind driverless cars, enabling them to recognize a stop sign or to distinguish a pedestrian from a lamppost. It is also useful in a variety of applications such as disease identification in bioimaging, industrial inspection, and robotic vision.

Object Recognition vs. Object Detection

Object detection and object recognition are similar techniques for identifying objects, but they vary in their execution. Object detection is the process of finding instances of objects in images. In the case of deep learning, object detection is a subset of object recognition, where the object is not only identified but also located in an image. This allows for multiple objects to be identified and located within the same image.

Deep learning techniques for object recognition

Deep learning techniques have become a popular method for doing object recognition. Deep learning models such as convolutional neural networks, or CNNs, are used to automatically learn an object’s inherent features in order to identify that object. For example, a CNN can learn to identify differences between cats and dogs by analyzing thousands of training images and learning the features that make cats and dogs different.

There are two approaches to performing object recognition using deep learning:

• Training a model from scratch: To train a deep network from scratch, you gather a very large labeled dataset and design a network architecture that will learn the features and build the model. The results can be impressive, but this approach requires a large amount of training data, and you need to set up the layers and weights in the CNN.

• Using a pretrained deep learning model: Most deep learning applications use the transfer learning approach, a process that involves fine-tuning a pretrained model. You start with an existing network, such as AlexNet or GoogLeNet, and feed in new data containing previously unknown classes. This method is less time-consuming and can provide a faster outcome because the model has already been trained on thousands or millions of images.

Remark:Most of the content above comes from the Matlab website, please refer to Object Recognition for more details.

Useful Links:

1. Object Recognition in Matlab

2. An Intuitive Explanation of Convolutional Neural Networks

3. CS231n: Convolutional Neural Networks for Visual Recognition

4. A Beginner's Guide To Understanding Convolutional Neural Networks

[HKL20] Recommendation System using Deep Learning

Deep learning methods have achieved great success in computer vision, natural language processing, etc.. Due to its powerful learning capacity, end-to-end architecture and GPU acceleration, deep learning algorithms are widely applied to recommendation systems. From this project, you will be able to learn and implement deep learning algorithms in content based and collaborative filtering based recommendation systems. The algorithms will be used include convolutional neural networks (CNN), recurrent neural networks (RNN), etc. These techniques can be used in part of prediction algorithms such as word embedding for user review text and item review text, or combined as a whole for better predictive performance.

Requirements: familiar with general computation of partial derivative and have coding experience in Python and shell scripting

Suitability: Suitable for students who want to gain knowledge and experience in deep learning for industrial applications

Available Datasets: https://github.com/caserec/Datasets-for-Recommender-Systems

Expected outcomes: understand and implement existed deep learning architectures for recommendation system, such as Wide Deep Neural Network, Deep Cooperative Neural Network, Recurrent Recommender Network, etc. After having good understanding and programming skills in this domain, creative improvements of deep learning models in terms of model modification, new system design and theoretical innovations are encouraged. Novel contributions will aim at publishing in AI conference or journals.

References:

1. Son J, Kim S B. Content-based filtering for recommendation systems using multiattribute networks. Expert Systems with Applications, 2017, 89: 404-412.
2. Khanal S S, Prasad P W C, Alsadoon A, et al. A systematic review: machine learning based recommendation systems for e-learning. Education and Information Technologies, 2019: 1-30.
3. Chen R, Hua Q, Chang Y S, et al. A survey of collaborative filtering-based recommender systems: From traditional methods to hybrid methods based on social networks. IEEE Access, 2018, 6: 64301-64320.
4. Nagarnaik P, Thomas A. Survey on recommendation system methods. 2015 2nd International Conference on Electronics and Communication Systems (ICECS). IEEE, 2015: 1603-1608.
5. Cheng H T, Koc L, Harmsen J, et al. Wide & deep learning for recommender systems. Proceedings of the 1st workshop on deep learning for recommender systems. 2016: 7-10.
6. Zheng L, Noroozi V, Yu P S. Joint deep modeling of users and items using reviews for recommendation. Proceedings of the Tenth ACM International Conference on Web Search and Data Mining. 2017: 425-434.
7. Wu C Y, Ahmed A, Beutel A, et al. Recurrent recommender networks. Proceedings of the tenth ACM international conference on web search and data mining. 2017: 495-503.

[HKL21] Unpaired Face Expression Translation using Generative Adversarial Networks

References:

1. Ian Goodfellow. Nips 2016 tutorial: Generative adversarial networks. arXiv preprint arXiv:1701.00160, 2016.

2. Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde- Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In Advances in neural information processing systems, pages 2672–2680, 2014.

3. Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A. Efros. Image-to- image translation with conditional adversarial networks. 2017 IEEE Con- ference on Computer Vision and Pattern Recognition (CVPR), Jul 2017.

4. Karol Kurach, Mario Lucic, Xiaohua Zhai, Marcin Michalski, and Sylvain Gelly. The GAN landscape: Losses, architectures, regularization, and nor- malization, 2019.

5. Youssef A. Mejjati, Christian Richardt, James Tompkin, Darren Cosker, and Kwang In Kim. Unsupervised attention-guided image to image trans- lation, 2018.

6. Mehdi Mirza and Simon Osindero. Conditional generative adversarial nets, 2014.

7. Guim Perarnau, Joost van de Weijer, Bogdan Raducanu, and Jose M. A ́lvarez. Invertible conditional gans for image editing, 2016.

8. Hao Tang, Wei Wang, Songsong Wu, Xinya Chen, Dan Xu, Nicu Sebe, and Yan Yan. Expression conditional gan for facial expression-to-expression translation. 2019 IEEE International Conference on Image Processing (ICIP), Sep 2019.

9. Gino Tesei. Generating realistic facial expressions through conditional cycle-consistent generative adversarial networks (ccyclegan). 2019.

10. Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. Un- paired image-to-image translation using cycle-consistent adversarial net- works. 2017 IEEE International Conference on Computer Vision (ICCV), Oct 2017.

Datasets:

1. https://ai.google/tools/datasets/google-facial-expression/ 
2. http://www2.ece.ohio-state.edu/~aleix/ARdatabase.html

[HKL22] Music Composition using Generative Adversarial Networks

 References

[HKL23] Generative Adversarial Networks with Medical Applications

Fig 1. The architecture of the networks for the unsupervised domain adaptation. The image is from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6241308/

In this research, we aim to apply Generative Adversarial Networks (GANs) to deal with medical image processing application which include but not limited to image denoising, reconstruction and segmentation and classification. GANs have shown great potential in medical image processing due to its advantages in 1). overcoming class imbalance and shortage of labelled medical image data; 2). powerful feature extraction capacity [1].

Different from the conventional GANs, in this research we focuses on making further efforts to improve the performance GANs through modifying their structures so as to make some layers have variable weights, which aims to help GANs adapt to different characteristics of input data and obtain better performance compared with conventional models. Also, by using GANs, we can overcome the shortcomings of conventional feature engineering and establish an automatic representation learning and dimensionality reduction algorithms of the inputs. TCGA dataset (https://www.nature.com/articles/nature12634) can be used in this project to test the performance of proposed method.

1. Kazeminia S, Baur C, Kuijper A, et al. GANs for medical image analysis[J]. arXiv preprint arXiv:1809.06222, 2018.