MSc Projects 202223
The following project descriptions and references should provide most details. The tools (such as programming languages, simulation platforms) and datasets for project realisation depend on what are available.
For enquiry, please attend the following Q&A Teams meeting sessions.
Q&A Session  Teams Meeting 
Mon 7 November 2022, 9am  10am, click here to join 
Keywords:Classification, Computer Programming (Matlab, Python, TensorFlow), Computational Intelligence, Convotional Neural Network, Deep Learning, eXplainable Artificial Intelligence (XAI), Fuzzy Logic, Graph Neural Network, Generative Adversarial Network, Neural Network, Machine Learning, PopulationBased Search Algorithms (Genetic Algorithm, Particle Swarm Optimisation, Qlearning, etc.), Reinforcement Learning, Support Vector Machine, SelfOrganizing Map
Video Lectures: I have uploaded some lectures related to the above techniques to youtube: https://www.youtube.com/channel/UCNg2UL3AEYXT98NRD5TCmw
Requirements: Hardworking, selfmotivated, creative, good programming skills, willing to learn.
Remarks:
 All projects are openended. They are basically at MSc level but can be taken into research level depending on problem formulation and the way approaching the problem.
 All project topics are challenging and students need to gain NEW knowledge for the projects.
 High Performence Computing (HPC) service can be used to speed up the learning process
 To creat an HPC account, raise a ticket in Rosalind ticket system https://helpdesk.rosalind.kcl.ac.uk/ (accessible from university network only) and complete the form after choosing “Request new account” in help topics.
 Further information about HPC can be found in https://userwiki.rosalind.kcl.ac.uk/doku.php?id=hpc_quick_start_guide
 I categorise the project outcomes into the following three levels (basic, merit and distinction) according to 1) the challenging level of the problem, 2) scientific and academic contributions (insights, ideas, knowledge, novelty), 3) the breadth and depth (coverage, quality, level of sophistication) of design, methodology, and results. The level of breadth and depth would give a sense of the contribution made to the project.
 Encourage to publish the contributions and achievements in a journal paper.
 Technical papers can be downloaded from IEEE Explore (https://ieeexplore.ieee.org/Xplore/home.jsp) and Science Direct (https://www.sciencedirect.com/) using your King's login details (through institutional login).
[HKL01] Solving Rubik's Cubes using Machine Learning
The Rubik's cube is a 3D mechanical puzzle (see https://en.wikipedia.org/wiki/Rubik%27s_Cube for details) that has different dimensions ranging from 1x1x1 to theoretically infinite dimensions. The most common ones are 2x2x2 and 3x3x3 Rubik's cubes (see pictures below). This project will develop a machine learning algorithm, e.g., deep learning algorithm, reinforcement learning algorithm, genetic algorithm, to solve Rubik's cubes. A simulator will be developed to visualise Rubik's cubes and show how the machine learning algorithm is to solve them. It could be taken to another level by implementing a robot (e.g., using Lego Mindstorm) to solve the Rubik's cubes physically.
A 1x1x1 rubik's cube. 
A 2x2x2 rubik's cube. 
A 3x3x3 rubik's cube. 
References:
 Solving the Rubik's cube with stepwise deep learning, https://onlinelibrary.wiley.com/doi/10.1111/exsy.12665, code: https://onlinelibrary.wiley.com/action/downloadSupplement?doi=10.1111%2Fexsy.12665&file=exsy12665sup0001supinfo.py
 Solving the Rubik's Cube Without Human Knowledge, https://towardsdatascience.com/learningtosolvearubikscubefromscratchusingreinforcementlearning381c3bac5476" target="_blank" datamcehref="https://arxiv.org/abs/1805.07470%20(https://towardsdatascience.com/learningtosolvearubikscubefromscratchusingreinforcementlearning381c3bac5476">https://arxiv.org/abs/1805.07470 (https://towardsdatascience.com/learningtosolvearubikscubefromscratchusingreinforcementlearning381c3bac5476), Code and Dataset: https://github.com/CVxTz/rubiks_cube
 Solving a Rubik’s Cube with Reinforcement Learning, https://medium.com/analyticsvidhya/solvingarubikscubewithreinforcementlearningpart14f0405dd07f2 (Part 1), //medium.com/@mgd67/solvingarubikscubewithreinforcementlearningpart2b4ff0f3522b3" target="_blank" datamcehref="https://medium.com/@mgd67/solvingarubikscubewithreinforcementlearningpart2b4ff0f3522b3">https://medium.com/@mgd67/solvingarubikscubewithreinforcementlearningpart2b4ff0f3522b3>,
 Solve the Rubik's Cube Using Deep Learning: https://deepcube.igb.uci.edu
 CubeTR: Learning to Solve The Rubiks Cube Using Transformers: https://arxiv.org/abs/2111.06036
 Solving the Rubik's Cube with Approximate Policy Iteration, https://openreview.net/pdf?id=Hyfn2jCcKm, https://www.nature.com/articles/s422560190070z
 AI learns to solve a Rubik's Cube in 1.2 seconds: https://www.engadget.com/20190717airubikscubemachinelearningneuralnetwork.html?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer_sig=AQAAADSZOoQIrJJAO7C_4jnI0cvLBU5Wbwr8kPxll5vH8XOkKQc_CdU9arCS1tu_u6ebZdqGgeNorbqzx2159mOjDLFfXSUoAUvUvnSrDJSErkHW0sKjqi39Sgv9x4h4SfTtZBO2AdjkZwcVQ2_zwHvu6yWOReakckMdWu3xYy0uooP
 Solving Rubik’s Cube with a Robot Hand: https://openai.com/blog/solvingrubikscube/
 Reinforcement Learning to solve Rubik’s cube: https://medium.datadriveninvestor.com/reinforcementlearningtosolverubikscubeandothercomplexproblems106424cf26ff
 CS231n: Convolutional Neural Networks for Visual Recognition
 A Beginner's Guide To Understanding Convolutional Neural Networks
 Reinforcement Learning: An Introduction (2nd Edition), http://incompleteideas.net/book/bookdraft2017nov5.pdf
 Solving Rubik’s cube using genetic algorithms, https://robertovaccari.com/blog/2020_07_07_genetic_rubik/
 Every position of Rubik's Cube™ can be solved in twenty moves or less, https://www.cube20.org
Simulators:
 Cube Simulator: https://alg.cubing.net
Source Code:
 RubiksCubeAI: https://github.com/CodeBullet/RubiksCubeAI
 rubiks_cube: https://github.com/CVxTz/rubiks_cube
 rubikscubereinforcementlearning: https://github.com/AashrayAnand/rubikscubereinforcementlearning
 pyrubikscubeml: https://github.com/feldi/pyrubikscubeml
Videos:
Skills: Programming, machine learning
[HKL02] Control of Articulated Robots using ReinforcementLearning GraphNeuralNetworkBased Techniques
Reinforcement Learning (RL) algorithm is one of the machine learning algorithms which generates optimal policy by maximising the expectation of accumulated rewards in a longterm consideration even without knowing the dynamics of the environment. Graph Neural Network (GNN) based on graph theory (describing a an object as a graph, i.e., a connected graph consists of nodes and edges and ) will integrate information from neighbour nodes for mining hidden features among nodes. The aim of this project is to design a RLGNN based controller for articulated/spiderlike robots for realising body moving and accomplish some simple objects such as position control. Performance of the controller should be evaluated by both designing reward functions with respect to objects and training rates comparing with other ANN training methods.
The project can be implemented (but not only restrict to) through following optional languages and platforms:
 OpenAI Gym: Possess several existing models from training, relatively userfriendly.
 ROS2: An opensource robotics middleware suite, consists plenty of available plugins & tools. Relatively difficult to start but more powerful.
 MATLAB/Simulink: A proprietary multiparadigm programming language and numeric computing environment, with plenty of downloadable powerful modules.
 Python: An interpreted highlevel generalpurpose programming language.
 TensorFlow: A free and opensource software library for machine learning.
Demonstration:
References:
 Graph Representation Learning (William L Hamilton)
 Reinforcement Learning An Introduction
 NerveNet Learning Structured Policy with Graph Neural Networks
Resources:
 TensorFlow: https://www.tensorflow.org/
 OpenAI Gym:
 Introduction: https://gym.openai.com/envs/Antv2/
 Source Code: https://github.com/openai/gym/blob/master/gym/envs/mujoco/ant.py
 ROS2 (Recommended working on Ubuntu Linux):
 Spider Robot Source Code: https://github.com/erlerobot/ros2_spider_maestro_controller
 GNN: Plenty of source code can be found in GitHub, mostly based on Python.
Skills: Robotics, control, programming, machine learning
[HKL03] Fall/Behaviour Detection/Prediction/Classification using
Graph Neural Networks
The idea of the project is to create a graph neural networks and machinelearning models for the detection/prediction/classification of falls or behaviours.
References:
 Graph Representation Learning (William L Hamilton)
 MartínezVillaseñor, L., Ponce, H., Brieva, J., MoyaAlbor, E., NúñezMartínez, J. and PeñafortAsturiano, C., 2019. UPfall detection dataset: A multimodal approach. Sensors, 19(9), p.1988.
 Hemmatpour, M., Ferrero, R., Montrucchio, B. and Rebaudengo, M., 2019. A review on fall prediction and prevention system for personal devices: evaluation and experimental results. Advances in HumanComputer Interaction, 2019.
 Tsai, T.H. and Hsu, C.W., 2019. Implementation of fall detection system based on 3D skeleton for deep learning technique. IEEE Access, 7, pp.153049153059.
 Cai, X., Liu, X., An, M. and Han, G., 2021. VisionBased Fall Detection Using Dense Block With MultiChannel Convolutional Fusion Strategy. IEEE Access, 9, pp.1831818325.
 Keskes, O. and Noumeir, R., 2021. Visionbased fall detection using STGCN. IEEE Access, 9, pp.2822428236.
 Chami, I., Ying, Z., Ré, C. and Leskovec, J., 2019. Hyperbolic graph convolutional neural networks. Advances in neural information processing systems, 32, pp.48684879.
 FallFree: Multiple Fall Scenario Dataset of Cane Users for Monitoring Applications Using Kinect, https://ieeexplore.ieee.org/document/8334766
 Realtime MultiPerson Pose Estimation, https://github.com/ZheC/Realtime_MultiPerson_Pose_Estimation
 CMU Perceptual Computing Lab/openposePublic, https://github.com/CMUPerceptualComputingLab/openpose
Datasets
 Fall Datasets
 TST FALL DETECTION DATASET V2, https://ieeedataport.org/documents/tstfalldetectiondatasetv2#files

FALLUP DATASET, https://sites.google.com/up.edu.mx/harup/
 CMU Panoptic Dataset, http://domedb.perception.cs.cmu.edu/
 The FARSEEING realworld fall repository: a largescale collaborative database to collect and share sensor signals from realworld falls, http://farseeingresearch.eu/thefarseeingrealworldfallrepositoryalargescalecollaborativedatabasetocollectandsharesensorsignalsfromrealworldfalls/
 KFall: A Comprehensive Motion Dataset to Detect Preimpact Fall for the Elderly based on Wearable Inertial Sensors, https://sites.google.com/view/kfalldataset
 Fall DatasetRelated Papers
 Analysis of Public Datasets for Wearable Fall Detection Systems, https://www.mdpi.com/14248220/17/7/1513
 A LargeScale Open Motion Dataset (KFall) and Benchmark Algorithms for Detecting Preimpact Fall of the Elderly Using Wearable Inertial Sensors, https://www.frontiersin.org/articles/10.3389/fnagi.2021.692865/full
Skills: Programming, machine learning
[HKL04] Classification of Cancer Cells using Explainable Machine Learning Techniques
Machinelearningbased classifiers will be developed for detecting and classifier cancer image samples. Advanced deeplearning structured networks (e.g., convolutional neural network, longshorttermmemory network, transformer, graph neural network, generative adversarial network, deep fuzzy network) combining with the stateoftheart techniques (machine attention, data augmentation, fuzzy logic) will be employed to develop classifiers for detection and classification purpose. The eXplainable Artificial Intelligence (XAI) techniques (e.g., modelagnostic techniques: Local Interpretable Modelagnostic Explanations (LIME), SHapley Additive exPlanations (SHAP), LayerWise Relevance Propagation (LPR)) will be employed to explain the decision/actions made by the machine learning models. An explaination interface will be created to present the explainable report to the user.
Publically available datasets will be used to support this project.
Useful Links:

CS231n: Convolutional Neural Networks for Visual Recognition

A Beginner's Guide To Understanding Convolutional Neural Networks
 "Why Should I Trust You?": Explaining the Predictions of Any Classifier, https://arxiv.org/abs/1602.04938
 https://github.com/marcotcr/lime
 A Unified Approach to Interpreting Model Predictions, https://arxiv.org/abs/1705.07874
 https://github.com/slundberg/shap
 Interpretable Machine Learning A Guide for Making Black Box Models Explainable, https://christophm.github.io/interpretablemlbook/
 A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI, https://ieeexplore.ieee.org/document/9233366
 InDepth: LayerWise Relevance Propagation, https://towardsdatascience.com/indepthlayerwiserelevancepropagation340f95deb1ea
 LayerWise Relevance Propagation: An Overview, https://link.springer.com/chapter/10.1007/9783030289546_10 (The book can be downloaded through Institional Login (Access via your institution, choose King's College London))
 Metrics for Explainable AI: Challenges and Prospects, https://arxiv.org/abs/1812.04608
 Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics, https://www.mdpi.com/20799292/10/5/593/pdf
 Explaining Artificial Intelligence Demos, https://lrpserver.hhi.fraunhofer.de/
 Explaining Deep Learning Models for Structured Data using LayerWise Relevance Propagation, https://arxiv.org/abs/2011.13429
 LayerWise Relevance Propagation for Explaining Deep Neural Network Decisions in MRIBased Alzheimer's Disease Classification, https://www.frontiersin.org/articles/10.3389/fnagi.2019.00194/full
 Explaining Deep Learning Models Through RuleBased Approximation and Visualization, https://ieeexplore.ieee.org/document/9107404
Skills: Programming, machine learning
[HKL05] Self Driving with Reinforcement Learning
Self driving is a concept that has risen widely attention in recent years. An auto driving vehicle would be able to grasp the driving skills and learn the traffic rules using tons of data from different sensors. As to make vehicles make proper decisions during driving, the training process turns out to be crucial. Reinforcement learning as a branch of machine learning algorithms could figure out optimal policie by interacting with environment by maximising the expectation of accumulated reward in long term has been widely used and proved to be an effective technique.
This project aims to apply reinforcement learning algorithms such as Qlearning algorithm, DQN (Deep QNetwork), DDPG (Deep Deterministic Policy Gradients) etc., to train vehicles to learn drive autonomously.
The eXplainable Artificial Intelligence (XAI) techniques (e.g., modelagnostic techniques: Local Interpretable Modelagnostic Explanations (LIME), SHapley Additive exPlanations (SHAP), LayerWise Relevance Propagation (LPR)) will be employed to explain the decision/actions made by the machine learning models. An explaination interface will be created to present the explainable report to the user.
Following are some papers and tutorials as reference:
 Learning to Drive in a Day: https://arxiv.org/pdf/1807.00412.pdf
 Deep Reinforcement Learning for Driving Policy:
 Selfsupervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation: https://arxiv.org/abs/1709.10489
 MIT 6.S094: Deep Reinforcement Learning for Motion Planning: https://www.youtube.com/watch?v=QDzM8r3WgBw
Useful Links:

CS231n: Convolutional Neural Networks for Visual Recognition

A Beginner's Guide To Understanding Convolutional Neural Networks
 "Why Should I Trust You?": Explaining the Predictions of Any Classifier, https://arxiv.org/abs/1602.04938
 https://github.com/marcotcr/lime
 A Unified Approach to Interpreting Model Predictions, https://arxiv.org/abs/1705.07874
 https://github.com/slundberg/shap
 Interpretable Machine Learning A Guide for Making Black Box Models Explainable, https://christophm.github.io/interpretablemlbook/
 A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI, https://ieeexplore.ieee.org/document/9233366
 InDepth: LayerWise Relevance Propagation, https://towardsdatascience.com/indepthlayerwiserelevancepropagation340f95deb1ea
 LayerWise Relevance Propagation: An Overview, https://link.springer.com/chapter/10.1007/9783030289546_10 (The book can be downloaded through Institional Login (Access via your institution, choose King's College London))
 Metrics for Explainable AI: Challenges and Prospects, https://arxiv.org/abs/1812.04608
 Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics, https://www.mdpi.com/20799292/10/5/593/pdf
 Explaining Artificial Intelligence Demos, https://lrpserver.hhi.fraunhofer.de/
 Explaining Deep Learning Models for Structured Data using LayerWise Relevance Propagation, https://arxiv.org/abs/2011.13429
 LayerWise Relevance Propagation for Explaining Deep Neural Network Decisions in MRIBased Alzheimer's Disease Classification, https://www.frontiersin.org/articles/10.3389/fnagi.2019.00194/full
 Explaining Deep Learning Models Through RuleBased Approximation and Visualization, https://ieeexplore.ieee.org/document/9107404
Simulators:
 CARLO: https://github.com/StanfordILIAD/CARLO
 ARLA: http://carla.org
 Donkey Car Simulator: https://github.com/tawnkramer/sdsandbox
Here are some applications on Youtube:
 Deep Learning Cars:
 Reinforcement Learning for Autonomous Driving Obstacle Avoidance using LIDAR
 Self Driving Digital Car using Python CNN Reinforcement learning
 Selfsupervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation
Skills: Programming, machine learning, robotics