MSc Projects 2022-23

The following project descriptions and references should provide most details. The tools (such as programming languages, simulation platforms) and datasets for project realisation depend on what are available.

For enquiry, please attend the following Q&A Teams meeting sessions.

Keywords:Classification, Computer Programming (Matlab, Python, TensorFlow), Computational Intelligence, Convotional Neural Network, Deep Learning, eXplainable Artificial Intelligence (XAI), Fuzzy Logic, Graph Neural Network, Generative Adversarial Network, Neural Network, Machine Learning, Population-Based Search Algorithms (Genetic Algorithm, Particle Swarm Optimisation, Q-learning, etc.), Reinforcement Learning, Support Vector Machine, Self-Organizing Map

Video Lectures: I have uploaded some lectures related to the above techniques to youtube: https://www.youtube.com/channel/UCNg2UL-3AEYXT98NRD5TCmw

Requirements: Hardworking, self-motivated, creative, good programming skills, willing to learn.

Remarks: 

• All projects are open-ended. They are basically at MSc level but can be taken into research level depending on problem formulation and the way approaching the problem.
• All project topics are challenging and students need to gain NEW knowledge for the projects.
• High Performence Computing (HPC) service can be used to speed up the learning process
  • To creat an HPC account, raise a ticket in Rosalind ticket system https://helpdesk.rosalind.kcl.ac.uk/ (accessible from university network only) and complete the form after choosing “Request new account” in help topics.
  • Further information about HPC can be found in https://user-wiki.rosalind.kcl.ac.uk/doku.php?id=hpc_quick_start_guide
• I categorise the project outcomes into the following three levels (basic, merit and distinction) according to 1) the challenging level of the problem, 2) scientific and academic contributions (insights, ideas, knowledge, novelty), 3) the breadth and depth (coverage, quality, level of sophistication) of design, methodology, and results. The level of breadth and depth would give a sense of the contribution made to the project.
• Encourage to publish the contributions and achievements in a journal paper.
• Technical papers can be downloaded from IEEE Explore (https://ieeexplore.ieee.org/Xplore/home.jsp) and Science Direct (https://www.sciencedirect.com/) using your King's login details (through institutional login).

[HKL01] Solving Rubik's Cubes using Machine Learning

The Rubik's cube is a 3D mechanical puzzle (see https://en.wikipedia.org/wiki/Rubik%27s_Cube for details) that has different dimensions ranging from 1x1x1 to theoretically infinite dimensions. The most common ones are 2x2x2 and 3x3x3 Rubik's cubes (see pictures below). This project will develop a machine learning algorithm, e.g., deep learning algorithm, reinforcement learning algorithm, genetic algorithm, to solve Rubik's cubes. A simulator will be developed to visualise Rubik's cubes and show how the machine learning algorithm is to solve them. It could be taken to another level by implementing a robot (e.g., using Lego Mindstorm) to solve the Rubik's cubes physically.

A 1x1x1 rubik's cube.	A 2x2x2 rubik's cube.	A 3x3x3 rubik's cube.

References:

1. Solving the Rubik's cube with stepwise deep learning, https://onlinelibrary.wiley.com/doi/10.1111/exsy.12665, code: https://onlinelibrary.wiley.com/action/downloadSupplement?doi=10.1111%2Fexsy.12665&file=exsy12665-sup-0001-supinfo.py 
2. Solving the Rubik's Cube Without Human Knowledge, https://towardsdatascience.com/learning-to-solve-a-rubiks-cube-from-scratch-using-reinforcement-learning-381c3bac5476" target="_blank" data-mce-href="https://arxiv.org/abs/1805.07470%20(https://towardsdatascience.com/learning-to-solve-a-rubiks-cube-from-scratch-using-reinforcement-learning-381c3bac5476">https://arxiv.org/abs/1805.07470 (https://towardsdatascience.com/learning-to-solve-a-rubiks-cube-from-scratch-using-reinforcement-learning-381c3bac5476), Code and Dataset: https://github.com/CVxTz/rubiks_cube 
3. Solving a Rubik’s Cube with Reinforcement Learning, https://medium.com/analytics-vidhya/solving-a-rubiks-cube-with-reinforcement-learning-part-1-4f0405dd07f2 (Part 1), //medium.com/@mgd67/solving-a-rubiks-cube-with-reinforcement-learning-part-2-b4ff0f3522b3" target="_blank" data-mce-href="https://medium.com/@mgd67/solving-a-rubiks-cube-with-reinforcement-learning-part-2-b4ff0f3522b3">https://medium.com/@mgd67/solving-a-rubiks-cube-with-reinforcement-learning-part-2-b4ff0f3522b3>, 
   
4. Solve the Rubik's Cube Using Deep Learning: https://deepcube.igb.uci.edu
5. CubeTR: Learning to Solve The Rubiks Cube Using Transformers: https://arxiv.org/abs/2111.06036
6. Solving the Rubik's Cube with Approximate Policy Iteration, https://openreview.net/pdf?id=Hyfn2jCcKm, https://www.nature.com/articles/s42256-019-0070-z
7. AI learns to solve a Rubik's Cube in 1.2 seconds: https://www.engadget.com/2019-07-17-ai-rubiks-cube-machine-learning-neural-network.html?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer_sig=AQAAADSZOoQIr-JJAO7C_4jnI0cvLBU5Wbwr8kPxll5vH8XOkKQc_CdU9arCS1tu_u6ebZdqGgeNorbqzx2159mOjDLFfXSUoAUvUvnSrDJSErkHW0sKjqi39Sgv9x4h4SfTtZBO2AdjkZwcVQ2_zwHvu6yWOReakckMdWu3xYy0uooP
8. Solving Rubik’s Cube with a Robot Hand: https://openai.com/blog/solving-rubiks-cube/
9. Reinforcement Learning to solve Rubik’s cube: https://medium.datadriveninvestor.com/reinforcement-learning-to-solve-rubiks-cube-and-other-complex-problems-106424cf26ff
10. CS231n: Convolutional Neural Networks for Visual Recognition
11. A Beginner's Guide To Understanding Convolutional Neural Networks
12. Reinforcement Learning: An Introduction (2nd Edition), http://incompleteideas.net/book/bookdraft2017nov5.pdf
13. Solving Rubik’s cube using genetic algorithms, https://robertovaccari.com/blog/2020_07_07_genetic_rubik/
14. Every position of Rubik's Cube™ can be solved in twenty moves or less, https://www.cube20.org

Simulators:

• Cube Simulator: https://alg.cubing.net

Source Code:

• RubiksCubeAI: https://github.com/Code-Bullet/RubiksCubeAI
• rubiks_cube: https://github.com/CVxTz/rubiks_cube
• rubiks-cube-reinforcement-learning: https://github.com/AashrayAnand/rubiks-cube-reinforcement-learning
•  py-rubiks-cube-ml: https://github.com/feldi/py-rubiks-cube-ml

Videos:

Skills:  Programming, machine learning

[HKL02] Control of Articulated Robots using Reinforcement-Learning Graph-Neural-Network-Based Techniques 

Reinforcement Learning (RL) algorithm is one of the machine learning algorithms which generates optimal policy by maximising the expectation of accumulated rewards in a long-term consideration even without knowing the dynamics of the environment. Graph Neural Network (GNN) based on graph theory (describing a an object as a graph, i.e., a connected graph consists of nodes and edges and ) will integrate information from neighbour nodes for mining hidden features among nodes. The aim of this project is to design a RL-GNN based controller for articulated/spider-like robots for realising body moving and accomplish some simple objects such as position control. Performance of the controller should be evaluated by both designing reward functions with respect to objects and training rates comparing with other ANN training methods.

The project can be implemented (but not only restrict to) through following optional languages and platforms:

• OpenAI Gym: Possess several existing models from training, relatively user-friendly.
• ROS2: An open-source robotics middleware suite, consists plenty of available plugins & tools. Relatively difficult to start but more powerful.
• MATLAB/Simulink: A proprietary multi-paradigm programming language and numeric computing environment, with plenty of downloadable powerful modules.
• Python: An interpreted high-level general-purpose programming language.
• TensorFlow: A free and open-source software library for machine learning.

Demonstration:

References:

1. Graph Representation Learning (William L Hamilton)
2. Reinforcement Learning- An Introduction
3. NerveNet- Learning Structured Policy with Graph Neural Networks

Resources:

• TensorFlow: https://www.tensorflow.org/
• OpenAI Gym:
  • Introduction: https://gym.openai.com/envs/Ant-v2/
  • Source Code: https://github.com/openai/gym/blob/master/gym/envs/mujoco/ant.py
• ROS2 (Recommended working on Ubuntu Linux):
  • Download: https://docs.ros.org/en/foxy/Installation.html
  • Tutorials: https://docs.ros.org/en/foxy/Tutorials.html; https://www.theconstructsim.com/
• Spider Robot Source Code: https://github.com/erlerobot/ros2_spider_maestro_controller
• GNN: Plenty of source code can be found in GitHub, mostly based on Python.

Skills:  Robotics, control, programming, machine learning 

[HKL03] Fall/Behaviour Detection/Prediction/Classification using 

Graph Neural Networks

The idea of the project is to create a graph neural networks and machine-learning models for the detection/prediction/classification of falls or behaviours.

References:

1. Graph Representation Learning (William L Hamilton)
2. Martínez-Villaseñor, L., Ponce, H., Brieva, J., Moya-Albor, E., Núñez-Martínez, J. and Peñafort-Asturiano, C., 2019. UP-fall detection dataset: A multimodal approach. Sensors, 19(9), p.1988.
3. Hemmatpour, M., Ferrero, R., Montrucchio, B. and Rebaudengo, M., 2019. A review on fall prediction and prevention system for personal devices: evaluation and experimental results. Advances in Human-Computer Interaction, 2019.
4. Tsai, T.H. and Hsu, C.W., 2019. Implementation of fall detection system based on 3D skeleton for deep learning technique. IEEE Access, 7, pp.153049-153059.
5. Cai, X., Liu, X., An, M. and Han, G., 2021. Vision-Based Fall Detection Using Dense Block With Multi-Channel Convolutional Fusion Strategy. IEEE Access, 9, pp.18318-18325.
6. Keskes, O. and Noumeir, R., 2021. Vision-based fall detection using ST-GCN. IEEE Access, 9, pp.28224-28236.
7. Chami, I., Ying, Z., Ré, C. and Leskovec, J., 2019. Hyperbolic graph convolutional neural networks. Advances in neural information processing systems, 32, pp.4868-4879.
8. FallFree: Multiple Fall Scenario Dataset of Cane Users for Monitoring Applications Using Kinect, https://ieeexplore.ieee.org/document/8334766
9. Realtime Multi-Person Pose Estimation, https://github.com/ZheC/Realtime_Multi-Person_Pose_Estimation
10. CMU Perceptual Computing Lab/openposePublic, https://github.com/CMU-Perceptual-Computing-Lab/openpose

Datasets

• Fall Datasets
  • TST FALL DETECTION DATASET V2, https://ieee-dataport.org/documents/tst-fall-detection-dataset-v2#files
  • FALL-UP DATASET, https://sites.google.com/up.edu.mx/har-up/
  • CMU Panoptic Dataset, http://domedb.perception.cs.cmu.edu/
  • The FARSEEING real-world fall repository: a large-scale collaborative database to collect and share sensor signals from real-world falls, http://farseeingresearch.eu/the-farseeing-real-world-fall-repository-a-large-scale-collaborative-database-to-collect-and-share-sensor-signals-from-real-world-falls/
  • KFall: A Comprehensive Motion Dataset to Detect Pre-impact Fall for the Elderly based on Wearable Inertial Sensors, https://sites.google.com/view/kfalldataset
• Fall Dataset-Related Papers
  • Analysis of Public Datasets for Wearable Fall Detection Systems, https://www.mdpi.com/1424-8220/17/7/1513
  • A Large-Scale Open Motion Dataset (KFall) and Benchmark Algorithms for Detecting Pre-impact Fall of the Elderly Using Wearable Inertial Sensors, https://www.frontiersin.org/articles/10.3389/fnagi.2021.692865/full

Skills:  Programming, machine learning  

[HKL04] Classification of Cancer Cells using Explainable Machine Learning Techniques

Machine-learning-based classifiers will be developed for detecting and classifier cancer image samples. Advanced deep-learning structured networks (e.g., convolutional neural network, long-short-term-memory network, transformer, graph neural network, generative adversarial network, deep fuzzy network) combining with the state-of-the-art techniques (machine attention, data augmentation, fuzzy logic) will be employed to develop classifiers for detection and classification purpose. The eXplainable Artificial Intelligence (XAI) techniques (e.g., model-agnostic techniques: Local Interpretable Model-agnostic Explanations (LIME), SHapley Additive exPlanations (SHAP), Layer-Wise Relevance Propagation (LPR)) will be employed to explain the decision/actions made by the machine learning models. An explaination interface will be created to present the explainable report to the user.

Publically available datasets will be used to support this project.

Useful Links:

1. Object Recognition in Matlab

2. An Intuitive Explanation of Convolutional Neural Networks

3. CS231n: Convolutional Neural Networks for Visual Recognition

4. A Beginner's Guide To Understanding Convolutional Neural Networks

5. "Why Should I Trust You?": Explaining the Predictions of Any Classifier, https://arxiv.org/abs/1602.04938
6. https://github.com/marcotcr/lime
7. A Unified Approach to Interpreting Model Predictions, https://arxiv.org/abs/1705.07874
8. https://github.com/slundberg/shap
10. Interpretable Machine Learning A Guide for Making Black Box Models Explainable, https://christophm.github.io/interpretable-ml-book/
11. A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI, https://ieeexplore.ieee.org/document/9233366
12. InDepth: Layer-Wise Relevance Propagation, https://towardsdatascience.com/indepth-layer-wise-relevance-propagation-340f95deb1ea
13. Layer-Wise Relevance Propagation: An Overview, https://link.springer.com/chapter/10.1007/978-3-030-28954-6_10 (The book can be downloaded through Institional Login (Access via your institution, choose King's College London))
14. Metrics for Explainable AI: Challenges and Prospects, https://arxiv.org/abs/1812.04608
15. Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics, https://www.mdpi.com/2079-9292/10/5/593/pdf
16. Explaining Artificial Intelligence Demos, https://lrpserver.hhi.fraunhofer.de/
17. Explaining Deep Learning Models for Structured Data using Layer-Wise Relevance Propagation, https://arxiv.org/abs/2011.13429
18. Layer-Wise Relevance Propagation for Explaining Deep Neural Network Decisions in MRI-Based Alzheimer's Disease Classification, https://www.frontiersin.org/articles/10.3389/fnagi.2019.00194/full
19. Explaining Deep Learning Models Through Rule-Based Approximation and Visualization, https://ieeexplore.ieee.org/document/9107404

Skills:  Programming, machine learning  

[HKL05] Self Driving with Reinforcement Learning

Self driving is a concept that has risen widely attention in recent years. An auto driving vehicle would be able to grasp the driving skills and learn the traffic rules using tons of data from different sensors. As to make vehicles make proper decisions during driving, the training process turns out to be crucial. Reinforcement learning as a branch of machine learning algorithms could figure out optimal policie by interacting with environment by maximising the expectation of accumulated reward in long term has been widely used and proved to be an effective technique.

This project aims to apply reinforcement learning algorithms such as Q-learning algorithm, DQN (Deep Q-Network), DDPG (Deep Deterministic Policy Gradients) etc., to train vehicles to learn drive autonomously.

The eXplainable Artificial Intelligence (XAI) techniques (e.g., model-agnostic techniques: Local Interpretable Model-agnostic Explanations (LIME), SHapley Additive exPlanations (SHAP), Layer-Wise Relevance Propagation (LPR)) will be employed to explain the decision/actions made by the machine learning models. An explaination interface will be created to present the explainable report to the user.

Following are some papers and tutorials as reference:

• Learning to Drive in a Day: https://arxiv.org/pdf/1807.00412.pdf
• Deep Reinforcement Learning for Driving Policy:

• Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation: https://arxiv.org/abs/1709.10489
• MIT 6.S094: Deep Reinforcement Learning for Motion Planning: https://www.youtube.com/watch?v=QDzM8r3WgBw

Useful Links:

1. Object Recognition in Matlab

2. An Intuitive Explanation of Convolutional Neural Networks

3. CS231n: Convolutional Neural Networks for Visual Recognition

4. A Beginner's Guide To Understanding Convolutional Neural Networks

5. "Why Should I Trust You?": Explaining the Predictions of Any Classifier, https://arxiv.org/abs/1602.04938
6. https://github.com/marcotcr/lime
7. A Unified Approach to Interpreting Model Predictions, https://arxiv.org/abs/1705.07874
8. https://github.com/slundberg/shap
9. Interpretable Machine Learning A Guide for Making Black Box Models Explainable, https://christophm.github.io/interpretable-ml-book/
10. A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI, https://ieeexplore.ieee.org/document/9233366
11. InDepth: Layer-Wise Relevance Propagation, https://towardsdatascience.com/indepth-layer-wise-relevance-propagation-340f95deb1ea
12. Layer-Wise Relevance Propagation: An Overview, https://link.springer.com/chapter/10.1007/978-3-030-28954-6_10 (The book can be downloaded through Institional Login (Access via your institution, choose King's College London))
13. Metrics for Explainable AI: Challenges and Prospects, https://arxiv.org/abs/1812.04608
14. Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics, https://www.mdpi.com/2079-9292/10/5/593/pdf
15. Explaining Artificial Intelligence Demos, https://lrpserver.hhi.fraunhofer.de/
16. Explaining Deep Learning Models for Structured Data using Layer-Wise Relevance Propagation, https://arxiv.org/abs/2011.13429
17. Layer-Wise Relevance Propagation for Explaining Deep Neural Network Decisions in MRI-Based Alzheimer's Disease Classification, https://www.frontiersin.org/articles/10.3389/fnagi.2019.00194/full
18. Explaining Deep Learning Models Through Rule-Based Approximation and Visualization, https://ieeexplore.ieee.org/document/9107404

Simulators:

• CARLO: https://github.com/Stanford-ILIAD/CARLO
• ARLA: http://carla.org
• Donkey Car Simulator: https://github.com/tawnkramer/sdsandbox 

Here are some applications on Youtube:

• Deep Learning Cars:

• Reinforcement Learning for Autonomous Driving Obstacle Avoidance using LIDAR

• Self Driving Digital Car using Python CNN Reinforcement learning

• Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation

Skills:  Programming, machine learning, robotics