UG Projects 2022-23

The following project descriptions and references should provide most details. The tools (such as programming languages, simulation platforms) and datasets for project realisation depend on what are available.

For enquiry, please attend the following Q&A Teams meeting sessions.

Q&A Session - Teams Meeting Link:
9am - 10am, Wednesday, 28 September 2022, Teams Link

Keywords:Classification, Computer Programming (Matlab, Python, TensorFlow), Computational Intelligence, Convotional Neural Network, Deep LearningeXplainable Artificial Intelligence (XAI)Fuzzy Logic, Graph Neural NetworkGenerative Adversarial NetworkNeural Network, Machine Learning, Population-Based Search Algorithms (Genetic Algorithm, Particle Swarm Optimisation, Q-learning, etc.), Reinforcement LearningSupport Vector Machine, Self-Organizing Map

Video Lectures: I have uploaded some lectures related to the above techniques to youtube: https://www.youtube.com/channel/UCNg2UL-3AEYXT98NRD5TCmw

Requirements: Hardworking, self-motivated, creative, good programming skills, willing to learn.

Remarks: 

  • All projects are open-ended. They are basically at UG level but can be taken into research level depending on problem formulation and the way approaching the problem.
  •  All project topics are challenging and students need to gain NEW knowledge for the projects.
  • High Performence Computing (HPC) service can be used to speed up the learning process
  • I categorise the project outcomes into the following three levels (basic, merit and distinction) according to 1) the challenging level of the problem, 2) scientific and academic contributions (insights, ideas, knowledge, novelty), 3) the breadth and depth (coverage, quality, level of sophistication) of design, methodology, and results. The level of breadth and depth would give a sense of the contribution made to the project.
  • Encourage to publish the contributions and achievements in a journal paper.
  • Technical papers can be downloaded from IEEE Explore (https://ieeexplore.ieee.org/Xplore/home.jsp) and Science Direct (https://www.sciencedirect.com/) using your King's login details (through institutional login).

 

[HKL01] Fuzzy Control of DC-to-DC Power Converters

Fuzzy control system is a control system based on fuzzy logic, which has advantages in dealing with systems with high non-linearity compared with linear controllers. Reinforcement learning algorithm is one of the machine learning algorithms which generates optimal policy by maximising the expectation of accumulated rewards in a long-term consideration even without knowing the dynamics of the environment. Several RL based type-1/2 fuzzy controllers have been proposed to improve the controlling performance in challenging situations, such as fuzzy Q-learning system [1], fuzzy actor-critic learning (FACL) system [2]. A DC-to-DC power converter [3] is a device or circuit that changes direct current (DC) usually from low voltage to high voltage level. This project aims to utilize the state-of-the-art RL algorithms (DQN [4], DDPG [5], A3C [6] etc.) combining with fuzzy controller to achieve the regulation of DC power.

The design of the fuzzy control policy will be achieved in simulation environment, i.e., the coverter's models, fuzzy controller and learning algorithms will be implemented by Matlab/Python. The design will then be physically realised using electronic components implementing the DC-DC converters and microcontrollers (e.g., arduino, Raspberry Pi).   

References:

  1. Fuzzy Q-learning system, https://ieeexplore.ieee.org/abstract/document/622790
  2. Fuzzy Inference System Learning by Reinforcement Methods, https://ieeexplore.ieee.org/abstract/document/704563
  3. DC-to-DC power converter, https://en.m.wikipedia.org/wiki/DC-to-DC_converter
  4. Playing Atari with Deep Reinforcement Learning, https://www.cs.toronto.edu/~vmnih/docs/dqn.pdf
  5. Continuous control with deep reinforcement learning, https://arxiv.org/abs/1509.02971
  6. Asynchronous Methods for Deep Reinforcement Learning, https://arxiv.org/pdf/1602.01783.pdf

Following are some extra papers and tutorials as reference:

  1. Reinforcement Learning: An Introduction (2nd Edition), http://incompleteideas.net/book/bookdraft2017nov5.pdf
  2. Adaptive Fuzzy PID Control Strategy for Spacecraft Attitude Control, https://link.springer.com/content/pdf/10.1007/s40815-018-0576-2.pdf
  3. A New Adaptive Type-II Fuzzy-Based Deep Reinforcement Learning Control: Fuel Cell Air-Feed Sensors Control, https://ieeexplore.ieee.org/abstract/document/8744606
  4. RL Course by David Silver, https://www.youtube.com/watch?v=2pWv7GOvuf0&list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ

Skills:  Power electronics, control, electronic, programming, machine learning 

[HKL02] Control of Articulated Robots using Reinforcement-Learning Graph-Neural-Network-Based Techniques 

Reinforcement Learning (RL) algorithm is one of the machine learning algorithms which generates optimal policy by maximising the expectation of accumulated rewards in a long-term consideration even without knowing the dynamics of the environment. Graph Neural Network (GNN) based on graph theory (describing a an object as a graph, i.e., a connected graph consists of nodes and edges and ) will integrate information from neighbour nodes for mining hidden features among nodes. The aim of this project is to design a RL-GNN based controller for articulated/spider-like robots for realising body moving and accomplish some simple objects such as position control. Performance of the controller should be evaluated by both designing reward functions with respect to objects and training rates comparing with other ANN training methods.

The project can be implemented (but not only restrict to) through following optional languages and platforms:

  • OpenAI Gym: Possess several existing models from training, relatively user-friendly.
  • ROS2: An open-source robotics middleware suite, consists plenty of available plugins & tools. Relatively difficult to start but more powerful.
  • MATLAB/Simulink: A proprietary multi-paradigm programming language and numeric computing environment, with plenty of downloadable powerful modules.
  • Python: An interpreted high-level general-purpose programming language.
  • TensorFlow: A free and open-source software library for machine learning.

Demonstration:

References:

  1. Graph Representation Learning (William L Hamilton)
  2. Reinforcement Learning- An Introduction
  3. NerveNet- Learning Structured Policy with Graph Neural Networks

Resources:

Skills:  Robotics, control, programming, machine learning 

[HKL03] Fall/Behaviour Detection/Prediction/Classification using 

Graph Neural Networks

The idea of the project is to create a graph neural networks and machine-learning models for the detection/prediction/classification of falls or behaviours.

References:

  1. Graph Representation Learning (William L Hamilton)
  2. Martínez-Villaseñor, L., Ponce, H., Brieva, J., Moya-Albor, E., Núñez-Martínez, J. and Peñafort-Asturiano, C., 2019. UP-fall detection dataset: A multimodal approach. Sensors, 19(9), p.1988.
  3. Hemmatpour, M., Ferrero, R., Montrucchio, B. and Rebaudengo, M., 2019. A review on fall prediction and prevention system for personal devices: evaluation and experimental results. Advances in Human-Computer Interaction, 2019.
  4. Tsai, T.H. and Hsu, C.W., 2019. Implementation of fall detection system based on 3D skeleton for deep learning technique. IEEE Access, 7, pp.153049-153059.
  5. Cai, X., Liu, X., An, M. and Han, G., 2021. Vision-Based Fall Detection Using Dense Block With Multi-Channel Convolutional Fusion Strategy. IEEE Access, 9, pp.18318-18325.
  6. Keskes, O. and Noumeir, R., 2021. Vision-based fall detection using ST-GCN. IEEE Access, 9, pp.28224-28236.
  7. Chami, I., Ying, Z., Ré, C. and Leskovec, J., 2019. Hyperbolic graph convolutional neural networks. Advances in neural information processing systems, 32, pp.4868-4879.
  8. FallFree: Multiple Fall Scenario Dataset of Cane Users for Monitoring Applications Using Kinect, https://ieeexplore.ieee.org/document/8334766
  9. Realtime Multi-Person Pose Estimation, https://github.com/ZheC/Realtime_Multi-Person_Pose_Estimation
  10. CMU Perceptual Computing Lab/openposePublic, https://github.com/CMU-Perceptual-Computing-Lab/openpose

Datasets

Skills:  Programming, machine learning  

[HKL04] Classification of Cancer Cells using Explainable Machine Learning Techniques

Machine-learning-based classifiers will be developed for detecting and classifier cancer image samples. Advanced deep-learning structured networks (e.g., convolutional neural network, long-short-term-memory network, transformer, graph neural network, generative adversarial network, deep fuzzy network) combining with the state-of-the-art techniques (machine attention, data augmentation, fuzzy logic) will be employed to develop classifiers for detection and classification purpose. The eXplainable Artificial Intelligence (XAI) techniques (e.g., model-agnostic techniques: Local Interpretable Model-agnostic Explanations (LIME), SHapley Additive exPlanations (SHAP), Layer-Wise Relevance Propagation (LPR)) will be employed to explain the decision/actions made by the machine learning models. An explaination interface will be created to present the explainable report to the user.

Publically available datasets will be used to support this project.

Useful Links:

  1. Object Recognition in Matlab

  2. An Intuitive Explanation of Convolutional Neural Networks

  3. CS231n: Convolutional Neural Networks for Visual Recognition

  4. A Beginner's Guide To Understanding Convolutional Neural Networks

  5. "Why Should I Trust You?": Explaining the Predictions of Any Classifier, https://arxiv.org/abs/1602.04938
  6. https://github.com/marcotcr/lime
  7. A Unified Approach to Interpreting Model Predictions, https://arxiv.org/abs/1705.07874
  8. https://github.com/slundberg/shap
  9. Interpretable Machine Learning A Guide for Making Black Box Models Explainable, https://christophm.github.io/interpretable-ml-book/
  10. A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI, https://ieeexplore.ieee.org/document/9233366
  11. InDepth: Layer-Wise Relevance Propagation, https://towardsdatascience.com/indepth-layer-wise-relevance-propagation-340f95deb1ea
  12. Layer-Wise Relevance Propagation: An Overview, https://link.springer.com/chapter/10.1007/978-3-030-28954-6_10 (The book can be downloaded through Institional Login (Access via your institution, choose King's College London))
  13. Metrics for Explainable AI: Challenges and Prospects, https://arxiv.org/abs/1812.04608
  14. Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics, https://www.mdpi.com/2079-9292/10/5/593/pdf
  15. Explaining Artificial Intelligence Demos, https://lrpserver.hhi.fraunhofer.de/
  16. Explaining Deep Learning Models for Structured Data using Layer-Wise Relevance Propagation, https://arxiv.org/abs/2011.13429
  17. Layer-Wise Relevance Propagation for Explaining Deep Neural Network Decisions in MRI-Based Alzheimer's Disease Classification, https://www.frontiersin.org/articles/10.3389/fnagi.2019.00194/full
  18. Explaining Deep Learning Models Through Rule-Based Approximation and Visualization, https://ieeexplore.ieee.org/document/9107404

Skills:  Programming, machine learning  

[HKL05] Self Driving with Reinforcement Learning

Self driving is a concept that has risen widely attention in recent years. An auto driving vehicle would be able to grasp the driving skills and learn the traffic rules using tons of data from different sensors. As to make vehicles make proper decisions during driving, the training process turns out to be crucial. Reinforcement learning as a branch of machine learning algorithms could figure out optimal policie by interacting with environment by maximising the expectation of accumulated reward in long term has been widely used and proved to be an effective technique.

This project aims to apply reinforcement learning algorithms such as Q-learning algorithm, DQN (Deep Q-Network), DDPG (Deep Deterministic Policy Gradients) etc., to train vehicles to learn drive autonomously.

The eXplainable Artificial Intelligence (XAI) techniques (e.g., model-agnostic techniques: Local Interpretable Model-agnostic Explanations (LIME), SHapley Additive exPlanations (SHAP), Layer-Wise Relevance Propagation (LPR)) will be employed to explain the decision/actions made by the machine learning models. An explaination interface will be created to present the explainable report to the user.

Following are some papers and tutorials as reference:

Useful Links:

  1. Object Recognition in Matlab

  2. An Intuitive Explanation of Convolutional Neural Networks

  3. CS231n: Convolutional Neural Networks for Visual Recognition

  4. A Beginner's Guide To Understanding Convolutional Neural Networks

  5. "Why Should I Trust You?": Explaining the Predictions of Any Classifier, https://arxiv.org/abs/1602.04938
  6. https://github.com/marcotcr/lime
  7. A Unified Approach to Interpreting Model Predictions, https://arxiv.org/abs/1705.07874
  8. https://github.com/slundberg/shap
  9. Interpretable Machine Learning A Guide for Making Black Box Models Explainable, https://christophm.github.io/interpretable-ml-book/
  10. A Survey on Explainable Artificial Intelligence (XAI): Toward Medical XAI, https://ieeexplore.ieee.org/document/9233366
  11. InDepth: Layer-Wise Relevance Propagation, https://towardsdatascience.com/indepth-layer-wise-relevance-propagation-340f95deb1ea
  12. Layer-Wise Relevance Propagation: An Overview, https://link.springer.com/chapter/10.1007/978-3-030-28954-6_10 (The book can be downloaded through Institional Login (Access via your institution, choose King's College London))
  13. Metrics for Explainable AI: Challenges and Prospects, https://arxiv.org/abs/1812.04608
  14. Evaluating the Quality of Machine Learning Explanations: A Survey on Methods and Metrics, https://www.mdpi.com/2079-9292/10/5/593/pdf
  15. Explaining Artificial Intelligence Demos, https://lrpserver.hhi.fraunhofer.de/
  16. Explaining Deep Learning Models for Structured Data using Layer-Wise Relevance Propagation, https://arxiv.org/abs/2011.13429
  17. Layer-Wise Relevance Propagation for Explaining Deep Neural Network Decisions in MRI-Based Alzheimer's Disease Classification, https://www.frontiersin.org/articles/10.3389/fnagi.2019.00194/full
  18. Explaining Deep Learning Models Through Rule-Based Approximation and Visualization, https://ieeexplore.ieee.org/document/9107404

Here are some applications on Youtube:

  • Deep Learning Cars:

  • Reinforcement Learning for Autonomous Driving Obstacle Avoidance using LIDAR

  • Self Driving Digital Car using Python CNN Reinforcement learning

  • Self-supervised Deep Reinforcement Learning with Generalized Computation Graphs for Robot Navigation

Skills:  Programming, machine learning, robotics

 [HKL06]  Self Parking with Reinforcement Learning

This project aims to let car or robots learn to park autonomously using reinforcement learning algorithms either in a simulation environment or in on a real platform. Q-learning with epsilon greedy method or policy gradient methods can be considered as to apply control to the move the car or robot to the desired parking slot.

Here are some papers and tutorials recommended as references:

Here are some applications on Youtube:

  • Automatic Parking with Reinforcement Learning:

  • Automatic Parking with Reinforcement Learning in Matplotlib

  • Learning to Park | A collaboration between NNAISENSE and Audi Electronics Venture

 

Skills:  Programming, machine learning, robotics