I’m Hao-Lun (Howard) Hsu, a third-year CS Ph.D. student at Duke University advised by Prof. Miroslav Pajic. I was supported by Duke Computer Science Ph.D. departmental fellowship and NSF TAST Fellowship. Prior to Duke, I received M.S. in Biomedical Engineering from Georgia Tech and B.S. in Mechanical Engineering from National Taiwan University. During my master's studies, I was advised by Prof. Sehoon Ha to collaborate with Pacific Northwest National Laboratory on developing safe RL algorithms for robotics and worked with Prof. Babak Mahmoudi on RL closed-loop control for neuromodulation.

My primary research concerns provably and practical decision-making, e.g., reinforcement learning (RL), multi-armed bandits (MAB). Specifically:

• Safe and robust RL: I study improving safety via unsupervised action planning and robustness via adversarial learning ( adversarial herding, adaptive adversary, and robust exploration).
• Exploration for decision-making: I mainly focus on exploration strategies under multi-agent settings (cooperative multi-agent RL and multi-agent MAB).
• Neuro-symbolic RL: I am interested in integrating neural networks with symbol representation for efficient RL algorithms.
• Generative AI: I investigate the integration of generative modeling with RL for robotics.

News

Publications/Preprints

Please see my google scholar for an up-to-date list

*: equal contribution

2024

16. Randomized Exploration in Cooperative Multi-Agent Reinforcement Learning

Hao-Lun Hsu*, Wexin Wang*, Miroslav Pajic, Pan Xu
In: Proc. of the 38th Conference on Advances in Neural Information Processing Systems (NeurIPS), 2024

15. StressFADS: Learning Latent Autonomic Factors of Stress in the Context of Trauma Recall and Neuromodulation

Asim H Gazi, Michael Chan, Hao-Lun Hsu, Douglas Bremner, Christopher Rozell, Omer T Inan
In: IEEE International Conference on Wearable and Implantable Body Sensor Networks (BSN), 2024

14. Steering Decision Transformers via Temporal Difference Learning

Hao-Lun Hsu, Alper Kamil Bozkurt*, Juncheng Dong*, Qitong Gao, Vahid Tarokh, Miroslav Pajic
In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024

13. Reinforcement Learning for Closed-loop Regulation of Cardiovascular System with Vagus Nerve Stimulation: A Computational Study

Parisa Sarikhani, Hao-Lun Hsu, Mahmoud Zeydabadinezhad, Yuyu Yao, Mayuresh Kothare, Babak Mahmoudi
In: Journal of Neural Engineering, 2024

12. Robust Exploration with Adversary via Langevin Monte Carlo

Hao-Lun Hsu, Miroslav Pajic
In: Proceedings 6th Learning for Dynamics and Control Conference(L4DC), 2024

11. REFORMA: Robust REinFORceMent Learning via Adaptive Adversary for Drones Flying under Disturbances

Hao-Lun Hsu, Haocheng Meng, Shaocheng Luo, Juncheng Dong, Vahid Tarokh, Miroslav Pajic
In: Proceedings of IEEE International Conference on Robotics and Automation(ICRA), 2024

10. ε-Neural Thompson Sampling of Deep Brain Stimulation for Parkinson Disease Treatment

Hao-Lun Hsu, Qitong Gao, Miroslav Pajic
In: Proceedings of International Conference on Cyber-Physical Systems(ICCPS), 2024

9. Finite-Time Frequentist Regret Bounds of Multi-Agent Thompson Sampling on Sparse Hypergraphs

Tianyuan Jin, Hao-Lun Hsu, William Chang, Pan Xu
In: Proceedings of Annual AAAI Conference on Artificial Intelligence(AAAI) (Oral, acceptance rate 2.3%), 2024

2023

8. Neuroweaver: a translational platform for embedding artificial intelligence in closed-loop neuromodulation systems

Parisa Sarikhani, Hanyang Xu, Shu-Ting Wang, Sean Kinzer, Hao-Lun Hsu, Yusen Zhu, Josh Krasney, Joseph R. Manns, Hadi Esmaeilzadeh, Babak Mahmoudi
In: Neuroscience 2023, 52nd Annual Meeting, 2023

2022

7. Improving Safety in Deep Reinforcement Learning Using Unsupervised Action Planning

Hao-Lun Hsu, Qiuhua Huang, Sehoon Ha
In: Proceedings of IEEE International Conference on Robotics and Automation(ICRA) , 2022

6. Automated Tuning of Closed-loop Neuromodulation Control Systems using Bayesian Optimization

Parisa Sarikhani, Hao-Lun Hsu, Babak Mahmoudi
In: 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society(EMBC), 2022

2021

5. Neuroweaver: Towards a Platform for Designing Translatable Intelligent Closed-loop Neuromodulation Systems

Parisa Sarikhani, Hao-Lun Hsu, Joon Kyung Kim, Sean Kinzer, Edwin Mascarenhas, Hadi Esmaeilzadeh, Babak Mahmoudi
In: Neural Information Processing Systems (NeurIPS) Research2Clinics Workshop, 2021

4. Safe Exploration for Reinforcement Learning Using Unsupervised Action Planning

Hao-Lun Hsu, Qiuhua Huang, Sehoon Ha
In: Robotics: Science & Systems (RSS) Workshop on Integrating Planning and Learning, 2021

3. Sparc: Adaptive Closed-loop Control of Vagal Nerve Stimulation for Regulating Cardiovascular Function Using Deep Reinforcement Learning: A Computational Study

Parisa Sarikhani, Hao-Lun Hsu, Mahmoud Zeydabadinezhad, Yuyu Yao, Mayuresh Kothare, Babak Mahmoudi
In: Neuroscience 2021, 50th Annual Meeting, 2021

2. Neuroweaver: A Platform for Designing Intelligent Closed-loop Neuromodulation Systems

Parisa Sarikhani, Hao-Lun Hsu, Ozgur Kara, Joon Kyung Kim, Hadi Esmaeilzadeh, Babak Mahmoudi
In: 4th International Brain Stimulation Conference, 2021

1. Functional Connectivity Correlates to Individual Difference in Human Brains during Working Memory Task and Resting State

Hao-Lun Hsu
In: IEEE EMBS North American Virtual International Student Conference, 2021

Services

Please see my CV for an up-to-date list.


• Reviewer/Program Committee
  • Conferences: L4DC'24, ICRA’23-24, IROS’23, NeurIPS’23, ICLR’24, AISTATS’24, ICML’24
  • Workshops: ICML’23 Frontiers4LCD, NeurIPS’23 AI4Science, NeurIPS’23 GenBio, ICML'24 AI4Science, ICML'24 SPIGM
  • Research Proposal: PURA (President’s Undergraduate Research Award) Fall'22

Invited Talks

Please see my CV for an up-to-date list.


• 02/2024: Duke Capital Partner
  Title: Reinforcement Learning for Cyber-Physical Sytems
• 11/2022: NCTPASS 2022 Annual Symposium
  Title: AI for Dynamical and Safety-critical Systems
• 07/2022: Curai Health ML paper club
  Title: Possible Reinforcement Learning Approaches to History Taking
• 03/2021: Artificial Intelligence Medicine Organization weekly webinar
  Title: Applications of Reinforcement Learning in healthcare and power grid control
• 03/2021: Prof. Constantine Dovrolis’s research group
  Title: Individual Difference in Humans’ Brains from Functional Connectivity for Working Memory

Teaching

Guest Lecturer

• Spring 2024 CS 370 Introduction To AI, Duke University
  Instructor: Tananun Songdechakraiwut
  Topic: Reinforcement Learning

Graduate Teaching Assistant at Duke University

• CompSci 535 Algorithmic Game Theory (Instructor: Kamesh Munagala) Spring 2024
• Compsci 590 Data Science (Instructor: Jian Pei) Spring 2023

Graduate Teaching Assistant at Georgia Tech

• CS 7280 Network Science: Methods and Applications (Instructor: Constantine Dovrolis) Spring, Summer, Fall 2021
  • Receive Thank a Teacher Award from the Center of Teaching and Learning, Georgia Tech

Teaching Assistant at National Taiwan University

• EE 5040 Clinical Application of Medical Electronic Device (Instructor: Chih-Ting Lin) Fall 2017
• Biomed 7110 Clinical Observation & Demands Exploration (Instructor: Fa-Hsuan Lin) Summer 2017

RL Research Mentoring (10~15 weeks)

• Stefan Dragos, St. Augustine Preparatory School (robot navigation) Summer 2024
• Yang Chen, BS student at UC Berkeley (safe surgical robotics) Fall 2023
• Alexander Wang, West Windsor Plainsboro High School North (fake news detection) Fall 2023
• Nirav Jaiswal, Foothill High School (cloud computing) Summer 2023
• Indu Arimilli, Redmond High School (diagnosis prediction) Summer 2023
• Ian Choe, St. Mark’s School (deep brain stimulation) Summer 2023