I’m Hao-Lun (Howard) Hsu, a third-year CS Ph.D. student at Duke University advised by Prof. Miroslav Pajic. I also collaborate with Prof. Pan Xu and Prof. Vahid Tarokh. I was supported by Duke Computer Science Ph.D. departmental fellowship and NSF TAST Fellowship. Prior to Duke, I received M.S. in Biomedical Engineering from Georgia Tech and B.S. in Mechanical Engineering from National Taiwan University. During my master's studies, I was advised by Prof. Sehoon Ha to collaborate with Pacific Northwest National Laboratory on developing safe reinforcement learning (RL) algorithms for robotics and worked with Prof. Babak Mahmoudi on RL closed-loop control for neuromodulation.

I study efficient and trustworthy (e.g., safety, robustness, generalization) decision-making.

• Safe and robust RL: safety (unsupervised action planning, parallel constrained MDP) and robustness via adversarial learning ( adversarial herding, adaptive adversary, and robust exploration)
• Posterior sampling for decision-making: Real-world applications (deep brain stimulation, ultrasound guidance) and methodologies in multi-agent settings (cooperative multi-agent RL and multi-agent MAB)
• Generative AI: Decision Transformer with TD learning, In-context RL

News

Publications/Preprints

Please see my google scholar for an up-to-date list

*: equal contribution

2024

16. Randomized Exploration in Cooperative Multi-Agent Reinforcement Learning

Hao-Lun Hsu*, Wexin Wang*, Miroslav Pajic, Pan Xu
In: Proc. of the 38th Conference on Advances in Neural Information Processing Systems (NeurIPS), 2024

15. StressFADS: Learning Latent Autonomic Factors of Stress in the Context of Trauma Recall and Neuromodulation

Asim H Gazi, Michael Chan, Hao-Lun Hsu, Douglas Bremner, Christopher Rozell, Omer T Inan
In: IEEE International Conference on Wearable and Implantable Body Sensor Networks (BSN), 2024

14. Steering Decision Transformers via Temporal Difference Learning

Hao-Lun Hsu, Alper Kamil Bozkurt*, Juncheng Dong*, Qitong Gao, Vahid Tarokh, Miroslav Pajic
In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024

13. Reinforcement Learning for Closed-loop Regulation of Cardiovascular System with Vagus Nerve Stimulation: A Computational Study

Parisa Sarikhani, Hao-Lun Hsu, Mahmoud Zeydabadinezhad, Yuyu Yao, Mayuresh Kothare, Babak Mahmoudi
In: Journal of Neural Engineering, 2024

12. Robust Exploration with Adversary via Langevin Monte Carlo

Hao-Lun Hsu, Miroslav Pajic
In: Proceedings 6th Learning for Dynamics and Control Conference(L4DC), 2024

11. REFORMA: Robust REinFORceMent Learning via Adaptive Adversary for Drones Flying under Disturbances

Hao-Lun Hsu, Haocheng Meng, Shaocheng Luo, Juncheng Dong, Vahid Tarokh, Miroslav Pajic
In: Proceedings of IEEE International Conference on Robotics and Automation(ICRA), 2024

10. ε-Neural Thompson Sampling of Deep Brain Stimulation for Parkinson Disease Treatment

Hao-Lun Hsu, Qitong Gao, Miroslav Pajic
In: Proceedings of International Conference on Cyber-Physical Systems(ICCPS), 2024

9. Finite-Time Frequentist Regret Bounds of Multi-Agent Thompson Sampling on Sparse Hypergraphs

Tianyuan Jin, Hao-Lun Hsu, William Chang, Pan Xu
In: Proceedings of Annual AAAI Conference on Artificial Intelligence(AAAI) (Oral, acceptance rate 2.3%), 2024

2023

8. Neuroweaver: a translational platform for embedding artificial intelligence in closed-loop neuromodulation systems

Parisa Sarikhani, Hanyang Xu, Shu-Ting Wang, Sean Kinzer, Hao-Lun Hsu, Yusen Zhu, Josh Krasney, Joseph R. Manns, Hadi Esmaeilzadeh, Babak Mahmoudi
In: Neuroscience 2023, 52nd Annual Meeting, 2023

2022

7. Improving Safety in Deep Reinforcement Learning Using Unsupervised Action Planning

Hao-Lun Hsu, Qiuhua Huang, Sehoon Ha
In: Proceedings of IEEE International Conference on Robotics and Automation(ICRA) , 2022

6. Automated Tuning of Closed-loop Neuromodulation Control Systems using Bayesian Optimization

Parisa Sarikhani, Hao-Lun Hsu, Babak Mahmoudi
In: 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society(EMBC), 2022

2021

5. Neuroweaver: Towards a Platform for Designing Translatable Intelligent Closed-loop Neuromodulation Systems

Parisa Sarikhani, Hao-Lun Hsu, Joon Kyung Kim, Sean Kinzer, Edwin Mascarenhas, Hadi Esmaeilzadeh, Babak Mahmoudi
In: Neural Information Processing Systems (NeurIPS) Research2Clinics Workshop, 2021

4. Safe Exploration for Reinforcement Learning Using Unsupervised Action Planning

Hao-Lun Hsu, Qiuhua Huang, Sehoon Ha
In: Robotics: Science & Systems (RSS) Workshop on Integrating Planning and Learning, 2021

3. Sparc: Adaptive Closed-loop Control of Vagal Nerve Stimulation for Regulating Cardiovascular Function Using Deep Reinforcement Learning: A Computational Study

Parisa Sarikhani, Hao-Lun Hsu, Mahmoud Zeydabadinezhad, Yuyu Yao, Mayuresh Kothare, Babak Mahmoudi
In: Neuroscience 2021, 50th Annual Meeting, 2021

2. Neuroweaver: A Platform for Designing Intelligent Closed-loop Neuromodulation Systems

Parisa Sarikhani, Hao-Lun Hsu, Ozgur Kara, Joon Kyung Kim, Hadi Esmaeilzadeh, Babak Mahmoudi
In: 4th International Brain Stimulation Conference, 2021

1. Functional Connectivity Correlates to Individual Difference in Human Brains during Working Memory Task and Resting State

Hao-Lun Hsu
In: IEEE EMBS North American Virtual International Student Conference, 2021

Services

Please see my CV for an up-to-date list.


• Reviewer/Program Committee
  • Conferences: L4DC'24, ICRA’23-24, IROS’23, NeurIPS’23, ICLR’24, AISTATS’24, ICML’24
  • Workshops: ICML’23 Frontiers4LCD, NeurIPS’23 AI4Science, NeurIPS’23 GenBio, ICML'24 AI4Science, ICML'24 SPIGM
  • Research Proposal: PURA (President’s Undergraduate Research Award) Fall'22

Invited Talks

Please see my CV for an up-to-date list.


• 02/2024: Duke Capital Partner
  Title: Reinforcement Learning for Cyber-Physical Sytems
• 11/2022: NCTPASS 2022 Annual Symposium
  Title: AI for Dynamical and Safety-critical Systems
• 07/2022: Curai Health ML paper club
  Title: Possible Reinforcement Learning Approaches to History Taking
• 03/2021: Artificial Intelligence Medicine Organization weekly webinar
  Title: Applications of Reinforcement Learning in healthcare and power grid control
• 03/2021: Prof. Constantine Dovrolis’s research group
  Title: Individual Difference in Humans’ Brains from Functional Connectivity for Working Memory

Teaching

Guest Lecturer

• Spring 2024 CS 370 Introduction To AI, Duke University
  Instructor: Tananun Songdechakraiwut
  Topic: Reinforcement Learning

Graduate Teaching Assistant at Duke University

• CompSci 535 Algorithmic Game Theory (Instructor: Kamesh Munagala) Spring 2024
• Compsci 590 Data Science (Instructor: Jian Pei) Spring 2023

Graduate Teaching Assistant at Georgia Tech

• CS 7280 Network Science: Methods and Applications (Instructor: Constantine Dovrolis) Spring, Summer, Fall 2021
  • Receive Thank a Teacher Award from the Center of Teaching and Learning, Georgia Tech

Teaching Assistant at National Taiwan University

• EE 5040 Clinical Application of Medical Electronic Device (Instructor: Chih-Ting Lin) Fall 2017
• Biomed 7110 Clinical Observation & Demands Exploration (Instructor: Fa-Hsuan Lin) Summer 2017

RL Research Mentoring (10~15 weeks)

• Stefan Dragos, St. Augustine Preparatory School (robot navigation) Summer 2024
• Yang Chen, BS student at UC Berkeley (safe surgical robotics) Fall 2023
• Alexander Wang, West Windsor Plainsboro High School North (fake news detection) Fall 2023
• Nirav Jaiswal, Foothill High School (cloud computing) Summer 2023
• Indu Arimilli, Redmond High School (diagnosis prediction) Summer 2023
• Ian Choe, St. Mark’s School (deep brain stimulation) Summer 2023