I'm a Ph.D. candidate in Computer Science at Duke University with concurrent master's degree in Economics, advised by prof. Miroslav Pajic. I also collaborate with Prof. Pan Xu and Prof. Vahid Tarokh. I develop autonomous agents through the lens of game-theoretic multi-agent aspects, RL, and LLMs. I’ve been fortunate to be supported by the Duke Computer Science Ph.D. departmental fellowship and NSF TAST Fellowship.

Prior to Duke, I earned an M.S. in Biomedical Engineering from Georgia Tech, where I worked with Prof. Sehoon Ha and Pacific Northwest National Laboratory on safe RL for robotics. I also conducted research with Prof. Babak Mahmoudi on RL-driven neuromodulation control. I completed my B.S. in Mechanical Engineering at National Taiwan University.

News

Publications/Preprints

Please see my google scholar for an up-to-date list

*: equal contribution

2026

19. Active guidance in ultrasound bladder scanning using reinforcement learning

Hao-Lun Hsu*, Mohsen Zahiri*, Gary Y Li, Rashid Al Mukaddim, HyeonWoo Lee, Martha Grewe Wilson, Joyce Grube, Stephen Schmidt, Goutam Ghoshal, Balasundar Raju
In: Scientific Reports, 2026

2025

18. Safe Cooperative Multi-Agent Reinforcement Learning with Function Approximation

Hao-Lun Hsu, Miroslav Pajic
In: Proceedings 7th Learning for Dynamics and Control Conference (L4DC), 2025

17. Variational Adversarial Training Towards Policies with Improved Robustness

Juncheng Dong*, Hao-Lun Hsu*, Qitong Gao, Vahid Tarokh, Miroslav Pajic
In: Proc. of the 28th International Conference on Artificial Intelligence and Statistics (AISTATS), 2025

2024

16. Randomized Exploration in Cooperative Multi-Agent Reinforcement Learning

Hao-Lun Hsu*, Wexin Wang*, Miroslav Pajic, Pan Xu
In: Proc. of the 38th Conference on Advances in Neural Information Processing Systems (NeurIPS), 2024

15. StressFADS: Learning Latent Autonomic Factors of Stress in the Context of Trauma Recall and Neuromodulation

Asim H Gazi, Michael Chan, Hao-Lun Hsu, Douglas Bremner, Christopher Rozell, Omer T Inan
In: IEEE International Conference on Wearable and Implantable Body Sensor Networks (BSN), 2024

14. Steering Decision Transformers via Temporal Difference Learning

Hao-Lun Hsu, Alper Kamil Bozkurt*, Juncheng Dong*, Qitong Gao, Vahid Tarokh, Miroslav Pajic
In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2024

13. Reinforcement Learning for Closed-loop Regulation of Cardiovascular System with Vagus Nerve Stimulation: A Computational Study

Parisa Sarikhani, Hao-Lun Hsu, Mahmoud Zeydabadinezhad, Yuyu Yao, Mayuresh Kothare, Babak Mahmoudi
In: Journal of Neural Engineering, 2024

12. Robust Exploration with Adversary via Langevin Monte Carlo

Hao-Lun Hsu, Miroslav Pajic
In: Proceedings 6th Learning for Dynamics and Control Conference (L4DC), 2024

11. REFORMA: Robust REinFORceMent Learning via Adaptive Adversary for Drones Flying under Disturbances

Hao-Lun Hsu, Haocheng Meng, Shaocheng Luo, Juncheng Dong, Vahid Tarokh, Miroslav Pajic
In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA), 2024

10. ε-Neural Thompson Sampling of Deep Brain Stimulation for Parkinson Disease Treatment

Hao-Lun Hsu, Qitong Gao, Miroslav Pajic
In: Proceedings of International Conference on Cyber-Physical Systems (ICCPS), 2024

9. Finite-Time Frequentist Regret Bounds of Multi-Agent Thompson Sampling on Sparse Hypergraphs

Tianyuan Jin, Hao-Lun Hsu, William Chang, Pan Xu
In: Proceedings of Annual AAAI Conference on Artificial Intelligence (AAAI) (Oral, acceptance rate 2.3%), 2024

2023

8. Neuroweaver: a translational platform for embedding artificial intelligence in closed-loop neuromodulation systems

Parisa Sarikhani, Hanyang Xu, Shu-Ting Wang, Sean Kinzer, Hao-Lun Hsu, Yusen Zhu, Josh Krasney, Joseph R. Manns, Hadi Esmaeilzadeh, Babak Mahmoudi
In: Neuroscience 2023, 52nd Annual Meeting, 2023

2022

7. Improving Safety in Deep Reinforcement Learning Using Unsupervised Action Planning

Hao-Lun Hsu, Qiuhua Huang, Sehoon Ha
In: Proceedings of IEEE International Conference on Robotics and Automation (ICRA) , 2022

6. Automated Tuning of Closed-loop Neuromodulation Control Systems using Bayesian Optimization

Parisa Sarikhani, Hao-Lun Hsu, Babak Mahmoudi
In: 44th Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2022

2021

5. Neuroweaver: Towards a Platform for Designing Translatable Intelligent Closed-loop Neuromodulation Systems

Parisa Sarikhani, Hao-Lun Hsu, Joon Kyung Kim, Sean Kinzer, Edwin Mascarenhas, Hadi Esmaeilzadeh, Babak Mahmoudi
In: Neural Information Processing Systems (NeurIPS) Research2Clinics Workshop, 2021

4. Safe Exploration for Reinforcement Learning Using Unsupervised Action Planning

Hao-Lun Hsu, Qiuhua Huang, Sehoon Ha
In: Robotics: Science & Systems (RSS) Workshop on Integrating Planning and Learning, 2021

3. Sparc: Adaptive Closed-loop Control of Vagal Nerve Stimulation for Regulating Cardiovascular Function Using Deep Reinforcement Learning: A Computational Study

Parisa Sarikhani, Hao-Lun Hsu, Mahmoud Zeydabadinezhad, Yuyu Yao, Mayuresh Kothare, Babak Mahmoudi
In: Neuroscience 2021, 50th Annual Meeting, 2021

2. Neuroweaver: A Platform for Designing Intelligent Closed-loop Neuromodulation Systems

Parisa Sarikhani, Hao-Lun Hsu, Ozgur Kara, Joon Kyung Kim, Hadi Esmaeilzadeh, Babak Mahmoudi
In: 4th International Brain Stimulation Conference, 2021

1. Functional Connectivity Correlates to Individual Difference in Human Brains during Working Memory Task and Resting State

Hao-Lun Hsu
In: IEEE EMBS North American Virtual International Student Conference, 2021

Services

Please see my CV for an up-to-date list.


• Reviewer/Program Committee
  • Conferences: L4DC'24-25, ICRA’23-24, IROS’23, 25, NeurIPS’23, 25, ICLR’24-25, AISTATS’24-25, ICML’24-25
  • Workshops: ICML’23 Frontiers4LCD, NeurIPS’23 AI4Science, NeurIPS’23 GenBio, ICML'24 AI4Science, ICML'24 SPIGM, FPI-ICLR'25
  • Research Proposal: PURA (President’s Undergraduate Research Award) Fall'22

Invited Talks

Please see my CV for an up-to-date list.


• 02/2024: Duke Capital Partner
  Title: Reinforcement Learning for Cyber-Physical Sytems
• 11/2022: NCTPASS 2022 Annual Symposium
  Title: AI for Dynamical and Safety-critical Systems
• 07/2022: Curai Health ML paper club
  Title: Possible Reinforcement Learning Approaches to History Taking
• 03/2021: Artificial Intelligence Medicine Organization weekly webinar
  Title: Applications of Reinforcement Learning in healthcare and power grid control
• 03/2021: Prof. Constantine Dovrolis’s research group
  Title: Individual Difference in Humans’ Brains from Functional Connectivity for Working Memory

Teaching

Guest Lecturer

• Spring 2024 CS 370 Introduction To AI, Duke University
  Instructor: Tananun Songdechakraiwut
  Topic: Reinforcement Learning

Graduate Teaching Assistant at Duke University

• CompSci 535 Algorithmic Game Theory (Instructor: Kamesh Munagala) Spring 2024
• Compsci 590 Data Science (Instructor: Jian Pei) Spring 2023

Graduate Teaching Assistant at Georgia Tech

• CS 7280 Network Science: Methods and Applications (Instructor: Constantine Dovrolis) Spring, Summer, Fall 2021
  • Receive Thank a Teacher Award from the Center of Teaching and Learning, Georgia Tech

Teaching Assistant at National Taiwan University

• EE 5040 Clinical Application of Medical Electronic Device (Instructor: Chih-Ting Lin) Fall 2017
• Biomed 7110 Clinical Observation & Demands Exploration (Instructor: Fa-Hsuan Lin) Summer 2017

RL Research Mentoring (10~15 weeks)

• Taiwei Wu, University High School (robot navigation), Spring'25
• Tony Yang, Northland christian School (Q-learning), Spring'25
• Stefan Dragos, St. Augustine Preparatory School (robot navigation) Summer 2024
• Yang Chen, BS student at UC Berkeley (safe surgical robotics) Fall 2023
• Alexander Wang, West Windsor Plainsboro High School North (fake news detection) Fall 2023
• Nirav Jaiswal, Foothill High School (cloud computing) Summer 2023
• Indu Arimilli, Redmond High School (diagnosis prediction) Summer 2023
• Ian Choe, St. Mark’s School (deep brain stimulation) Summer 2023