📝 Publications

Conference Paper

• Episodic Novelty Through Temporal Distance. Yuhua Jiang*, Qihan Liu*, Yiqin Yang, Xiaoteng Ma, Dianyu Zhong, Hao Hu, Jun Yang, Bin Liang, Bo Xu, Chongjie Zhang, Qianchuan Zhao. International Conference on Learning Representations (ICLR), 2025.

• Cross-Domain Offline Policy Adaptation with Optimal Transport and Dataset Constraint. Jiafei Lyu, Mengbei Yan, Zhongjian Qiao, Runze Liu, Xiaoteng Ma, Deheng Ye, Jing-Wen Yang, Zongqing Lu, Xiu Li. International Conference on Learning Representations (ICLR), 2025.

• NeuralPlane: An Efficiently Parallelizable Platform for Fixed-wing Aircraft Control with Reinforcement Learning. Chuanyi Xue*, Qihan Liu*, Xiaoteng Ma*, Xinyao Qin, Ning Gui, Yang Qi, Jinsheng Ren, Bin Liang, Jun Yang. Advances in Neural Information Processing Systems Datasets and Benchmarks Track (NeurIPS), 2024.

• Single-Trajectory Distributionally Robust Reinforcement Learning. Zhipeng Liang*, Xiaoteng Ma*, Jose Blanchet, Jiheng Zhang, Zhengyuan Zhou. International Conference on Machine Learning (ICML), 2024.

• Efficient Multi-agent Reinforcement Learning by Planning. Qihan Liu*, Jianing Ye*, Xiaoteng Ma*, Jun Yang, Bin Liang, Chongjie Zhang. International Conference on Learning Representations (ICLR), 2024.

• SEABO: A Simple Search-Based Method for Offline Imitation Learning. Jiafei Lyu, Xiaoteng Ma, Le Wan, Runze Liu, Xiu Li, Zongqing Lu. International Conference on Learning Representations (ICLR), 2024.

• Learning Diverse Risk Preferences In Population-based Self-play. Yuhua Jiang*, Qihan Liu*, Xiaoteng Ma, Chenghao Li, Yiqin Yang, Jun Yang, Bin Liang, Qianchuan Zhao. AAAI Conference on Artificial Intelligence, (AAAI), 2024. (Oral)

• Cross-Domain Policy Adaptation via Value-Guided Data Filtering. Kang Xu, Chenjia Bai, Xiaoteng Ma, Dong Wang, Bin Zhao, Zhen Wang, Xuelong Li, Wei Li. Advances in Neural Information Processing Systems (NeurIPS), 2023.

• Uncertainty-driven Trajectory Truncation for Model-based Offline Reinforcement Learning. Junjie Zhang*, Jiafei Lyu*, Xiaoteng Ma, Jiangpeng Yan, Jun Yang, Le Wan, Xiu Li. European Conference on Artificial Intelligence (ECAI), 2023.

• What Is Essential for Unseen Goal Generalization of Offline Goal-conditioned RL? Rui Yang, Yong Lin, Xiaoteng Ma, Hao Hu, Chongjie Zhang, Tong Zhang. International Conference on Machine Learning (ICML), 2023.

• Mildly Conservative Q-Learning for Offline Reinforcement Learning. Jiafei Lyu*, Xiaoteng Ma*, Xiu Li, Zongqing Lu. Advances in Neural Information Processing Systems (NeurIPS), 2022. (Spotlight)

• RORL: Robust Offline Reinforcement Learning via Conservative Smoothing. Rui Yang*, Chenjia Bai*, Xiaoteng Ma, Zhaoran Wang, Chongjie Zhang, Lei Han. Advances in Neural Information Processing Systems (NeurIPS), 2022. (Spotlight)

• Exploiting Reward Shifting in Value-Based Deep RL. Hao Sun, Lei Han, Rui Yang, Xiaoteng Ma, Jian Guo, Bolei Zhou. Advances in Neural Information Processing Systems (NeurIPS), 2022.

• Offline Reinforcement Learning with Value-based Episodic Memory. Xiaoteng Ma*, Yiqin Yang*, Hao Hu*, Qihan Liu, Jun Yang, Chongjie Zhang, Qianchuan Zhao, Bin Liang. International Conference on Learning Representations (ICLR), 2022.

• Efficient Continuous Control with Double Actors and Regularized Critics. Jiafei Lyu*, Xiaoteng Ma*, Jiangpeng Yan, Xiu Li. AAAI Conference on Artificial Intelligence, (AAAI), 2022. (Oral)

• Believe What You See: Implicit Constraint Approach for Offline Multi-Agent Reinforcement Learning. Yiqin Yang*, Xiaoteng Ma*, Chenghao Li, Zewu Zheng, Qiyuan Zhang, Gao Huang, Jun Yang and Qianchuan Zhao. Advances in Neural Information Processing Systems (NeurIPS), 2021. (Spotlight)

• Average-Reward Reinforcement Learning with Trust Region Methods. Xiaoteng Ma, Xiaohang Tang, Jun Yang, Li Xia, Qianchuan Zhao. International Joint Conference on Artificial Intelligence (IJCAI), 2021.

• Modeling the Interaction between Agents in Cooperative Multi-Agent Reinforcement Learning. Xiaoteng Ma*, Yiqin Yang*, Chenghao Li*, Yiwen Lu, Qianchuan Zhao and Jun Yang. International Conference on Autonomous Agents and MultiAgent Systems (AAMAS), 2021.

• Wasserstein Distance guided Adversarial Imitation Learning with Reward Shape Exploration. Ming Zhang, Yawei Wang, Xiaoteng Ma, Li Xia, Jun Yang, Zhiheng Li and Xiu Li. IEEE Data Driven Control and Learning Systems Conference (DDCLS), 2020.

• Fairness Control of Traffic Light via Deep Reinforcement Learning. Chenghao Li, Xiaoteng Ma, Li Xia, Qianchuan Zhao and Jun Yang. IEEE International Conference on Automation Science and Engineering (CASE), 2020.

• Bi-level Proximal Policy optimization for Stochastic Coordination of EV Charging Load with Uncertain Wind Power. Teng Long, Xiaoteng Ma, Qing-Shan Jia. IEEE Conference on Control Technology and Applications (CCTA), 2019.

• Attendance and security system based on building video surveillance. Kailai Sun, Qianchuan Zhao, Jianhong Zou, Xiaoteng Ma. International Conference on Smart City and Intelligent Building (ICSCIB), 2018.

Journal Paper

• CVaR-Constrained Policy Optimization for Safe Reinforcement Learning. Qiyuan Zhang, Shu Leng, Xiaoteng Ma, Qihan Liu, Xueqian Wang, Bin Liang, Yu Liu, Jun Yang. IEEE Transactions on Neural Networks and Learning Systems (TNNLS), 2024.

• A unified algorithm framework for mean-variance optimization in discounted Markov decision processes. Shuai Ma, Xiaoteng Ma, Li Xia. European Journal of Operational Research (EJOR), 2023.

• Mean-Semivariance Policy Optimization via Risk-Averse Reinforcement Learning. Xiaoteng Ma, Shuai Ma, Li Xia, Qianchuan Zhao. Journal of Artificial Intelligence Research (JAIR), 2022.

• MPSN: Motion-aware Pseudo-Siamese Network for Indoor Video Head Detection in Buildings. Kailai Sun*, Xiaoteng Ma*, Peng Liu*, Qianchuan Zhao. Building and Environment (BAE), 2022.

• An optimistic value iteration for mean–variance optimization in discounted Markov decision processes. Shuai Ma, Xiaoteng Ma, Li Xia.Results in Control and Optimization (RICO), 2021.

• Learning to Discover Task-Relevant Features for Interpretable Reinforcement Learning. Qiyuan Zhang, Xiaoteng Ma, Yiqin Yang, Chenghao Li, Jun Yang, Yu Liu and Bin Liang. IEEE Robotics and Automation Letters (RA-L), 2021.

• Reinforcement learning for fluctuation reduction of wind power with energy storage. Zhen Yang, Xiaoteng Ma, Li Xia, Qianchuan Zhao and Xiaohong Guan. Results in Control and Optimization (RICO), 2021.

Preprint

• DSAC: Distributional Soft Actor Critic for Risk-Sensitive Reinforcement Learning. Xiaoteng Ma, Li Xia, Zhengyuan Zhou, Jun Yang and Qianchuan Zhao.

• Distributionally Robust Offline Reinforcement Learning with Linear Function Approximation. Xiaoteng Ma*, Zhipeng Liang*, Jose Blanchet, Mingwen Liu, Li Xia, Jiheng Zhang, Qianchuan Zhao, Zhengyuan Zhou.