LLMs such as ChatGPT are trained in two stages. They first learn from a huge amount of text, and then they are tuned with reinforcement learning, i.e., the model tries an answer, gets a score, and updates itself. The second stage is what makes a model follow instructions and reason step by step. However, this stage is unstable and expensive. A small change in the reward could make the training collapse, and human feedback is costly to collect.

We study how to make this stage stable and data-efficient. E.g., StaRPO adds stability terms to policy optimization for LLM reasoning, and Proto-RM learns a reward model from a small number of human labels. We also study on-policy distillation, in which a small model learns to reason from its own outputs under the guidance of a larger model.

• StaRPO: Stability-Augmented Reinforcement Policy Optimization. NeurIPS 2026.
• RATT: A Thought Structure for Coherent and Correct LLM Reasoning. AAAI 2025.
• Entropy-based Exploration Conduction for Multi-step Reasoning. ACL 2025.
• Prototypical Reward Network for Data-Efficient RLHF. ACL 2024.

Many science and engineering problems are decision problems, e.g., how much fluid to inject into a geothermal well, or which ambulance to send to an emergency call. Physical simulators can evaluate these decisions, but they are slow, and real-world data is limited. To tackle this problem, we train RL policies on fast learned surrogates of the simulator, and we study how to transfer a policy trained in simulation to the real world.

• Efficient Geothermal Well-Control Optimization via Diffusion-Surrogate Reinforcement Learning. IEEE BigData 2026.
• Beyond Simulate-Then-Optimize: Geothermal AI for Geothermal Dynamics Prediction, Design, and Discovery. ACM SIGKDD Explorations 2026.
• City Editing: Hierarchical Agentic Execution for Dependency-Aware Urban Geospatial Modification. SIGSPATIAL 2026.
• Ambulance Dispatch via Deep Reinforcement Learning. SIGSPATIAL 2020.

A Transformer becomes slow when the input is long, because the cost of attention grows with the square of the input length. This is a problem for long documents and for devices without a GPU. We study linear attention and delta networks, whose cost grows linearly with the input length. We also study test-time training, in which a model keeps updating a small part of itself while it reads the input. This line started in 2026.

• Visual Representation and History Modeling for Navigation World Models. IEEE BigData 2026.

As we all know, data decides the upper bound of a machine learning model. In practice, people spend a lot of time cleaning data and designing features by hand. Our perspective is to let an AI agent do this work. Specifically, we formulate feature selection and feature generation as a reinforcement learning problem. The agents decide which features to keep and which new features to create, and the accuracy of the downstream model is the reward. Recently, we also use LLMs and retrieval to bring domain knowledge into the generated features.

• Retrieval-Augmented Feature Generation for Domain-Specific Classification. ICDM 2025.
• Adaptive Context-Infused Performance Evaluator for Iterative Feature Space Optimization. CIKM 2025.
• Automated Feature Selection: A Reinforcement Learning Perspective. IEEE TKDE 2023.
• Dynamic and Adaptive Feature Generation with LLM. IJCAI 2025.