Efficiency
The core thread: reduce memory, tokens, and inference cost.
主线：降低显存、token 与推理成本。
Research line研究主线
Jitai Hao 郝继泰 郝继泰 Jitai Hao
我在哈尔滨工业大学（深圳）攻读智能科学与技术博士学位，导师为 俞俊教授、黄强教授。我的研究聚焦于大模型架构设计与高效训练、推理，涵盖长上下文、稀疏推理引擎、统一多模态模型与流式视频理解。
I am a Ph.D. student in Intelligent Science and Technology at Harbin Institute of Technology (Shenzhen), supervised by Prof. Jun Yu and Prof. Qiang Huang. I study LLM architectures and efficient training and inference, including long-context serving, sparse inference engines, unified multimodal models, and streaming video understanding.
以一作／共一第一发表 4 篇 CCF-A 类顶会论文，包括 NeurIPS 2025 Spotlight（Top 3.2%）。曾获国家奖学金、两枚 ACM/ICPC 亚洲区域赛银牌。
Four CCF-A conference papers as first author or first-listed co-first author, including a NeurIPS 2025 Spotlight (Top 3.2%). Recipient of the National Scholarship and two ACM/ICPC Asia Regional silver medals.
OmniKV 首次提出跨层注意力索引复用，相关思路被智谱、腾讯混元等引用并用于基础模型，包括 GLM-5.2、GLM-5.3、Hy4、DeepSeek-V4.1。
OmniKV pioneered cross-layer attention-index reuse, an idea subsequently cited and adopted in foundation models by Zhipu, Tencent Hunyuan, and others, including GLM-5.2, GLM-5.3, Hy4, and DeepSeek-V4.1.
Research Lineage研究脉络
Efficient training, inference, and multimodal models 高效训练、推理与多模态模型MEFT
Memory-efficient fine-tuning with sparse adapters.
通过稀疏 adapter 做内存高效微调。
ACL 2024From Efficiency承接 Efficiency
OmniKV
Cross-layer attention-index reuse for long contexts.
通过跨层注意力索引复用，加速长上下文推理。
ICLR 2025From MEFT承接 MEFT
ResidualKV
Residual-based KV cache compression.
基于残差的 KV Cache 压缩。
arXiv 2026From OmniKV承接 OmniKV
SparseEngine
Sparse-first serving with 15 sparse methods.
以稀疏性为核心，统一支持 15 种方法。
arXiv 2026From OmniKV承接 OmniKV
LRC
Low-rank modules clone teacher knowledge for token-efficient training.
用低秩模块克隆教师知识，提升每个训练 token 的效率。
NeurIPS 2025 SpotlightFrom MEFT承接 MEFT
Unified Multimodal Models
The second thread: unify understanding and generation.
第二主线：统一理解与生成。
Research line研究主线Uni-X
Two-end-separated architecture for modality conflict.
两端分离架构，缓解模态冲突。
ICLR 2026From Unified Multimodal Models承接 Unified Multimodal Models
News
最新动态
Recent papers and milestones across the two research lines.
两条研究主线上的近期论文与进展。
Oct 2026
2026年10月
New preprints: HEAR on bidirectional coordination between agent harnesses and inference engines, and Beyond Refusal Patterns on safe-role internalization for LLM safety alignment.
新预印本：HEAR 探索 Agent 应用与推理引擎的双向协同；Beyond Refusal Patterns 研究通过安全角色内化实现大模型安全对齐。
Sep 2026
2026年9月
Released SparseEngine and ShallowStream. The updated ResidualKV preprint, previously titled DeltaKV, is also available.
发布 SparseEngine 和 ShallowStream 预印本；ResidualKV（原名 DeltaKV）的修订版也已公开。
Feb 2026
2026年2月
Our new paper Uni-X on mitigating modality conflict for unified multimodal models has been accepted to ICLR 2026 (Poster).
我们关于统一多模态模型训练中模态冲突缓解的新论文 Uni-X 已被 ICLR 2026 (Poster) 接收。
Sep 2025
2025年9月
Our paper on efficient knowledge distillation for LLMs has been accepted to NeurIPS 2025 (Spotlight). We propose Low-Rank Clone (LRC), an efficient pretraining method for SLMs.
我们关于大模型高效知识蒸馏的新论文已被 NeurIPS 2025 (Spotlight) 接收。我们提出 Low-Rank Clone (LRC)，一种高效的 SLM 预训练方法。
Publications
研究成果
Peer-reviewed papers. * denotes equal contribution; preprints and submissions are listed separately below.
已发表论文。* 表示共同贡献；预印本与在投论文单独列于下方。
Uni-X: Mitigating Modality Conflict with a Two-End-Separated Architecture for Unified Multimodal Models
Co-first author (listed first)共一第一
TL;DR: 提出“中间共享、两端分离”架构，缓解统一多模态模型中的模态冲突；Uni-X-3B 匹配当时 7B SOTA 模型的性能。积累了多机多卡训练经验（32 卡 H800）。
TL;DR: A shared-middle, separated-ends architecture mitigates modality conflict, with Uni-X-3B matching contemporary 7B state-of-the-art models. This work gave me multi-node training experience on 32 H800 GPUs.
A Token is Worth over 1,000 Tokens: Efficient Knowledge Distillation through Low-Rank Clone
First author第一作者
TL;DR: Low-Rank Clone（LRC）仅训练一组低秩投影矩阵，使用仅 10B-20B tokens 即可从头训练、蒸馏出高性能小语言模型，达到与使用数万亿 tokens 预训练的模型相当的性能。
TL;DR: Low-Rank Clone trains only low-rank projection matrices, using just 10B-20B tokens to train a high-performing small language model from scratch through distillation, competitive with models pretrained on trillions of tokens.
OmniKV: Dynamic Context Selection for Efficient Long-Context LLMs
Co-first author (listed first)共一第一
TL;DR: 首次提出跨层注意力索引复用，利用层间注意力相似性，结合动态上下文选择与 KV-cache offloading，在保留完整历史信息的同时降低长上下文推理成本。
TL;DR: Pioneered cross-layer attention-index reuse, combining inter-layer attention similarity, dynamic context selection, and KV-cache offloading to reduce long-context inference cost while retaining the full history.
The idea was cited by Zhipu's IndexCache (IndexShare) and adopted in foundation models, including GLM-5.2, GLM-5.3, Hy4, and DeepSeek-V4.1.
相关思路被智谱 IndexCache（IndexShare） 引用并应用于基础模型，包括 GLM-5.2、GLM-5.3、Hy4、DeepSeek-V4.1。
Multi-Defendant Legal Judgment Prediction via Hierarchical Reasoning
Co-first author (listed second)共一第二
TL;DR: 通过分层推理建模多被告案件中的复杂实体关系与推理链条，提高法律判决预测的准确性。
TL;DR: Uses hierarchical reasoning to model relationships and reasoning chains in multi-defendant cases, improving legal judgment prediction.
Preprints & Submissions
预印本与在投论文
Ongoing work; submission venues do not indicate acceptance. * Equal contribution.
以下为预印本与在投工作，投稿会议／期刊不代表已接收。* 表示共同贡献。
SparseEngine: Sparse-First Inference Engine
Co-first author (listed first) · Project & Paper共一第一 · Project & Paper
TL;DR: Designed around sparsity as a first principle, co-optimizing sparse attention, KV-cache management, and execution across 15 sparse methods. It delivers over 10× decode throughput with KV eviction and over 2× end-to-end agent speedup while maintaining method quality.
TL;DR: 以稀疏性为第一性设计原则，协同优化稀疏注意力、KV 缓存管理与执行流程，统一支持15 种稀疏方法。在保持方法质量的实验设置下，KV 淘汰场景 Decode 吞吐提升超过 10×，Agent 基准端到端加速超过 2×。
ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding
Co-first author (listed first)共一第一
TL;DR: Index with shallow layers, answer with the full model. A lightweight video index supports evidence retrieval at query time, reducing stream-processing time to about one third.
TL;DR: 提出浅层定位证据、完整深层回答：以浅层 KV 建立轻量视频索引，提问时检索相关证据并执行完整深层推理，将视频流式处理时间缩短至约 1/3。
Can Agent Harnesses and Inference Engines Hear Each Other? The HEAR Protocol for Agentic LLM Serving
Co-first author (listed third) · Project Leader共一第三 · 项目负责人
TL;DR: I originated the core idea and led the project. HEAR provides a bidirectional harness-engine protocol, exposing prefix-cache matches and engine load so agents can schedule requests more intelligently and smooth workload peaks.
TL;DR: 我提出核心 idea 并负责项目。HEAR 构建 Agent 应用与推理引擎之间的双向通信协议，让应用获知当前请求可命中的 prefix cache 与引擎负载，从而更智能地排布请求、削峰填谷。
ResidualKV: Residual-Based KV Cache Compression for Efficient Long-Context Inference
First author第一作者
TL;DR: Formerly DeltaKV. Encodes each token as reference KV + quantized residual and reconstructs selected states on demand, shrinking KV storage to as little as about 1/8 with near-lossless quality. Revised LongBench results use 13%-16% KV storage and 30% attention computation.
TL;DR: 原名 DeltaKV。将 token 的 KV 表示为reference KV + 量化 residual，按需重建选中的状态，在接近无损质量下将KV 缓存最低约为原来的 1/8。修订版 LongBench 实验使用 13%-16% 的 KV 存储与 30% 的注意力计算。
Beyond Refusal Patterns: Safe-Role Internalization for Robust and Generalizable LLM Safety Alignment
TL;DR: Supervised Safe-Role Fine-Tuning (SSRFT) teaches models to internalize a safe role, improving robustness to jailbreaks and generalization to unseen attacks while reducing over-refusal.
TL;DR: 提出 Supervised Safe-Role Fine-Tuning（SSRFT），通过内化安全角色提升对越狱攻击的鲁棒性和对未见攻击的泛化能力，并减少对正常问题的过度拒答。
Projects
项目
Systems that turn the efficiency line into reusable infrastructure.
把效率主线沉淀成可复用系统。
SparseEngine
稀疏优先的推理引擎，支持 15 种方法，并通过 Chain Cache 与可控 Prefix-Cache Pruning 协调跨请求缓存状态。
A sparse-first inference engine with 15 methods, Chain Cache, and controllable Prefix-Cache Pruning for cross-request state management.
ShallowStream
浅层索引、深层回答：面向实时视频流的高效多模态推理。
Shallow indexing and deep answering for efficient multimodal inference over continuous video streams.
哈尔滨工业大学点子基金
HIT Idea Fund
主持 · 项目负责人。研究统一多模态模型的架构改良，以及高效训练与推理。
Project lead. Research on unified multimodal architectures and efficient training and inference.
Education
教育背景
Harbin Institute of Technology (Shenzhen)
哈尔滨工业大学（深圳）
Ph.D. student in Intelligent Science and Technology
智能科学与技术 · 博士在读
Advisors: Prof. Jun Yu and Prof. Qiang Huang
导师：俞俊教授、黄强教授
Shandong University
山东大学
B.Sc. and M.Sc. in Computer Science and Technology · 2018-2025
计算机科学与技术 · 学士与硕士 · 2018-2025
Advisor: Prof. Zhaochun Ren
导师：任昭春教授
Internship Experience
实习经历
- 百度文心一言，研究实习生，2025年3月 - 2025年10月
研究成果：Low-Rank Clone（NeurIPS 2025 Spotlight）、Uni-X（ICLR 2026）。 - 蚂蚁集团，研究实习生，2024年5月 - 2024年10月
研究成果：OmniKV（ICLR 2025）。
- Baidu, ERNIE, Research Intern, Mar. 2025 - Oct. 2025
Research: Low-Rank Clone (NeurIPS 2025 Spotlight) and Uni-X (ICLR 2026). - Ant Group, Research Intern, May 2024 - Oct. 2024
Research: OmniKV (ICLR 2025).
Awards & Honors
奖项与荣誉
- 国家奖学金
- ACM/ICPC 亚洲区域赛银牌，2 枚
- 大学生软件创新大赛 OPPO 杯，国家一等奖
- National Scholarship
- ACM/ICPC Asia Regional Contest, Silver Medal, x2
- National First Prize, University Student Software Innovation Competition, OPPO Cup