郝继泰

Jitai Hao

Email GitHub Scholar

关于我

About Me

我的主要研究方向为 Efficient AI/LLMs。我于 2022 和 2025 年分别在山东大学获得了计算机科学学士和硕士学位，导师是 任昭春教授。我现在是哈尔滨工业大学（深圳）的博士生，导师为 俞俊教授。

My main research focus is on Efficient AI/LLMs. I obtained my Bachelor's and Master's degrees in Computer Science from Shandong University in 2022 and 2025, respectively, under the supervision of Prof. Zhaochun Ren. Currently, I am a Ph.D. student at Harbin Institute of Technology (Shenzhen), supervised by Prof. Jun Yu.

最新动态

News

[2025年9月] 我们关于大模型高效知识蒸馏的新论文已被 NeurIPS 2025 (Spotlight) 接收！我们提出了 Low-Rank Clone (💖LRC💖)，一种创新的 SLM 高效预训练方法。 LRC 仅需约 10B-20B tokens 即可达到甚至超越需要数万亿 (Trillions) tokens 训练的SOTA模型。

[Sep 2025] Our new paper on efficient knowledge distillation for LLMs has been accepted to NeurIPS 2025 (Spotlight)! We propose Low-Rank Clone (💖LRC💖), an innovative and efficient pretraining method for SLMs. LRC can achieve, or even surpass, the performance of SOTA models trained on trillions of tokens with only about 10B-20B tokens.

研究成果

Publications

A Token is Worth over 1,000 Tokens: Efficient Knowledge Distillation through Low-Rank Clone

Jitai Hao, Qiang Huang, Hao Liu, Xinyan Xiao, Zhaochun Ren, Jun Yu
NeurIPS 2025 (Spotlight)
PDF Code
TL;DR: 我们提出 Low-Rank Clone (LRC)，一种能显著提高模型训练效率的创新方法。LRC 仅需约 10B-20B tokens 数据，就能达到甚至超越那些用数万亿 tokens 训练的顶尖模型（如 Qwen3, Llama3）的性能。
TL;DR: We propose Low-Rank Clone (LRC), an innovative method to significantly improve model training efficiency. With only about 10B-20B tokens, LRC can match or even surpass the performance of SOTA models like Qwen3 and Llama3, which are trained on trillions of tokens.

OmniKV: Dynamic Context Selection for Efficient Long-Context LLMs

Jitai Hao*, Yuke Zhu*, Tian Wang, Jun Yu, Xin Xin, Bo Zheng, Zhaochun Ren, Sheng Guo
ICLR 2025
PDF Code
TL;DR: 通过创新地提出层间注意力相似性，OmniKV 能够动态选择最重要的上下文信息，从而在处理长文本时，显著提升 LLM 的效率和性能，同时降低计算成本。
TL;DR: By innovatively proposing inter-layer attention similarity, OmniKV dynamically selects the most crucial context information, significantly enhancing the efficiency and performance of LLMs for long-context tasks while reducing computational costs.

MEFT: Memory-Efficient Fine-Tuning through Sparse Adapter

Jitai Hao, Weiwei Sun, Xin Xin, Qi Meng, Zhumin Chen, Pengjie Ren, Zhaochun Ren
ACL 2024
PDF Code
TL;DR: MEFT 是一种内存效率更高的模型微调方法。它通过引入稀疏适配器（Sparse Adapter）来减少微调时占用的内存，使得在大模型上进行微调更加可行和高效。
TL;DR: MEFT is a memory-efficient fine-tuning method. It reduces memory usage during fine-tuning by introducing a sparse adapter, making it more feasible and efficient to fine-tune large models.

Multi-Defendant Legal Judgment Prediction via Hierarchical Reasoning

Yougang Lyu*, Jitai Hao*, Zihan Wang, Kai Zhao, Shen Gao, Pengjie Ren, Zhumin Chen, Fang Wang, Zhaochun Ren
EMNLP 2023 Findings
PDF Code
TL;DR: 本文研究如何通过分层推理来预测涉及多个被告的法律判决结果，旨在理解案件中复杂的实体关系和逻辑链条，以提高预测的准确性。
TL;DR: This paper investigates how to predict legal judgment outcomes involving multiple defendants through hierarchical reasoning, aiming to understand complex relationships and logical chains to improve prediction accuracy.

实习经历

Internship Experience

  • 百度 (2025年3月 - 至今)，研究实习生
  • 蚂蚁集团 (2024年5月 - 2024年10月)，研究实习生
  • Baidu (Mar. 2025 - Present), Research Intern
  • Ant Group (May 2024 - Oct. 2024), Research Intern

奖项与荣誉

Awards & Honors

  • 国家奖学金，一等学业奖学金等
  • ACM/ICPC 亚洲区域赛银牌 (2枚)
  • 大学生软件创新大赛 (OPPO 杯) 国家一等奖
  • National Scholarship, First-Class Academic Scholarship, etc.
  • ACM/ICPC Asia Regional Contest, Silver Medal (x2)
  • National First Prize, University Student Software Innovation Competition (OPPO Cup)