郝继泰

Jitai Hao

Email GitHub Scholar

关于我

About Me

我的主要研究方向为 高效大模型训练和推理。我于 2022 和 2025 年分别在山东大学获得了计算机科学学士和硕士学位（人工智能实验班）。我的本科和硕士导师是 任昭春教授。我即将成为哈尔滨工业大学（深圳）的博士生，导师为 俞俊教授。

My main research focus is on efficient training and inference of large language models (LLMs). I obtained my Bachelor's and Master's degrees in Computer Science from Shandong University in 2022 and 2025, respectively (Artificial Intelligence Experimental Class). My advisor for my Bachelor's and Master's degrees was Zhaochun Ren. I will soon be joining HITSZ as a Ph.D. student, advised by Jun Yu. During my undergraduate studies, I was awarded the ACM/ICPC Regional Contest Silver Medal, demonstrating strong coding abilities.

最新动态

News

[2025年5月] 我们关于大模型高效知识蒸馏的新论文已发布在 arXiv！我们提出了 Low-Rank Clone (💖LRC💖)，一种创新的 SLM 高效预训练方法。 LRC 仅需约 10B-20B tokens 即可达到甚至超越需要数万亿 (Trillions) tokens 训练的SOTA模型。

[May 2025] Our new paper on efficient knowledge distillation for LLMs is now available on arXiv! We propose Low-Rank Clone (💖LRC💖), an innovative and efficient pretraining method for SLMs. LRC can achieve, or even surpass, the performance of SOTA models trained on trillions of tokens with only about 10B-20B tokens.

研究成果

Publications

A Token is Worth over 1,000 Tokens: Efficient Knowledge Distillation through Low-Rank Clone

Jitai Hao, Qiang Huang, Hao Liu, Xinyan Xiao, Zhaochun Ren, Jun Yu
arXiv:2505.12781 [cs.CL], 2025
PDF Code
TL;DR: 我们提出 Low-Rank Clone (LRC)，一种能显著提高模型训练效率的创新方法。LRC 仅需约 10B-20B tokens 数据，就能达到甚至超越那些用数万亿 tokens 训练的顶尖模型（如 Qwen3, Llama3）的性能。
TL;DR: We propose Low-Rank Clone (LRC), an innovative method to significantly improve model training efficiency. With only about 10B-20B tokens, LRC can match or even surpass the performance of SOTA models like Qwen3 and Llama3, which are trained on trillions of tokens.

OmniKV: Dynamic Context Selection for Efficient Long-Context LLMs

Jitai Hao*, Yuke Zhu*, Tian Wang, Jun Yu, Xin Xin, Bo Zheng, Zhaochun Ren, Sheng Guo
ICLR 2025
PDF Code
TL;DR: 通过创新地提出层间注意力相似性，OmniKV 能够动态选择最重要的上下文信息，从而在处理长文本时，显著提升 LLM 的效率和性能，同时降低计算成本。
TL;DR: By innovatively proposing inter-layer attention similarity, OmniKV dynamically selects the most crucial context information, significantly enhancing the efficiency and performance of LLMs for long-context tasks while reducing computational costs.

MEFT: Memory-Efficient Fine-Tuning through Sparse Adapter

Jitai Hao, Weiwei Sun, Xin Xin, Qi Meng, Zhumin Chen, Pengjie Ren, Zhaochun Ren
ACL 2024
PDF Code
TL;DR: MEFT 是一种内存效率更高的模型微调方法。它通过引入稀疏适配器（Sparse Adapter）来减少微调时占用的内存，使得在大模型上进行微调更加可行和高效。
TL;DR: MEFT is a memory-efficient fine-tuning method. It reduces memory usage during fine-tuning by introducing a sparse adapter, making it more feasible and efficient to fine-tune large models.

Multi-Defendant Legal Judgment Prediction via Hierarchical Reasoning

Yougang Lyu*, Jitai Hao*, Zihan Wang, Kai Zhao, Shen Gao, Pengjie Ren, Zhumin Chen, Fang Wang, Zhaochun Ren
EMNLP 2023 Findings
PDF Code
TL;DR: 本文研究如何通过分层推理来预测涉及多个被告的法律判决结果，旨在理解案件中复杂的实体关系和逻辑链条，以提高预测的准确性。
TL;DR: This paper investigates how to predict legal judgment outcomes involving multiple defendants through hierarchical reasoning, aiming to understand complex relationships and logical chains to improve prediction accuracy.

实习经历

Internship Experience

  • 百度 (2025年3月 - 至今)，研究实习生
  • 蚂蚁集团 (2024年5月 - 2024年10月)，研究实习生
  • Baidu (Mar. 2025 - Present), Research Intern
  • Ant Group (May 2024 - Oct. 2024), Research Intern

奖项与荣誉

Awards & Honors

  • 国家奖学金，一等学业奖学金等
  • ACM/ICPC 亚洲区域赛银牌 (2枚)
  • 大学生软件创新大赛 (OPPO 杯) 国家一等奖
  • National Scholarship, First-Class Academic Scholarship, etc.
  • ACM/ICPC Asia Regional Contest, Silver Medal (x2)
  • National First Prize, University Student Software Innovation Competition (OPPO Cup)