Jianfeng Cai

Jianfeng Cai

Master's student at the School of Information Science and Technology, USTC.

The Basic Knowledge of Distributed Training Pipeline

Published Updated less than 1 minute read

Work in progress. This note is still being refined and may be incomplete.

敬请期待🤪

You May Also Enjoy

The Basic Knowledge of RLHF Training Pipeline

Updated 33 minute read

这篇博客主要讲解 RLHF 具体训练的框架 (DeepSpeedChat，OpenRLHF，verl) 的具体细节，包括每个框架的整体架构，架构内的各部分细节 (包括逻辑细节和代码细节)。(建议先阅读我之前关于 RLHF 的博客 The Basic Knowledge of RLHF (Reinforce Learning with Human Feedback))

Toolformer

Updated 1 minute read

Sampling API Calls: Prompt P(x), sample i from 1 ~ n, filter p_i = p_M(|P(x),x_{1:i-1}) > t_s (up to k) -> I. m API calls, for i in I, using prefix [P(x), x_{1:i-1}, ] until generate -> c_i^1 ~ c_i^m Executing API Calls: for eac...