Jianfeng Cai

Jianfeng Cai

Master's student at the School of Information Science and Technology, USTC.

Toolformer

Published Updated 1 minute read

  1. Sampling API Calls:
    1. Prompt P(x), sample i from 1 ~ n, filter p_i = p_M(|P(x),x_{1:i-1}) > t_s (up to k) -> I.
    2. m API calls, for i in I, using prefix [P(x), x_{1:i-1}, ] until generate -> c_i^1 ~ c_i^m
  2. Executing API Calls: for each c_i^j -> a single text sequence r_i^j.
  3. Filtering API Calls:

You May Also Enjoy

The Basic Knowledge of RLHF Training Pipeline

Updated 32 minute read

这篇博客主要讲解 RLHF 具体训练的框架 (DeepSpeedChat，OpenRLHF，verl) 的具体细节，包括每个框架的整体架构，架构内的各部分细节 (包括逻辑细节和代码细节)。(建议先阅读我之前关于 RLHF 的博客 The Basic Knowledge of RLHF (Reinforce Learning with Human Feedback))