Jianfeng Cai

Jianfeng Cai

Master's student at the School of Information Science and Technology, USTC.

The Basic Knowledge of Distributed System Cluster

Published Updated 1 minute read

Reference

  1. Network address translation (Wikipedia)
  2. What Is NAT (Network Address Translation)?
  3. What is the Gateway ?
  4. Clustering vs Distributed Systems
  5. Distributed Cluster System
  6. Introduction to Cluster Computing
  7. Introduction to Parallel Computing Tutorial
  8. An Overview of Cluster Computing
  9. Difference between Cloud Computing and Cluster Computing

You May Also Enjoy

The Basic Knowledge of RLHF Training Pipeline

Updated 32 minute read

这篇博客主要讲解 RLHF 具体训练的框架 (DeepSpeedChat，OpenRLHF，verl) 的具体细节，包括每个框架的整体架构，架构内的各部分细节 (包括逻辑细节和代码细节)。(建议先阅读我之前关于 RLHF 的博客 The Basic Knowledge of RLHF (Reinforce Learning with Human Feedback))

Toolformer

Updated 1 minute read

Sampling API Calls: Prompt P(x), sample i from 1 ~ n, filter p_i = p_M(|P(x),x_{1:i-1}) > t_s (up to k) -> I. m API calls, for i in I, using prefix [P(x), x_{1:i-1}, ] until generate -> c_i^1 ~ c_i^m Executing API Calls: for eac...