Tree all-reduce · 4 GPUs
A = ⊕aᵢ
B = ⊕bᵢ
C = ⊕cᵢ
D = ⊕dᵢ
i = 0…3 · GPU0 = root
Previous
Next
Input
Step 0 of 4 · Local vectors