53.
two main parallelisms to understand for distributed training
two main parallelisms to understand for distributed training id probably group most of the parallelism to fall under inter-op or intra-op parallelism or both. inter-op parallelism 1. assign different operators to different devices. The se