network stucture issue

I just go through the model in MS_Deeplab.  In forward pass, the input image x has been pass through Resnet 4 times with different scales. But according to original paper, multi scale should happen after layer5 in resnet. May I know why implementation this way?