Montgomery was optioned to Triple-A Scranton/Wilkes-Barre following Wednesday's doubleheader with the Indians.
Montgomery, who was tagged for four runs over four innings of work in Game 2 of Wednesday's doubleheader, will head back to the minors after serving as the 26th man. The 24-year-old should be back with the Yankees in the near future to serve as a long reliever out of the bullpen down the stretch. He would also be the top option for a spot start should the Yankees need a sixth starter at any point in September.