Qualls, 33, was 1-1 with a 4.60 ERA in 35 appearances for the Phillies, finishing six games. In 31 1/3 innings, he allowed 39 hits and nine walks, striking out 19.
He was designated for assignment by Philadelphia on June 28 and is expected to join the Yankees on Monday at Tropicana Field.
"He's another guy that can get up to 95 mph, somewhat of a power guy -- a sinker guy [who is] very tough on right-handers," Yankees manager Joe Girardi said. "We mix and match a bit here, so we hope he's really effective for us."
Qualls has a 39-35 record with 51 saves and a 3.82 ERA over nine seasons with the Astros, D-backs, Rays, Padres and Phillies. He could be in line to take the roster spot of struggling right-hander Cory Wade, who owns a 5.79 ERA in 33 appearances and had a 13.97 ERA in 12 June appearances.
To make room for Qualls on the 40-man roster, the Yankees designated infielder Matt Antonelli for assignment. The 27-year-old was batting .203 (29-for-143) with two home runs and 10 RBIs in 41 combined games for Triple-A Norfolk, an Orioles affiliate, and Triple-A Scranton/Wilkes-Barre this season.