Sure, general manager Brian Cashman desperately needs the 38-year-old Pettitte and his left arm, but it is extremely important to bring closure to this winter's ongoing speculation that Pettitte hasn't been serious about calling it a career.
The feeling was the Yankees would make an offer he couldn't refuse. There was even one unconfirmed report they had offered him $12 million to pitch in 2011.
So, unless he mimics Roger Clemens and fires up the body and makes a comeback mid-season, one of the great Yankee careers is over.
Once the Yankees were unable to land free-agent lefty Cliff Lee, the status of Pettitte became a priority. Cashman couldn't go an hour without somebody asking if Pettitte was really retired.
Cashman reiterated to me 10 days ago he had every reason to believe Pettitte was going to retire. They had talked and that was the indication the GM got from the left-hander.
"He has made a decision," Cashman said, "it's just if he changes his mind. I wouldn't even say he's undecided. He's decided not to play."
But still, there was hope, especially last week, when it was learned the pitcher was working out near his Texas home.
Pettitte's decision leaves the Yankees with a rotation headed by left-hander CC Sabathia, and right-handers A.J. Burnett and Phil Hughes.
This is a big blow for the Yankees and their rotation. They desperately need a premier left-hander in addition to Sabathia to combat the Red Sox's lefty-hitting-dominated batting order.
Even though he's 38, Pettitte could still be one of the best left-handers in the Major Leagues. Had he returned, at least for one year, he would have softened the hurt the Yankees suffered when Lee turned down their offer and signed with Philadelphia -- for less money!
When Cashman signed rehab veterans Freddy Garcia and Bartolo Colon to Minor League contracts last week, I had a hunch Pettitte wasn't coming back.
But Garcia, Colon and righties Ivan Nova and Segio Mitre are no Pettitte.
The Yankees, if they are to contend with the Red Sox and other American League powers, need a starting pitcher. To put it bluntly, they're pitching-short now.
Pettitte's decision leaves only Derek Jeter, Mariano Rivera and Jorge Posada of the "Core Four," the group of homegrown Yankees who helped them win their past five World Series championships.
I remember years ago when the Yankees almost traded a young Pettitte to the Phillies. He had to wonder then what his future with New York held.
What he did was build a career that, in the end, is second only to Whitey Ford among Yankees lefties.
He won 240 games and had a .635 winning percentage, but what I remember most about Pettitte is his postseason work.
There is no pitcher in the history of the game who can match his 42 postseason starts.
Close your eyes and you can see Andy staring over that big, black glove at hitters. He was intimidating and dominating.
In those 42 starts, he was 19-10 with a 3.83 ERA. Even more important, in the World Series, he was 7-2 with a 3.63 ERA.
Or as teammate Nick Swisher said this week at a banquet: "If he chooses to ride off into the sunset, he's definitely earned that. We would love to have him back, but if he's got to go, he's going with glory."
The only blemish on Pettitte's career was his admission in 2007 that he used HGH to recover from an elbow injury in '02.
"If what I did was an error in judgment on my part, I apologize," Pettitte said then. "I accept responsibility for those two days."
But even with that negative, Andy turned it into a positive by, unlike many others, admitting the use. He moved on, and I believe the fact he was so up front lessened the media attention that such events have caused.
Assuming Pettitte doesn't reverse course and make a mid-season return, the next discussion of his career will be if he is Hall of Fame-worthy.
That will become a spirited debate between now and the time he gets on the ballot for the first time in 2016. (Candidates must be retired for five years).
I don't believe his 240 wins are enough for Cooperstown, but when you factor in the awesome postseason record and five World Series rings, his candidacy becomes more viable.
That's years away.
For now, there's a lot of sadness at Yankee Stadium that Pettitte has chosen a rocking chair over pinstripes.