On Thursday, Torre's visit to the facility was under much happier circumstances.
In his new role as Major League Baseball's executive vice president of baseball operations, Torre visited his old office and met with Yankees manager Joe Girardi and general manager Brian Cashman, among others.
"I was sort of looking forward to it and dreading it at the same time, because I knew it would be emotional," said Torre, who was hired by MLB in late February after retiring from managing the Dodgers last season.
"I don't think the emotion will ever go out of it because of what these 12 years that I spent here meant to me. It's not sad by any stretch of the imagination. It was a great run, and you can't appreciate the good times unless there's some bumps along the way. I wouldn't change a thing."
Torre, 70, won four World Series titles during his employment as Yankees manager, running from 1996 through the 2007 season, directing the team to 12 consecutive postseason appearances.
2010 Spring Training - null
Sights & Sounds
Spring Training Info
He said wandering his old domain "does feel strange, but it feels good," and he is well past the odd conclusion to his Yankees career, which ended with Torre getting back on a plane to New York on the afternoon Oct. 18, 2007, contract left unsigned.
"It's water under the bridge, I guess," Torre said. "I never would have changed anything, to be honest with you. You wish that the time I left could have ended differently, but I don't know how it could have. I don't know if either one of us knew how to say goodbye at that point."
Though the relationship between Torre and the Yankees seemed frosty at first, especially after Torre and co-author Tom Verducci collaborated on a book, "The Yankee Years," the thawing process began last year when Torre attended the unveiling of Steinbrenner's Monument Park tribute last year.
Torre also said that he has been invited to this year's Old-Timer's Day festivities at Yankee Stadium on June 26, an event he plans to attend.
"It means a lot," Torre said. "Without this opportunity here, my whole professional career would have been different. This was the main part of my career -- I know I played for a long time and had some memorable times, but the time in New York, I'll never forget what it meant."
During his visit to the clubhouse on Thursday, Torre also said he chatted with Derek Jeter, Jorge Posada, Robinson Cano and catcher Russell Martin, who played for Torre with the Dodgers.
"This has been great for me this spring, because every single camp I've gone to, there's a connection with somebody," Torre said. "When you're old, you have a connection with a lot of people."
Bryan Hoch is a reporter for MLB.com. This story was not subject to the approval of Major League Baseball or its clubs.