After spending season four of "Jersey Shore" in Italy, the Seaside gang headed right back to Jersey, as fans saw after last week's season-five premiere. But while they wasted no time getting into their old routine of hitting the gym, tanning and fist-pumping the night away, one castmember wasn't really up for GTL.

"I had to go home to recharge; Pauly is just not enough," Vinny told MTV News , about his departure on Thursday night's (January 11) episode.

And although Ronnie assured us that fans of the hit MTV show would see a much "lighter side" of the crew this year, it seems Vinny had a hard time adjusting to life back in Seaside Heights. With little personal time between the Florence and New Jersey seasons, Vinny decided to pack his bags and head home to Staten Island for a breather.

"We just came from Italy, and you know we all hit a wall at some point," Vinny explained. "We all kind of reached our breaking point in some way because it was so hard, the schedule, you will see my personal problems happen in the first couple of episodes. I'm just the type of person, when I'm feeling like that, I need certain things."

"Jersey Shore" fans know that Pauly D and Vinny share a special bond, so how did his bro handle the Staten Islander's exit? "We almost had a blowup when he left," Pauly revealed. "You just expect him to be there; you don't realize what you got till it's gone, I'll tell you that."

Mike "The Situation" added, "I know [Pauly] was genuinely upset. We were all upset, but Pauly took it to heart, I know that for a fact."

It wasn't just Pauly, though. All of the housemates were affected by Vinny's departure. "It hit home for a lot of us because we all had to kind of face our emotions, because Vinny's the person that's always our voice of reasoning and [now he] is saying, 'I can't take it anymore,' " JWoww explained. "This is a family, when you have ... a brother or sister leave, it's heartbreaking, it's sad. You really do feel like something is missing in the house."

For continuing "Jersey Shore" coverage, be sure to check in with MTV's Remote Control blog.