NEW YORK -- Ed Sheeran spent much of this year opening for Taylor Swift on her RedTour, and on Friday night, he saw her return the favor!

Dressed in similar tees -- with Taylor wearing a red "I heart Ed" shirt and Ed in an "I heart NY" T-shirt -- the two hit the stage at Madison Square Garden with their guitars (Taylor had her bedazzled red microphone) during the second night of his three-date gig at the famed arena.

Sheeran told the crowd he owed a lot to the "I Knew You Were Trouble" singer, before he and Swift gave a moving performance of their hit single, "Everything Has Changed." Maybe it was the shock of her surprise appearance, but the screams from fans were deafening.

But even with a cameo from Swift and assistance from opening act Tori Kelly, there was no question that this was Sheeran's show -- and he proved why it's a must-see. He kicked off a New York City takeover of sorts with his first show on Tuesday at MSG, where he also performed a new track.

Then for Halloween on Thursday, Sheeran dressed as a Gingerbread Man and returned to the Mercury Lounge, the place where he had his first New York gig 18 months ago. He treated the intimate crowd to fan favorite tracks as well as three new songs, including "Forever."

So by the time we got to the Garden on Friday, we could be forgiven for wondering if Sheeran might be a little worn down. We had nothing to worry about though: The British singer mesmerized with hits like "Give Me Love," "Lego House" and the "A Team."

And his command of the crowd was equally as impressive. During the opening song, "Give Me Love," Sheeran told the audience, "Sing the words if you knew them or make them up if you don't," dividing fans into three sections and turning them into his back-up singers as they harmonized.

When Sheeran wanted the attention to be solely on the music, he instructed his Sheerios to participate in a "universal shh," silencing the sold-out crowd, who, surprisingly went with it. Twenty thousand fans stood in silence (and awe) as Sheeran sang his slower-tempo tracks and treated them to a brand-new song. Fittingly titled "New York," it's about a night with a friend that "turns into morning," and presumably off his upcoming sophomore album.

Apart from his concerts, the busy "Lego House" star found time to shoot the video for "Old School Love" with Lupe Fiasco. Sheeran also spent some time "hanging with his friends," the Muppets, as he filmed a guest spot on "Sesame Street," which will air in September 2014.

1st date of MSG is today, then two more, then the two and a half year tour for this album is done. what a way to end it

— Ed Sheeran (@edsheeran) October 29, 2013

Sheeran concludes his run at Madison Square Garden on November 7, which will mark the end of two and a half years on the road promoting his debut album.