While some may have expected the two to break out into one of John's classic tracks, they stuck to "The A Team," a dark ballad that describes a woman's battle with addiction. Sheeran kicked it off on guitar and was joined by John behind the piano, wearing a black jacket and bright blue shirt with matching blue tinted glasses.
Though it ultimately lost to "We Are Young," Sheerean's track was nominated for Song of the Year.
This is hardly the first time that John has dueted on the Grammys stage: He also teamed up with Eminem in 2001 and Lady Gaga in 2010. However, when it came to performing with Sheeran, it was entirely the Rocket Man's idea.
"He put it together," Sheeran recently told MTV News. "He rung me up one day and said, 'How would you feel about performing with me at the Grammys?' I was like, 'I'll check my schedule.' "
Even though the two only rehearsed once, Sheeran knew that this Grammys performance was key for his career.
"The thing is, when I've done it in England, they are my peers, I know them ... but I'm performing in front of people that have never even heard of me," Sheeran said, "So I've got to really make an impression. It's gonna be fun."