The Hong Kong originally formed in New Orleans, but became more established after regrouping in Brooklyn. They are known for mixing a 1960s feel with modern indie rock sensibilities. They have also been compared to Blondie. In 2000, the band released their first album, the instrumental Lights At Night, which included a post-rock sound that would become much less evident in the group's future efforts. 2003 saw the release of Rock The Faces, which Allmusic said "is junk-shop pop at its finest. Built out of parts of various great bands and artists, it shines like a brand new artifact" (link).
In January 2005, The Hong Kong went into the studio to record with Ric Ocasek of The Cars producing. Soon after, they signed to his label, Inverse Records. That Ocasek took the band under his wing was evidenced by their backing him up at live shows, such as a September 2005 performance at CBGB.
In 2006, the band released a video for a song called "Tongue-Tied." The video featured Culpepper with Dean Wareham (Luna / Galaxie 500) as Bonnie Parker and Clyde Barrow.