"X Factor" boss Simon Cowell sat down with Ellen DeGeneres on her talk show on Tuesday (September 10) to talk about lots of things, including opening up about his impending fatherhood and why his experiment with Britney Spears on "Factor" was a bust.

You might recall the hype that surrounded the addition of Spears to the "Factor" panel last year.

"There was a time where I think everybody was trying to up themselves in terms of who could find the biggest known star to be on a panel," Cowell said, naming "American Idol" judge Jennifer Lopez and the "other one ... blondie" on "The Voice," in reference to Christina Aguilera.

"And so we thought, 'well, we'll take Britney. And she was fine, but I think she found it very uncomfortable," he said, perhaps explaining Spears' GIF-worthy series of odd faces, bland platitudes and lack of engagement during her single-season run.

"She's not a big talker," DeGeneres offered. "I didn't realize that," Cowell admitted with a laugh. "I kind of booked someone who couldn't talk, which is a bit of a problem when you want someone to judge." Cowell further explained that Spears was also not comfortable giving negative critiques and was even less at ease with sassy contestants who talked back to her.

"She was just uncomfortable ... I liked her ... so we decided to change it this year and I thought, 'I'll work with three girls,' because I like working with girls," Cowell said, by way of explaining why he filled Spears' and departed judge L.A. Reid's spots with Kelly Rowland and Paulina Rubio.

After remaining mostly silent about Spears' tenure, Cowell began speaking out about the failed experiment [article id="1711714"]last month[/article] at the Television Critics Association panel, saying it was a flop from day one. "It's like having a dinner party. You invite people for dinner and sometimes it's a fun night, and other times it's not as fun as you hoped it would be," he said. This season, however, he promised a much more fun dinner party.

Cowell also opened up about his impending fatherhood. "Oh my God this is exciting," Simon said about getting the first call with the news. He realized, of course, that the next few weeks would be a bit of a nightmare due to the media scrutiny about him getting his friend's wife, Lauren Silverman, pregnant.

"It was kind of uncomfortable because of the situation ... these things happen," he said. But, Cowell, 53, is ecstatic about his upcoming first-time fatherhood. "I'm actually happier than I've ever been."