After starring on the show every season since it debuted in 2002, Abdul left Idol in 2009 after failing to negotiate a new contract and then Cowell quit the series the following year to pursue The X Factor. The third original judge, Randy Jackson, stuck around for many seasons afterward.
"Paula was the right leg and I was the left leg [of American Idol]. And with both legs gone, it wasn't the same show," Cowell told the New York Daily News with only the finale of Season 15 remaining until the franchise comes to an end.
"To be honest I think it changed more significantly once Paula left. At least for me, the show was never the same because she and I had such a connection and such a shorthand together."
At its peak in 2006, American Idol reportedly drew an average of 36 million viewers. The currently-airing season has averaged about 8 million.
"I haven't seen it at all, so I wouldn't have a clue as to what it looks like now," said Cowell, who joined NBC's America's Got Talent as a judge for its Summer 2016 season.
Despite his separation from the show, Cowell has remained close friends with Jackson and host Ryan Seacrest.
Cowell was known on American Idol for making snide -- and sometimes rude and offensive -- comments to singers auditioning. While many fans appreciated his honesty and sarcastic sense of humor, others found him to be "the mean judge," and that didn't sit well with Cowell.
"To be honest I found that to be embarrassing," Cowell told the Daily News of his reputation.
"In real auditions you can't tell someone who is terrible that they're good and promise them that they're going to become a star. When we did the show, I would behave as I did in real life. I didn't take it too seriously when they were bad so I found most situations quite funny."
However, Cowell's personality brought in the ratings for Idol, which prompted other shows to hire judges who would imitate his snarky attitude.
"Suddenly, as it works in the TV industry, there were these ridiculous people being hired all over the world to come up with sound bites -- half of them didn't even work in the music industry," Cowell explained. "It's sort of a compliment, but I found it excruciating."