"All four are gone," a source told The Wrap. "They feel they've lost their core audience and they want it back."
Fox reportedly has expensive contract options with all four judges but is choosing not to pick them up.
"No more big checks for divas," added the website's source, referencing Carey's reported $18 million package, Minaj's $12 million salary and former judge Jennifer Lopez's $15 million.
In addition, executive producer Nigel Lythgoe -- who only returned to American Idol this year after leaving in 2008 -- will also reportedly be replaced, and despite recent rumors, Lopez is not being considered to return to the judging panel.
The reality singing competition has experienced a steady decline in viewership since it peaked in Season 6. Idol's ratings are reportedly down 22% percent this season and hit an all-time low last Thursday when only 11 million viewers tuned in, according to The Wrap.
Fox Broadcasting chairman Kevin Reilly has reportedly been put in charge of "fixing Idol."
The alleged shakeup in judges and production is allegedly an attempt to boost ratings by attracting a younger audience. Idol's format will also reportedly change, as they'll do away with dated tribute nights.
A Fox spokesperson and each of the current judges' representatives declined comment on the matter.