"He cannot be on television because the firm is very private," a source told The Post.
In response, Colodne -- who served as Patriarch Partners LLC's president for two years until January 22 -- filed a lawsuit last month in Manhattan federal court against his former employer for $55 million for allegedly canning him without cause, The Post reported.
Colodne reportedly learned of his ousting when a Patriarch lawyer was "horrified" to learn he appeared on the show and sent him a letter of termination.
"I'm suggesting that one of the reasons for his absences in the office was because he was filming a tawdry show," Hillary Richard, a lawyer representing the company, told The Post. "They found out he was on the show when promos ran. He's on a show and never mentioned it. Any other company would have fired him on the spot. He also was not performing or bringing in new deal flow."
Colodne appeared on The Real Housewives of New York City due to his romantic relationship with Frankel, The Apprentice: Martha Stewartrunner-up who was cast for the Bravo spin-off despite the fact that she's not married and has no children.
Colodne appeared most prominently in a The Real Housewives episode that promoted a charity that raises money for neurofibermatosis, which his sister has, The Post reported.
"His appearance as an incidental on the show does not violate any of the terms of his employment contract, and Patriarch is using that appearance in an attempt to avoid paying him tens of millions of dollars in hard-earned compensation," his lawyer Robert Kraus told The Post.
In addition, Avery -- the 13-year-old daughter of Frankel'sThe Real Housewives of New York City co-star Ramona Singer -- said the word "lezzies" on an episode of the show, according to The Post. She has since been "spoken to" about it by the headmaster at her Upper East Side all-girls school.
"They were so shocked at how much publicity the show got that they wanted to make sure Avery could handle the spotlight," an insider told The Post.