Drew Lachey and Cheryl Burke feel Dancing with the Stars' viewers shouldn't believe everything they read -- especially when it comes to the recent report that the second-season champs had an affair during the show's recently concluded tour.

"It's nonsense. Anybody that knows me -- and those are the people I care about -- they all know that it's nonsense," Lachey told OK! Weekly at Hollywood's William S. Paley Television Festival.  "I'm trying to keep it in perspective. As long as my wife knows that it's nonsense, I could really care less. Life is too short to worry about what other people think about you. I know in my heart what happened."

According to Star Magazine's March 31 issue, a Dancing with the Stars - The Tour "insider" claimed Lachey and Burke had an affair while on the road this past winter.

"Drew and I have been friends for a couple of years, same with the whole Lachey family and it's just completely absurd what they're saying about the whole thing," Burke told OK!.  "It's a shame that people have to sell magazines by selling stories that are not true."

Despite discrediting the report, Lachey said it's still been difficult to deal with since he has a 2-year-old daughter with his wife Lea, whom he's been married to since 2000.

"I'd be lying if I didn't say that it hurts and it affects me," he told OK!.

Burke -- who is currently paired with telenovela actor and ABC-billed "Latin American heartthrob" Christian de la Fuente for Dancing with the Stars sixth season -- said she is simply trying to "ignore" the affair rumor.

"I'm busy," she told OK!.  "I'm working, doing something I love to do. I don't have a lot of time on my hands to sit home and feel sorry for myself."

Lachey -- who recently served as host for ABC's Dance War: Bruno vs. Carrie Ann reality competition series -- said he's also trying to keep busy, which included celebrating his daughter's birthday over the weekend. 

In addition, he's also focusing on his career in an attempt to remove himself from the spotlight.

"I'm starting a production company, trying to move behind the scenes as well because you never see a producer being talked about, do you?" he told OK!.  "I'm on the wrong side of things."