"It's hard for me to talk about a little bit because I still really love her and care about her," said James on Nightline.
James and Bullock's marriage first hit the rocks in March when James was accused of carrying out an 11-month affair with tattoo model Michelle "Bombshell" McGee while Bullock was in Atlanta filming The Blind Side -- for which she recently won the Oscar for Best Actress and praised James for his support.
"In a million years, the last person I would ever want to hurt or harm is her or the kids. I spent my whole life with the kids and with her trying to protect her, protect her from everything," explained James on Nightline.
"The most hurt she's ever been caused is by the person that wanted to protect her. It's terrible and she's going to be hurt for a long time."
James' Nightline interview will air Tuesday at 11:35PM ET/PT on ABC.