"It's that time of year [when] you're always going to get asked that question," Cowell told People following Thursday night's live results show on Fox. "And I'm not going to answer it."
Cowell seems to think there are pros and cons with having the young singer serve as a mentor and judge on the show alongside himself, L.A. Reid and Britney Spears -- with two of the pros being she's lively and feisty on the panel.
"She's fun," said Cowell, who often bickers with Lovato over artists' performances in the reality singing competition. "She really, really is a tough little thing. I know she plays the 'I've been bullied' [card, but] trust me, this one gives as good as she gets."
However, Lovato's tendency to speak her mind sometimes rubs Cowell the wrong way, especially when she has harshly criticized his The X Factor second-season groups Emblem 3 and Fifth Harmony.
"It did get a bit bitchy and unpleasant," Cowell told People of Lovato's most recent comments, which he strongly disagreed with. "I've got to look after my artists. Luckily, they didn't listen to her."