President Rodrigo Duterte has said that his critics, particularly those from the "yellow" group, want him removed from his post but claims the communist rebels would not allow it.

In a speech on Tuesday night before the Filipino community in Cambodia where he was on a state visit, Duterte hit the "yellow" group for the demonstration.

"Only the yellow wants me removed because they cannot accept defeat," he said.

Duterte did not specify who were those who belong to the yellow group.

But he was apparently referring to the Liberal Party, the political party of former President Benigno Aquino III. The party's color is yellow.

Duterte expressed belief he could not be ousted since the leftist rebels would not allow it.

"But the left, they would kill you if you remove me because I am left. I'm a son of poor (parents). I'm the first left leaning president," he said.

The Duterte administration has been negotiating with the Maoist group to end the more than 45 years communist insurgency in the country. Celerina Monte/DMS