Since taking office in 2016, the controversial Filipino leader has presided over a bloody so-called war on drugs in which more than 7,000 people have been extrajudicially killed by police or vigilantes.
The protesters say they are concerned the leaders are not giving human rights enough attention and will not be addressing poverty in the region as they would like to see it.
Another 2,290 people have been murdered in drug-related crimes, while thousands of other deaths remain unsolved, according to government data.
But rights groups warn he may be orchestrating a crime against humanity.
Domestic opponents have appealed to the International Criminal Court to investigate, pointing to the jailing of opponents, a compliant congress and intimidated judiciary as reasons to step in.
He said what took place was a "rather lengthy discussion" of the Philippines' war on drugs "with President Duterte doing most of the explaining".
The Davao-based solon also scoffed at Trump's racist anti-migrant policy which is targeting not just millions of Filipinos in the U.S. but millions more of migrants from other countries.
Abe, whom Duterte has called a "true friend", has not criticised the drug war.
This even as a White House official said the matter was "briefly" tackled during the meeting between the two leaders.
"Aside from his horrendous military and economic policies, President Trump is also a patent sexist that he continues to spew anti-women remarks and attacks women's rights", he said.
The premiers of China and Russian Federation, two other important Duterte backers, will also be in Manila.