"It would be best if they imposed economic sanctions on Turkey", demonstrator Sinan Anin said.
Tensions are already running high between Berlin and Ankara after German authorities refused to allow some Turkish ministers to campaign in the country for a "yes" vote in the April 16 referendum that would hand Erdogan an executive presidency. Turkish authorities have blamed the attempted coup that left 248 people dead on the Fethullahist Terrorist Organization (FETÖ), whose leader is Gülen.
Significantly more people turned up than organisers had been expecting for the rally, which took place ahead of the annual Newroz festival when Kurds mark the traditional New Year.
Many demonstrators carried symbols of the outlawed Kurdistan Workers Party (PKK) which has battled the Turkish state for over three decades in a continuing insurgency.
About 30,000 pro-Kurdish protesters gathered in the central city of Frankfurt on March 18 to protest against the referendum sought by Erdogan, which would give the president greater power.
Turkey's foreign ministry expressed its displeasure to Germany's ambassador in Ankara, according to a statement by the ministry.
The protest brought swift condemnation from Turkey, which said Germany was allowing open support for terrorism.
Germany and Turkey have been locked in a deepening row after Berlin banned some Turkish ministers from speaking to rallies of expatriate Turks ahead of the referendum next month, citing public safety concerns.
"Turkey has tried to convince us of that at every level, but so far it has not succeeded", Bruno Kahl was quoted as saying.
The spokesperson also condemned some European countries for trying to "interfere the referendum on April 16 in Turkey and take up a position in favor of "No" vote".
Ankara blames Gulen's network of followers in the military for a failed coup attempt in July, when a group of rogue soldiers seized tanks, helicopters and war planes to attack parliament and attempt to overthrow the government.
Gulen, a former Erdogan ally who has lived in self-imposed exile in the United States since 1999, has denied the charges and condemned the coup.