Zack Polanski has called on politicians to treat antisemitism with “consideration, care and nuance” as he accused Keir Starmer of trying to play political games with the issue.

The Green leader’s comments come after the prime minister accused him of playing down recent antisemitic incidents. Polanski’s party is facing increasing scrutiny for recent comments by some candidates and members.

Some Green members and officials have expressed concern to the Guardian about what they say is the recent arrival of members who are vehemently anti-Israel to an extent that crosses into antisemitism, and the difficulties of countering this in a highly decentralised party.

Speaking on a visit to a synagogue, Starmer labelled Polanski “disgraceful” for saying it was important to distinguish between an actual threat to the Jewish community in the UK after recent arson attacks and the “perception of unsafety”.

Labour have also criticised Polanski for saying in another interview that he had been wrong to previously criticise Jeremy Corbyn for not properly dealing with antisemitism as Labour leader, and that the issue had been weaponised at the time.

Asked about the latter remarks on a campaign visit to Glasgow on Friday, Polanski – who leads the Greens in England and Wales but was helping the Scottish Greens’ campaign – said: “In terms of the weaponisation, it’s important to have absolute clarity that antisemitism definitely exists, and sometimes people will say things that are antisemitic.

“At the same time, I’ve seen reports in the press of things that people have supposedly said that are antisemitic that absolutely aren’t antisemitic. They are pro-Palestinian. And they are very, very different things.”

Polanski added: “When people create false allegations of antisemitism, or blur the lines between challenging the Israeli government and antisemitism, as a Jewish person, that makes me feel a lot less safe. That makes me feel like I’m being held accountable for a country that I don’t believe has anything to do with my Jewish identity.”

He accused Starmer of “playing silly games with a really serious issue”, adding: “It’s incumbent on all politicians, particularly the prime minister at this point, to treat antisemitism with the consideration and care and nuance that a really serious issue requires.”

He added: “We’ve got into a bizarre situation in this country where a non-Jewish prime minister is attacking the one Jewish leader on a case of antisemitism.”

On reports about comments by Green candidates, Polanski stressed the need for “perspective” over what he called a “handful” of accusations among 4,500 people standing for the party in local elections across England and Wales.

He added: “That being said, we know that there’s a worrying rise of antisemitism in this country as, by the way, there is a rise of Islamophobia and hate crime generally. And so it’s really important for anyone who’s in public life, and certainly a leader of political party, so they take that seriously.”

Polanski’s comments about Labour under Corbyn have prompted accusations of complacency in the Greens about antisemitism within the party. The number of members has almost quadrupled since he became leader in September.

Many of the accusations on around Gaza. Several Green candidates have posted comments that appear to sympathise with the Hamas attack on Israel, question the right of Israel to exist, or raise tropes about the supposed global influence of Israel and Jewish people more generally.

Some Green officials concede that the party’s highly decentralised structure can make it difficult to keep control of what members say. Vetting is still carried out by local parties, and there is at least one example of someone suspended as a general election candidate reappearing on the list for the elections on 7 May.

There is a similar lack of central control over policy motions submitted to the party’s conferences, including one put forward to the most recent event in March, which proposed that “Zionism is racism”.

One senior Green member said the sheer size of the party made its existing structures, based on direct member participation, open to abuse. “You can’t have direct democracy with 220,000 members,” they said. “The new members are a real mix, but some of them are very clearly single-issue Palestine activists who want to push an anti-Israel agenda to the exclusion of anything else.”

The party says that internal procedures are being beefed up, with a particular focus on antisemitism. “There are people meeting on a daily basis discussing this issue. It is not being ignored,” one official said. “But we have expanded so quickly, and we are fielding a record number of candidates, and so are catching up to an extent.”

The “Zionism is racism” motion was blocked at the March event, but is due to be debated at the conference in autumn.

Asked about the motion, Polanski said: “The way our members can vote on policy is a really important democratic principle. I also think the motion is a distraction [from what is going on in Gaza].”