A far-right Catholic sect’s plan to ordain its own bishops on the first day of July has placed it on a collision course with the Vatican – posing a possible crisis for Pope Leo a little over a year into his papacy, and straining the Roman Catholic church’s already fraught relationship with rightwing and traditionalist Catholics in the US and elsewhere.

Founded in Switzerland in 1970 to oppose liberalizing reforms in the Catholic church, the Society of St Pius X (SSPX) has gained significant followings in the US, France, Argentina and other countries. The order, which has a large base of operations in Kansas, claims that more than half a million people worldwide attend its masses, though these numbers are difficult to verify. It counts nearly 1,500 priests, seminarians and other vocational members among its members.

Pope Leo told journalists in Rome last week that he was “considering making another appeal to say: ‘Do not do this, let us try to live in communion within the church.’” But it was the SSPX’s “choice”, he said, whether to continue on a trajectory that threatens schism.

“If they make that choice,” Leo added, “I am sorry, but we must move forward.”

Under Catholic canon law, ordaining bishops without the Vatican’s authorization is grounds for immediate excommunication. So far, both sides in the game of brinkmanship are refusing to blink. The Guardian contacted the Holy See and the SSPX for comment but neither responded.

The SSPX maintains that its planned ordinations of four new bishops – two French, one Swiss and one American – are made from practical necessity and “do not proceed from any desire to claim a power of jurisdiction or to establish a parallel authority within the Church”.

The relationship has seen decades of standoffs, stalled negotiations and failed attempts at reconciliation. The first and last time that the SSPX ordained bishops, in 1988, the Holy See excommunicated those who participated, including the SSPX’s founder.

In 2009, the conservative Pope Benedict agreed to lift those excommunications as a gesture of goodwill. He also granted greater permission for the use of the Latin mass, which traditionalist Catholics favor but has been largely replaced by vernacular liturgy.

Benedict’s more liberal successor, Pope Francis, abolished a commission set up three decades earlier to negotiate with the SSPX, though he also made the unusual decision to recognize the order’s sacraments as valid for the purposes of marriage and confession.

The SSPX exclusively practices the Latin mass. The order also advocates strict gender roles. Women are discouraged from wearing trousers, and often wear head coverings to church.

Yet the sect’s contentions with the Vatican are more fundamental, Massimo Faggioli, a professor of theology at Trinity College Dublin, said, and difficult to resolve or accommodate.

The SSPX rejects doctrines of reform, formulated during the 1962-1965 second Vatican council, that are core to the modern Catholic church. “It’s not something that you can solve by saying: ‘OK, you can celebrate mass in Latin,’” Faggioli said.

The second Vatican council promoted unity between Christian churches, acknowledged a universal freedom of religion, argued that the teachings of other world religions could “reflect a ray of truth”, condemned antisemitism and disavowed the notion that Jews bore collective responsibility for the death of Jesus Christ.

The SSPX believes that the council’s reforms were essentially heretical, Faggioli said, and has not given any sign that it will shift position. If the Vatican excommunicates the SSPX, he said, the big question is how conservative Catholics who are not in the order, but are sympathetic to some of its views, react to the schism.

The mounting tension between the Vatican and the SSPX comes as rightwing Catholics have shown an increasing willingness to tussle with the Vatican over political and theological disagreements. Some Catholics in the US, where the most influential lay members tend to be both conservative and wealthy, have supported the Trump administration even as its stances on immigration and foreign policy clash with those of the Vatican.

The founder of the SSPX, Marcel Lefebvre, was a French royalist who was fiercely opposed to communism, decolonization and secularism. Lefebvre was one of a small percentage of bishops who voted against key documents of the second Vatican council. He died in 1991.

The sect has been dogged throughout its history by accusations of antisemitism and ties to the extreme right.

The Nazi collaborator and convicted war criminal Paul Touvier was arrested at an SSPX priory in France in 1989. (The SSPX said it had taken him in as an act of charity.) In 2009, an SSPX bishop told the press that he believed that no more than 300,000 Jews were killed in the Holocaust. In 2013, the SSPX sparked outrage in Italy by officiating a funeral for a convicted Nazi war criminal, Erich Priebke, who had been denied burial by the Catholic diocese of Rome.

The SSPX has said that it “completely rejects the false claim that it teaches or practices antisemitism, which is a racial hatred of the Jewish people because of their ethnicity, culture or religious beliefs”.