There’s a common greeting in the Outer Hebrides: the lineage-establishing “Who do you belong to?” By the time this question is posed to 22-year-old gay Harris islander John-Calum Macleod, or Cal, in Douglas Stuart’s new novel, there is a sense that Cal is his father John’s beyond the ordinary claims of blood – the latter’s sway containing undercurrents of domineering ownership.

The book opens with the two conducting a strange ritual over the phone, performed regularly ever since Cal moved to Edinburgh to study textiles: John, a precentor, reads to Cal in Gaelic from the New Testament and has him sing back “with the full power of his belief”. The verse John recites – which prefigures the novel’s themes of repression and self-denial – urges the faithful to guide the errant and to stay vigilant against temptation. After receiving Cal’s assent, John orders him to return home, ostensibly because Cal’s maternal grandmother, Ella, is sick. Though John lives with Ella in her croft house, she is his ex-wife’s mother and thus not his responsibility.

Set within a tight-knit Free Presbyterian community of farmers, weavers and fishers in what appears to be the 1990s, John of John tells the story of Cal’s uneasy homecoming. It’s a reprise of the parable of the prodigal son and an ardent exploration of the half-lives of queer men condemned to love, pine and suffer in silence. Intimate yet epic in scale, it contains equal parts pastoral drama, tale of familial fracture, love story and inquiry into various forms of loneliness: the loneliness that can reside between fathers and sons, between lovers, between man and God, and between a small place and the big world.

John disapproves of Cal’s appearance, his sartorial choices and his long, “flame-coloured” hair, disturbed “by the confused signal they were sending, the strange tension between the masculine and the feminine”. Cal’s disinclination to be “saved” creates a rift between them that later erupts in violence. Meanwhile, childhood friend and hookup partner Doll gives Cal the brush-off, cross that he’s been away for so long. Wearied by his ultraconservative environment, where connection feels out of reach, Cal takes a fancy to his dad’s sole friend, confirmed bachelor Innes MacInnes. Cal is struck by Innes’s “gentleness, his benevolence – which Cal had never appreciated before, which, if he were honest, he would have said he found boring, unsexy in younger men”.

This, however, can never be the merry May-December romance Cal wishes it to be. Innes and John are lovers, we learn fairly early on, and it is this pair’s tortured relationship since their teenage years – kept secret from everyone, including Cal – that forms the novel’s centre of gravity. Masters of discretion, John and Innes are, to townsfolk, neighbouring sheep farmers. The first time we see them alone together, at Innes’s, they go through the motions of a long-established routine, allowing themselves to draw close only after John has made sure each room is empty and they are really alone. Later, as John prepares to leave, Innes loudly seeks his assistance over an unspecified “two-man job”, “all in case someone should find out and ask what exactly John Macleod was doing upstairs in the MacInnes house at such an ungodly hour”.

The novel tries their bond in ways small and big. Aside from the difficulty of Cal, there is the matter of John’s other liaison with a married man, and the tenancy of Ella’s house soon to be transferred to Cal’s mother. Innes floats the idea of John moving in with him but intuits “how, even under the threat of homelessness, a life together with him seemed no consolation at all”. John is a man tormented by the idea of his own depravity: “He loved God. He loved Innes. He loved God and God hated how he loved Innes.” At one point he entertains the possibility of Innes, Cal and himself being a family, but even in fantasy, the thought of Cal being gay, like him, remains unimaginable: “They would live like this every day, be useful, peaceful, happy on their land, looking forward to the day Cal married a local girl and filled their croft with grandchildren.”

The novel is outstandingly canny and wrenching on self-contempt, on the toilsome art of deceit, and on the contradictions we all contain, as well as the friction that can exist between the personal and the collective. As secular values gain ground, there is the suggestion that John and Innes living together could deal a death blow to their local congregation, leaving us wondering whether John and Cal will – or can – come out to one another. Amid all this, Stuart finds the space to touch on crofter subservience to absentee landowners, the scorn and prejudice of mainlanders, and the place of the Western Isles within the English imagination.

John of John is certainly enthralling, but the ambient Weltschmerz and the characters’ frequent self-pity can be draining. Stuart’s first two novels, the Booker-winning Shuggie Bain and its follow-up, Young Mungo, were feats of heartfelt, operatic storytelling, composed as though in defiant response to our age of irony and subtlety. Despite their occasionally miserabilist tenor, the emotions felt guileless and real, whether Shuggie’s love for his doomed, alcoholic mother, Agnes; Jodie’s for her brother Mungo; Mungo’s for his birdkeeping neighbour James or his own doomed, alcoholic mother, Maureen. The impoverished Glaswegian milieus where they were set – marked by Thatcherite ruination, homophobia, sexual predation and sectarian strife – made for sobering reading; but these were novels so lavishly and graciously imagined, so very moving, that you gladly faced up to their gloom.

Here Stuart leans heavily on melodrama and sensationalism as a shortcut to tragedy. Towards the end, the novel is eventful to a fault and surfeited with pathos: we have a pregnancy; an attempted shotgun wedding (“What in the world of Thomas Hardy?” says Cal); a death and a momentous departure from the island. While this book will not appeal to those with a low tolerance for excess, diehard romantics will find much to love; I see Cal, John and Innes – knottily entangled and imperfectly endearing – being cherished with readerly devotion. And that is no small feat.