It is a platitude to say that great art always springs from unhappiness, but there may be more than a grain of truth in the cliché when it comes to Gwen and Augustus John. The brother and sister had a deeply unhappy past, which undoubtedly shaped not only their personalities, but their relationship.

They were born within two years of each other – Gwen in 1876, Augustus in 1878. Within six years of Gwen’s birth, Augusta, the kind, pretty mother who had introduced them to art and drawing, was dead. They and their brother and sister were then brought up by a mixture of religiously zealous aunts (including one whose first act was to dismiss the children’s nurse because she felt they were too fond of her) and their bitter, uptight solicitor father, Edwin, whose own father had died only four months before Augusta.

Who would the two middle children look to as the only kindly and stable parental influences in their lives? Each other, it seems. They were lost babes in the wood. Apparently, when Augustus read Hans Christian Andersen’s Snow Queen, he was so upset he had to retire to bed. The story tells of two children, Gerda and Little Kay. The boy is captured by the Snow Queen and his heart turns to ice. Gerda eventually finds him and the warmth of her tears melts the ice. It is irresistible not to imagine that Augustus identified with Little Kay – with his sister playing Gerda.

After their mother died, the family lived in a miserable house in Haverfordwest, south Wales. ‘I felt at last that I was living in a kind of mortuary where everything was dead,’ wrote Augustus later. ‘From the stuffed doves in their glass domes, fleshless as the abominable skeleton clock on the mantelpiece. it all reflected the frozen immobility of its curator’s [their father’s] mind.’ The whole family was tormented by “gloom by day and horror at night’. He described his father as a ‘revolting personage’, and when Edwin eventually died, Gwen didn’t bother to go to his funeral.

The four acutely shy children were left to their own devices in the Welsh countryside. Painting and drawing, with all their memories of their artistic mother, bound Gwen and Augustus together. Augustus wrote that on the beach at Tenby: ‘Gwen and I, full of curiosity, would approach as near as we dared, to watch the mystery of painting. Even at that early age we were vaguely aware of Art and Beauty.’

After their time together at the Slade, living for a while in chaotic rooms in Fitzrovia (‘There were none I loved more than Augustus and Gwen John,’ wrote John Rothenstein, the former director of Tate, ‘but they could scarcely be called “comfortable’ friends” ’) separation was inevitable. An emotionally incestuous relationship is too dangerous to flourish, and Gwen gradually set up residence in Paris, where she lived for most of the rest of her life.

‘I think if we are to do beautiful pictures, we ought to be free from family conventions and ties,’ wrote Gwen. She felt intruded upon if her brother ever remarked about the way she worked. Yet both suffered from a lingering sense of unrequited love – an emotional state common among those who have experienced a loss in childhood. They constantly searched for the perfect parent substitute. In Augustus’s case, he became a compulsive womaniser, sweeping women off their feet with his satyr-like sexuality (girls are said to have fainted when they saw him in the Café Royal). He bedded them and dropped them in his desperate bid to find something that would stave off his frequent bouts of despair. The most urgent of these quests was the pursuit of the beautiful Dorothy McNeill, who he felt could make his life whole at last, and who he renamed Dorelia. Even though he was married, he longed to possess her. When she escaped his clutches, it was Gwen who helped to entice her back with a fantastic and seductive letter which made out that her and Augustus’s love was something transcendental, unlike other loves. It was essential, she said, that Dorelia return to the John fold. With her own affection for Dorelia – they even spent time travelling to Rome together – it seems as if Gwen, for a moment, suspected that Dorelia’s love might heal her emotional wounds too. Certainly, all through her life she had major crushes on women as well as men, none of which materialised into anything realistic.

While Augustus saw Dorelia as his saviour, Gwen developed a self-destructive obsession with the sculptor Rodin, 36 years her senior. Interestingly, Augustus was almost as complicit in this affair as Gwen was in his with Dorelia, for it was he who suggested that she offer herself to the sculptor as a model in the first place. Rodin, hardly surprisingly, was not unlike Augustus – a giant, womanising, bearded charmer. Gwen became so besotted with him that she gave up painting almost entirely for two years, wrote him up to three letters a day and hung around his studio. She looked to him as a father. ‘If you chide me,’ she wrote, ‘do it gently as to a child.’ After Rodin died, she turned to God and converted to Catholicism to assuage her longings. Here was someone who did not fall ill, grow old, let her down, or die. ‘He loves me,’ she wrote.

As Augustus wrote later: ‘Gwen and I were not opposites but much the same really, but we took a different attitude.’ Their different attitude shows nowhere more clearly than in their painting. While Augustus’s works are romantic, full of promise, joyful pictures of families, gypsies, people smiling, open and exuberant – perhaps fantasies of what he longed for life to be –  Gwen’s are restrained and tortured. Her models, always women, are haunted, pleading creatures, bleached of life and colour, sitting apparently full of repressed longing and misery, hands invisibly shackled in their laps. Even the interiors reek of loneliness – the single cup, a single chair, a lonely cat, one doll on a bed.

Despite these differences, Augustus loved and understood his sister’s work, perhaps because he could see in it the repressed longing that he himself managed to resolve in his pictures. Rothenstein remembers seeing him ‘peer fixedly, almost obsessively, at pictures by Gwen as though he could discern in them his own temperament in reverse’. Augustus was angry that at one of her exhibitions the critics overlooked her pictures, ‘two rare blossoms from the most delicate of trees’. He wrote: ‘To me the little pictures are almost painfully charged with feeling, even as their neighbours are empty of it. Gwen’s pictures are simply staggering.’ He even predicted that in 50 years he would be known as the brother of Gwen John, rather than vice versa.

In middle age Gwen realised she had to be alone. Childless, she became almost hermit-like. This is not unusual for someone as sensitive as Gwen. Such a person finds it almost intolerable to be with other people because they appear always to be saying or doing things to hurt. ‘Gwen would change expressions so many times that it was easy to read in her face indications that one had quite unintentionally offended her, and in ways it would have been impossible to offend anyone else,’ Augustus wrote. And he was much the same – ‘He is offended by everything I do or don’t do,’ his sister wrote to a friend.

When a priest she had known committed suicide, Gwen agonised that she might have been able to prevent it. When her cat, Edgar Quinet, disappeared, she spent days camping out in the woods like a gypsy hoping to find him. She never killed the insects she found in her house, but collected them in a box and put them out in the street. Although it might be thought that Augustus, rumbustious, sensual and the father of nine children, would shun solitude, he agreed with his sister that ‘loneliness is a great thing. Decidedly, it is inspiring to lie alone at times. I fear continued cosiness is risky.’ For both Gwen and Augustus, big, romantic, all-enveloping and unrealistic love – the kind a baby has for its mother – was fine, but ‘cosiness’ or the normal close affection between men and women was seen as dangerous.

In the end it was Augustus who provided much more of the parenting role for Gwen than she for him. When she was in London, he described her as ‘the waif of Pimlico’. He disapproved of where she lived – ‘a kind of dungeon’ – and tried to include her in social events. When she was in Paris, his letters to her read like the solicitations of a mother. He wanted her to come back to England during the war. ‘Let me know what you decide and if I can help in any way. With love, Gus,’ he wrote. On another occasion: ‘Don’t let yourself get frozen, dearest. Take exercises. Love from Gus.’ He sent her Sanatogen tonic wine and a Jaeger blanket. ‘Gwen utterly neglects herself for some bloody mystical reason,’ he said, with all the tuttuttings of a mother hen. When Rodin died, he went to Paris to comfort her. ‘Do not for God’s sake allow it to make you morbid,’ he wrote when another friend died.

Augustus introduced her to John Quinn, the American art collector, and he often sent small sums of money. In her later years, he even made all the arrangements and loaned her the money for the purchase of a cottage in England – but she stayed there only once. She found it difficult to accept his help. And surely her almost aggressive independence showed how hard she was fighting to control her frighteningly urgent inner dependence.

When his sister died in 1939, Augustus said to his son, Edwin: ‘I don’t fancy strangers writing about her somehow.’ She might have been odd and difficult, but he clearly loved her and thought he understood her better than anyone else.

There remained about both of them a strand of unhappiness throughout their lives. ‘In talking, shyness and timidity distort the very meaning of my words,’ wrote Gwen. ‘I don’t pretend to know anybody well. People are like shadows to me and I am like a shadow.’ She told a friend: ‘When I was a child, I used to cry all the time.’ Another friend described her as: ‘Savagely proud, yet childish, even affectionate, wanting love, yet refusing it.’ She once told Jeanne Robert Foster that though she wanted to die, she didn’t want to commit suicide.

Her sense of self was fragile, and it showed when she arranged sitters to pose for her. She insisted they sat in positions that she herself would normally take. Foster wrote: ‘She has me sit as she does and I feel the absorption of her personality as I sit. She is more myself than I am when I am with her.’ This lack of boundaries is commonly experienced by children who have had very little proper parenting.

Augustus, who died in 1961, never really found what he was looking for. Don Juans never do. ‘I am in a curious state,’ he once wrote to a friend, ‘wondering who I am. I watch myself closely without yet being able to classify myself. I evade definition – and that must mean I have no character. Do you understand yours?’

They were both, in their own ways, lost and lonely orphans. Augustus, the slightly more secure, gave Gwen as much support and comfort as he could, but they could never find in each other, being brother and sister, what they really needed. Nor could they find it in their relationships. The only truly successful way they related to the world was, in the end, through their art.

David Fraser Jenkins on Augustus John’s Dorelia Standing Before a Fence c.1903–4 Purchased by Tate Members and presented to Tate in 2003

Augustus John was a fantasist, and the strength of his art was in the power of his imagination. His infatuation for the woman in this portrait, Dorothy McNeill, transformed her into an imaginary gypsy with a new name: ‘Dorelia’. When they first met he wrote her love letters in Romany, even though she couldn’t understand the language and he had to send her a word list by the next post. His admiration for the gypsies was profound and genuine. In his day there were bands of them wandering across Europe, speaking different dialects and forming part of a totally distinct culture. They appealed to Augustus as exotic, preserving an elusive but authentic vitality. He stayed with them in Wales and France, learned how to speak to them in their own words, and was proud of being accepted by them. He instinctively took their side when they were threatened by the modern life which he hated, and he longed himself for a nomadic, out-of-doors existence.

His vision of his own situation may have been far from the whole truth, but it inspired his art, and to our knowledge now, can only be true to his feelings. In fact when he produced this painting his family was living in a ménage à trois, with both his wife Ida Nettleship and his mistress Dorelia bearing his children. They all lived for a time in a caravan and a tent on Dartmoor, at other times in a country house in Essex and for several years in Paris, with Augustus departing often to his London studio, or on commissions to paint portraits. These commissions provided enough money to keep them together, though occasionally the women moved out or banded against him, and they had always to look after an increasing number of babies. But they knew that his art depended on their being able to act the characters he needed. Both women designed and made their own clothes, and sought out costumes in second-hand shops that might appeal to him.

The long skirt and jacket that Dorelia probably made herself for this portrait transforms her into a full-length version of the Mona Lisa, standing poised with a baffling expression in front of an impossible and remote landscape. The painting has always been called Dorelia Standing Before a Fence, and this odd sounding title is exactly right. The old-fashioned wording takes the fence away from any kind of common garden-divider to become a feature on a hilltop, sloping at both sides, just on the boundary between here and beyond. It is a fantasy portrait of a woman removed from the modern world, on the threshold of what may be an entrance, as nearly open as is her jacket, held together only by the last one of a long row of buttons.

It was the reputation of this kind of picture by John that appealed to the young Picasso, who was taken to Augustus’s studio in Paris in 1907, probably in August. On the return visit, John saw Picasso’s circus and gypsy paintings, as well as the unfinished Demoiselles d’Avignon. The experience encouraged his taking further these imaginative figures into life-scale mural paintings of women and children. Some were of his family, and were evidence of his nostalgic wish to re-create his gypsy kingdom after Ida died in childbirth. Some were bought by his American patron John Quinn, who loaned his Family Group 1908 to become a star of the famous Armory Show of 1913, the first exhibition of modern art in America.