Wallis Simpson is one of the most famous women in British royal history: Her romance with Edward VIII led to the abdication of 1936 and changed the line of succession. Not only was Britain spared the reign of a very questionable king, but it led to the accession of his niece, Elizabeth II, who is Britain’s longest reigning monarch.

And yet, despite her immense influence, Wallis was never queen. And like her, some of the most influential people in British history have been the monarch’s lover rather than their spouse. Here are 11 royal lovers who left their mark.

1. Piers Gaveston and Edward II

Piers Gaveston first met Edward in 1300 when he joined the Prince’s household. Both were about 16, and it was said that Edward “immediately felt such love for him that he entered into a covenant of constancy, and bound himself with him before all other mortals with a bond of indissoluble love, firmly drawn up and fastened with a knot.”

There is no firm evidence that their relationship was sexual, but Edward was certainly devoted to Piers — to the detriment of others. Gaveston was low-born, arrogant, prone to insulting the nobility, and was hoovering up titles and wealth at the expense of those who thought themselves more entitled. The writer of the “Vita Edwardi Secundi,” a 14th-century chronicle of the king’s reign, declared that “I do not remember to have heard that one man so loved another . . . our king was incapable of moderate favour.” Edward created the title of “Earl of Cornwall” for him and gave him extensive lands. Edward also arranged for his niece, Margaret de Clare, to marry Piers. The one person who Gaveston seemed to show respect to was Edward’s wife, Isabella of France.

Against Edward’s wishes, Gaveston was forced to leave England three times between 1307 and 1311, though he always came back. But by 1312 the nobility had had enough: Despite guarantees for his safety by the Earl of Pembroke, he was seized, subjected to a mock trial, and then executed on the orders of the Earl of Warwick.

Edward’s bond with his barons never recovered; he spent the next 10 years plotting his revenge. He soon found another favorite in Hugh Despenser, and the same pattern began to repeat itself. But Despenser overstepped the mark when he appropriated Isabella’s lands and took control of her four children. In retaliation, she led a rebellion that resulted in Edward’s death in 1327 and the succession of their son, Edward III.

2. Alice Perrers and Edward III

Alice Perrers was the widow of the king’s jeweler, Janyn Perrers, and one of the queen’s damsels when she met Edward III. The most likely date for the beginning of their relationship is 1364, when she would have been no older than 18 and the king 55. The birth of the first of their three children sometime occurred between 1364 and 1366.

There’s no record of Edward having a mistress before Perrers, and out of respect for his ailing wife, Phillippa of Hainault, the affair was initially kept low-key. Perrers became more prominent at court after Phillippa’s death in 1369. Over the next eight years, as the king’s health deteriorated, he showered her with gifts, gave her jewelry once belonging to the queen, made her his “Lady of the Sun” at a public tournament, and allowed her to accumulate enough land and wardships to make her the richest and most powerful woman in England. She was also an independent businesswoman, moneylender, and property owner, and although she remarried in 1375 (without the king’s knowledge), she retained her image as a self-reliant woman (a femme sole).

Perrers was not the only person seeking to use the aging king’s failing mental health for their own end, but her gender made her a target for the chroniclers of the time. The most famous was Thomas of Walsingham’s unreliable description of her as “a shameless, impudent harlot . . . [who] was not attractive or beautiful, but knew how to compensate for these defects with the seductiveness of her voice.” Her business acumen only served to antagonize the patriarchal hierarchy further, and the Good Parliament of 1376 resulted in her temporary banishment.

She soon returned and remained with the king until his death a year later in June 1377. Although Perrers was not responsible for many of the failings of Edward’s government at the end of his reign, the king’s reputation fell from one of respect and authority to someone whose mistress had “such a hold over him that he allowed important and weighty affairs of the realm to be decided on her advice.”

3. Katherine Swynford and John of Gaunt

Katherine Swynford met John of Gaunt, Duke of Lancaster and King Edward III’s third son, while she was a damoiselle in his wife Blanche’s household. 

In September 1371, John married his second wife, Constance of Castille, following Blanche’s death. It was a purely dynastic union that gave the duke a claim to the Castilian throne. Swynford’s husband died two months later, leaving her a widow with three children. Though their relationship seems to have remained platonic prior to her husband’s passing, by spring 1372, Swynford was openly acknowledged as John’s mistress.

Between 1373 and 1379, Swynford and John had four children, all given the surname Beaufort. By 1381, the duke’s reputation was at an all time low, and Swynford was targeted as “an abominable temptress.” John was forced to make a public denouncement of her and end the relationship, but this was a ruse. The two continued to meet in private.

Constance died in 1394, and two years later, amid a public scandal, John and Swynford were married at Lincoln Cathedral. Their children were legitimized by the Pope and, despite being barred from the line of succession by John’s eldest son from his first marriage, Henry IV, they would in fact go on to change history. Every English monarch since Edward IV (1461) and Scottish monarch since James II (1437) has been descended from Swynford. She is also the ancestor of numerous American Presidents, including George Washington, Thomas Jefferson, Theodore Roosevelt, and George W. Bush.

4. Anne Boleyn and Henry VIII

For the four years after her return from the French court in 1522, Anne Boleyn lived a dazzling if inconspicuous life as a lady-in-waiting to Katherine of Aragon. If she had married Henry Percy, the future Earl of Northumberland, as she had hoped, she would have been just a footnote in history. But in 1526, she caught the attention of Henry VIII.

Henry may never have set out to replace his wife with Boleyn. He had a history of infidelity and illegitimate children — including one with Anne’s sister, Mary. To further complicate matters, his Catholic faith prevented him from seeking a divorce. But Henry had only a daughter to succeed him and no male heir, and it was Boleyn’s good fortune that she was the woman who piqued his interest just when he came to the conclusion that he needed a new wife.

Henry’s pursuit of Boleyn had repercussions that would irrevocably change England’s religious identity. The Reformation did not happen because of Anne Boleyn — it was already a growing force — but her continual assertion that she would not be just another mistress fueled Henry’s desire for the annulment from Katherine, no matter the cost. Boleyn’s own support for the reformers also significantly advanced Protestantism’s progress in England. But their marriage only lasted a little over three years. At Henry’s connivance, Boleyn was charged with treason and executed on May 19, 1536.

5. James Hepburn, Earl of Bothwell and Mary I of Scotland

James Hepburn, Earl of Bothwell — commonly known as Lord Bothwell and described by the English Ambassador as a “[vain] glorious, rash, and hazardous young man” — first met Mary in 1560 when she was still Queen of France. Although he was a Protestant, he was a supporter of the Scotland’s Catholic regent, and in 1561 he was appointed to the privy council by the newly widowed Mary on her return to Scotland. Despite being described as having a “near sybbe [close friendship] unto her grace,” there is no evidence that they were lovers at this time — in fact, Mary was said to be besotted by Henry Stuart, Lord Darnley, who she married in July 1565.

There does seem to have been a change, however, by June 1566. The English diplomat Henry Killigrew wrote that, “Bothwell’s credit with the Queen is greater than all the rest together.” Mary’s son, James, had been born five days before, and although there is no question that he was Darnley’s son, her relationship with her husband had now completely broken down thanks to his involvement in the murder of her secretary David Riccio the previous March.

Mary’s relationship with Bothwell grew. When Darnley was found half-naked and smothered in the garden of his bombed house in 1567, both she and the earl were accused of arranging his murder. Abandoned by the Protestant nobles who had also been complicit, Mary continued to support her lover and sat on the sidelines as he was prosecuted and acquitted for Darnley’s murder. It seems likely that she knew of his plan to abduct her on April 24, 1567, although perhaps not of the violent assault that followed. With her position compromised, and no one left to support her, she married Bothwell on May 15, 1567.

After being forced to abdicate because of the scandal, Mary fled to England where she was executed on February 8, 1587, for plotting to murder Elizabeth I.

6. Robert Dudley, Earl of Leicester and Elizabeth I

Although Elizabeth may have known Robert Dudley as a child, and may have even had contact with him during her imprisonment in the Tower of London, any relationship that existed between them probably didn’t start until sometime shortly before her accession in 1558. By then he was already firmly entrenched as one of her most intimate advisors, and within a year she had become so emotionally reliant on him that the Spanish Ambassador noted that “they say she is in love with Lord Robert and never lets him leave her.”

Dudley already had a wife, who now prevented him from marrying the queen. He had wed Amy Robsart for love as a teenager in 1550. If Elizabeth ever intended to marry Dudley, Amy’s death under suspicious circumstances in 1560 ended any chance of that. Elizabeth was too savvy a politician to risk her throne as Mary I of Scotland had, and although Dudley would spend the next 18 years trying to get her to change her mind, Elizabeth never married him.

There is no evidence that they were ever physically lovers — the Spanish Ambassador recorded that Elizabeth herself swore that “as God was her witness nothing improper had ever passed between them,” and Robert had numerous sexual relationships with other women, including Lettice Devereux, who he married in 1578. But despite this, they were inseparable until Dudley’s death in 1588 and it’s said he remained her great love. Elizabeth was reportedly never happy when he was absent, and politician Sir Thomas Shirley told Dudley in 1586 that “you knowe the queen and her nature best of anny man.” She kept the last letter Dudley ever sent her in a casket by her bedside until she died in 1603.

7. George Villiers, Duke of Buckingham and James VI and I

James’s inclination for male company was well known when George Villiers engineered his way into the king’s presence in August 1614. James was immediately attracted to the man a bishop described as, “the handsomest-bodied in England; his limbs so well compacted and his conversation so pleasing and of so sweet a disposition.” But Villiers had to wait two years before the king’s current favorite, the Earl of Somerset, fell from grace.

Villiers immediately filled the void, seeming to confirm that they were lovers in a letter to James where he questioned, “whether you loved me now . . . better than at the time which I shall never forget at Farnham [in 1615], where the bed’s head could not be found between the master and his dog.” The aging James was besotted by the younger man, who he called his “sweete Steenie,” raising him through the ranks to become the Duke of Buckingham in 1623.

The danger was not necessarily that it was a gay affair, but rather that James had once again chosen a lover who was unequal to the job of being a key advisor. James’s blindness to Villiers’s corruption and incompetence put the country in danger. For his own ends, the new duke ensured the impeachment of the man trying to reform the king’s finances, nurtured the row between James and parliament to hide his own illegal dealings in Ireland, and called for a war with Spain to avert attention from his disastrous negotiations that almost resulted in the Prince of Wales becoming a hostage.

If James was ever aware of this, he forgave him. “I desire,” he wrote to Villiers, “to live in this world for your sake, and that I had rather live banished in any part of the earth with you than live a sorrowful widow’s life without you. And so God bless you, my sweet child and wife, and grant that ye may ever be a comfort to your dear dad and husband.”

8. Barbara Palmer, Duchess of Cleveland and Charles II

Barbara Palmer‘s father was the half-nephew of the Duke of Buckingham, but her family was impoverished when she married her husband, Roger, in 1659. She was already known for her scandalous lifestyle; within weeks of meeting King Charles II in 1660, she became his mistress.

Their first child was born in February 1661, and by December that year her husband had been made Baron Limerick and Earl of Castlemaine. Palmer had another four children with Charles, who declared in 1662 that “whosoever I finde to be my Lady Castlemaine’s enemy in this matter, I do promise upon my word, to be his enemy as long as I live.”

Her relationship with the king was never exclusive, but she remained his most important mistress between 1660 and 1672. The chronicler Samuel Pepys recorded that she “commands the King as much as ever, and hath and doth what she will.” She used this influence to her own advantage, selling access to the king and negotiating positions of high office for anyone who either paid her or was her lover. Worse still, she took bribes from the Spanish and French to intercede on their behalf, and she passed them information that would help them in their negotiations. Pepys believed that, “at the great ball she was much richer in jewells than the Queen and Duchess put together,” and an Italian diplomat recorded that “the prodigious amount of money dissipated by this woman, who has no moderation or limit in her desires, passes all bounds and exceeds all belief.” The number of portraits of her surpassed those of the queen.

In 1670 she was given a further three titles, one of which made her the duchess of Cleveland in her own right. But her relationship with Charles was on the decline. Gradually it became one of friendship and she went abroad, though they remained in contact regarding their children. The king spent an evening enjoying her company only a week before he died in 1685.

9. Lady Sarah Churchill, Duchess of Marlborough and Queen Anne

By the time of Anne‘s accession in 1702, she and Churchill had been together for 29 years. They had supported each other through the turbulent years of the Glorious Revolution and the combined death of 20 of their children. Anne’s loyalty to Churchill had even driven a wedge between her and her sister, Queen Mary II, and although both women were married — Sarah to John Churchill, later Duke of Marlborough, and Anne to Prince George of Denmark — their relationship was much more than that of monarch and servant.

The nature of their relationship can’t be known for sure, but it is generally accepted that Anne and Churchill’s bond went beyond friendship, thanks mainly to the latter’s kiss-and-tell memoir. Anne’s letters were full of affection, as when she told Churchill that “tis impossible for you ever to believe how much I love you except you saw my heart.” They adopted names which made them equal in their letters — Anne signed herself as Mrs Morley while Churchill took the name Mrs Freeman — and they expressed their passion unreservedly. “If I writ whole volumes I could never express how well I love you,” the queen wrote to Churchill, although we will never know the responses because the duchess made Anne burn all of her replies.

Both were equally devoted to their husbands, and it was not unusual in the 18th century for women to write romantic-friendship letters in the same language as they used with male lovers. It may be that the sentiments expressed were only pronouncements of a strong and deep friendship.

It is remarkable that the two women remained together for as long as they did. They were very different personalities: Churchill was overly blunt and unnecessarily cruel in how she treated the shy and fragile queen. Their differences over the evolving makeup of Parliament — with Churchill’s support of the Whig Party contradictory to Anne’s natural leanings toward the Tories — drove them apart, though the former’s absence from court, her cruelty when Anne’s husband died, and the arrival of Abigail Masham, whose treatment of the queen was considerably kinder, played a role as well. Churchill had one final meeting with Anne in 1710, and although she regained some of her status under George I, she was never reconciled with her former friend.

10. Melusine von der Schulenburg, Duchess of Kendal and George I

For much of his reign, George I remained unpopular, in no small part thanks to the actions of his mistress, Melusine von der Schulenburg. She had been George’s lover since about 1690, when he was still Elector of Hanover and she was his mother’s maid of honor. George’s marriage was particularly complicated — he had divorced his wife for her infidelity in 1694, five months after the murder of her lover, for which he has remained chief suspect.

Von der Schulenburg and the three daughters illegitimate daughters she shared with George accompanied him to Britain when he became king in 1714. As with many royal mistresses whose position relied completely on the king, she made sure to secure her future. She quickly established herself as a way to access and influence the king for a price, and the politician Robert Walpole claimed that, “her Interest did Everything; that she was, in effect, as much Queen of England as ever any was; that he did Everything by her.”

Regardless of what society thought — or the ridicule she received for her looks — von der Schulenburg remained devotedly by George’s side for the rest of his life. They were well suited, with the aristocrat Lady Mary Wortley Montagu noting that von der Schulenburg was “so much of [the king’s] own temper that I do not wonder at the Engagement between them.” George showered her with peerages, including making her Duchess of Munster in 1714 and Duchess of Kendal in 1719, and her creation as Princess of Eberstein by the Holy Roman Emperor at George’s request may indicate that he had secretly married her. When George died in Hanover in 1727, Lady Mary Wortley Montagu, returned to Britain where her grief was recognized by the new queen: “My first thought, my dear Duchess, has been of you . . .  I know well your devotion and love for the late King . . .  I hope you realize that I am your friend.”

11. Alice Keppel and Edward VII

Keppel was a descendant of Robert III of Scotland and a suitable bride for the younger son of the Earl of Albemarle, who she married in 1891. She was a lavish society hostess but, with little money, she soon began affairs with numerous wealthy aristocrats — with her husband’s blessing — to fund their lifestyle. In 1894, he acknowledged her eldest daughter, Violet, as his, although this is unlikely.

It was almost inevitable that Alice would eventually catch the attention of the Prince of Wales, a notorious womanizer who delighted in the company of the wives of other men. They met in February 1898 when he was 56 and she 29, and their affair would last 12 years and make the Keppels wealthy enough to afford numerous houses both in the UK and abroad, in addition to providing for her brother. In return, she was discreet and a good listener, and, according to one contemporary, made the king “a much pleasanter child.” Edward used her to promote his interests with the government; the government in turn used her as an intercessor. One diplomat wrote that, “there were one or two occasions when the King was in disagreement with the Foreign Office, and I was able, through her, to advise the King with a view to the foreign policy of the government being accepted.”

While she is not one of his most famous mistresses, she was one of his favorites. Keppel’s great-granddaughter is Camilla, Duchess of Cornwall and wife — and former mistress — of the present Prince of Wales.