In 1230 Robert Arsic died leaving his estates to his two daughters Joan and Alexandra, and from then on the lordship of Somerton remained divided until the early sixteenth century. Joan and her husband soon sold their half to Walter de Grey, Archbishop of York. His family, who were major landholders in Oxfordshire, built the medieval castle in Somerton, described in 1295 as having a court with dovecote, fishponds, curtilages and gardens. It is commonly supposed that the present Old School House and old schoolroom stand on the site. The remains of the fishponds can still be seen.
The de Greys and their descendants held their share of the manor until 1485, when it reverted to the Crown as Francis Lord Lovell was attainted. Francis Lovell was created 1st Viscount Lovell in 1485 — but the very year of his elevation was the year of his ruin. He was one of Richard III’s closest allies, a companion from Middleham Castle, and bore the third sword of state at Richard’s coronation in 1483. When Henry Tudor defeated and killed Richard at Bosworth, Lovell lost everything. He fought on in subsequent rebellions and at the Battle of Stoke Field in 1487 — the engagement that finally ended the Wars of the Roses — he was reported killed. But he was seen trying to swim the Trent on horseback and may have escaped to Minster Lovell in Oxfordshire, where legend says he lived hidden in a sealed vault and died of starvation. When a new chimney was built at Minster Lovell in 1708, a skeleton was found in a secret chamber — a man seated at a table with a book, paper and pen — supposed to be Lovell’s. The Crown held Somerton for the next 25 years, until 1512. After the division of 1230, Alexandra’s share had passed through marriage to the Gardinis family and then to their relatives, the Giffards, who seem never to have lived here, having preferred their estates in Twyford in Buckinghamshire.
The church saw much development during this period. The chancel probably dates from the late thirteenth century. On the south side of the nave there are two fourteenth-century arches showing that there may have been a longer south aisle before the Fermor Chapel was built. The bell tower was also probably built in the fourteenth century. On the outside of its north wall is a finely carved holy rood, probably made at the same time as the tower was built.
The connection between the manor and the church was particularly close in the fourteenth century, when two members of the de Gardinis family were rectors for many years. Richard de Gardinis was rector from 1316 to 1349 and probably built a lady chapel and chantry. William de Gardinis, who became rector by 1377 and remained until 1392, was more controversial — he was alleged to have committed robbery with violence.
John Aston, a relation of the Giffards by marriage, held land at Twyford and in 1438 exchanged these for the Giffard estates in Somerton. He apparently lived at Troy Farm — then called Somertons — and when he died in 1459 left one hundred shillings in his will for repairs to Somerton church.
John Aston's son William inherited, but in 1504, shortly before his death, his share was conveyed to William Fermor. Eight years later, in 1512, the Crown granted the de Greys' share also to William Fermor, who thus reunited the manor. William proceeded to build a fine Manor House, some remains of which can still be seen behind Fermor House, to the east of the Heyford Road.
— Somerton Village History Project archive