Castle Acre village is home to their very own castle, although now laid to ruin. It was built soon after the Norman Conquest in 1066 by William de Warenne who was given the land, and was intended to be a fine stone-built fortified house for himself and his family and included a planned settlement in the town. His son and descendants laid out the town and founded the priory. The castle's defences were strengthened in the 12th century, turning it into a great tower and fortress. It had a large moat, a gatehouse, a large keep, which was converted into the main house and tower, a chapel, stables and houses. It was strengthened during the wars between King Stephen and Queen Maud (around 1140) into a more martial stone keep, protected by stone curtain walls and an elaborate system of earthworks, which still exist today. William de Warenne's descendants lived at the castle throughout the 13th century but by the Middle Ages the castle fell out of favour due to the disbandment of the monasteries and was abandoned and left to ruin after King Henry VII ordered that the defenses be demolished after the owners failed to obtain a royal assent for the planned fortification of the defenses. The remains of it today are protected by the English Heritage and is one of the finest historical castle ruins in the country. It is a popular visitors spot in the quaint Norfolk village.