Compton Castle is a fortified manor house in the village of Compton (formerly Compton Pole), I n the county of Devon England.Compton Castle's most famous inhabitant was Sir Humphrey Gilbert.(1539ââ¬â1583), coloniser of Newfoundland and half-brother of Sir Walter Raleigh;