A Real Life Isle of Pines
One book that I clearly remember reading during my undergraduate days is Henry Neville's The Isle of Pines, a sort-of dystopian work about an island community all descended from one European male and his four female consorts. Something along these lines is clearly possible, as shown by the example of Palmerston, part of the Cook Islands. According to this BBC feature, 59 of its 62 inhabitants are descended from William Marsters, an Englishman who settled on the island in 1863. Sounds like a remarkable place, despite some inbreeding.