I thought this was one of the most beautiful structures in Ghana.  While lacking the historical information you get at Cape Coast Castle and Elmina, Fort Princess Town is worth a look simply because it is a beautiful old fort that's a little bit off the beaten track in a small fishing village.  The fact that not too many tourists make it out this far means that you don't tend to get the hassles that you do in places like Cape Coast.  The locals here are wonderful and really just want to chat and have will often ask to have their photo taken if you have a camera handy.