Yorktown is a historic place. Cornwallis and battles and all that. Visit the museum!
Be sure to walk in the park along the river. It's a great place for a family gathering, cooking out, fishing off the rocks... or if you walk a little further down, you can walk on the beach.
Across the river is Gloucester Point. There is a beach there as well. There is also a pier from which to fish. Be sure you have a fishing license either place.
Be sure to have bug repellent.