Koh Samet is a great family choice.. Just do NOT go on a Thai holiday as the island will be jammed with over 40K Thais.. When traveling down to Samet, if you take your own vehicle you'll be parking it in secure parking at Ban Phe the port facing Koh Samet which is about 4km off shore from Ban Phe. If you take a bus it will drop you off about 150m down the port road from the ferry/boat terminal. At the terminal/pier you'll find mainland offices of many of the resorts/hotels/bungalows. You can take a hired speedboat directly to whatever beach you're staying on, or take the ferry across for a cheaper alternative. Hotels/resorts and bungalows abound on the east side of Samet and their prices range in accordance with how fancy they are. There is minimal "night life" of the bar sort on Samet's beaches. They do have some club/restaurants that party into the wee hours. But, in my many years of visiting the island I've seen only a very few "working" girls actually plying their trade.
Sai Kaew villa on Sai Kaew
http://www.saikaew.com/ (jewel beach)is a great place and on the same beach is a cheaper place Sinsamut Bungalows, a mere 50m down the beach from Sai Kaew villa and lower end, but the owners are quite nice
http://www.sinsamut-kohsamed.com/eng/index.html. I've had my family, kids/grandkids at both Sai Kaew villa and Sinsamut and each visit they had a great time..
A caution on Samet...!! Avoid coming outside from about 1800-1900 as the mosquitos from the swamps behind the beach are horrendous!! After 1900 they completely go away as nice evening sea breezes from the east rise up. Most folks hit the beach restaurants in the evening. The venues are many with most setting up bamboo mats and Thai style short tables and axe pillows to relax under the stars on the sand while you eat your dinner.