Who did you complain to? You said you felt uncomfortable, why did you not call the MaitreDe' over & make them feel uncomfortable if they were improperly dressed, I would have! As for some not turning up they were probably in the specialty restaurants or the dine on a tray trough.
If you have this problem go ASAP to the Restuarant Manager or your Maitre'd and ask for a table change, just for two if necessary, or if you have met others who would not mind you joining them a switch to their table. This sort of behaviour is not acceptable.