Not sure whether you're arriving in Venice (eg airport) at 4pm or embarking at that time, but in the former case you're not going to have much daylight left after baggage reclaim, transfer etc, and in either case the major buildings (St Mark's cathedral, Campanile, museums etc) will be closed. My advice would be to just take the shuttle boat provided by the ship (there'll be a charge)into the city and stroll around St Mark's Square and the streets behind it. There'll be plenty of atmosphere and I've always found Venice safe at night. If you have a drink in one of the cafes in the Square itself though, it will be extremely expensive!
Venice is such a small city it is very easy to see on your own, just hop on a vaporetto from the port and enjoy the sights. Or walk the streets, get lost, stop for a gelato, ...the main attractions are all signposted. This is a very useful site to use for your research and will show you the vaporetto routes. the port is at marritima FYI. http://europeforvisitors.com/venice/articles/local-transportation-index.htm.