When we were in Cyprus we booked a 3 day cruise that took in Egypt and Israel. We booked it while we were there and it was so much cheaper than booking it from England. We talked to the locals and they recommended someone who could organise the trip. He came to our hotel, organised the transport to the port and tickets for the cruise. People on our trip booked along with their holiday or through the rep and paid double. Not sure if Israel is such a good place to visit at the moment but Im sure you will find them going somewhere.
I believe Louis Cruise lines do short cruises from Cyprus. You could do an internet search for them. Also believe Thomson may have some cruises ex Limassol. You could book them 'cruise only' if you want to stay on the island under your own arrangements.