Tricky one. Saga are the only 'senior' cruise line - my mother won't travel with anyone else. I guess I would suggest the top end lines like Seabourn, SilverSeas, Oceania or Azamara - not necessarily an over-50 crowd but definitely a classier set and few if not no ankle-biters (children). Itinerary also dictates the type of guest - transAtlantics usually have an older crowd, as do longer itineraries, as those of us who are still workers cannot get the time off!
P&O Artemis? Small ship (but bigger than Saga Ruby) and adult only.
Fred. Olsen if you avoid school holiday times.