Hire a car, you can drive around the island in a day.
Valley of the butterflies, take water, it's generally hot, all uphill (with a cafe at the top). Check the butterflies are there before you go.
Tsampika monastery is beautiful, on a hill with great views. Any woman walking up barefoot is supposed to get pregnant.
Lindos again is interesting, beautiful, but don't miss out on the small beach on the south side as you drive past. You can park right by it and it's a small bay with beautiful clear water.
If driving there from Rhodes direction there's an archery place where you can shoot 3d rubber animals around the hillside.
Monolithios and the castle, Kritinia castle, the water park looked pretty good but shut the day before we arrived, Filerimos has a huge cross on a hill you can walk up steps to the top.