I solo traveled in Ninh Binh December 2019 and I can totally relate when you said you had to stop at random spots because the views are amazing. Everywhere is picturesque!
Ninh Binh is so far my favourite place in Vietnam. I stayed in Trang An which is a lot more quieter than Tam Coc. Tam Coc boat ride can be really good or complete garbage depending on your boatman. I was in luck with mine. The lady was an absolute gem! She even stopped at the viewpoint and we hiked there together. At the end she received a hefty tip w/o even asking. She was amazing. But a boat ride I would really recommend is the Trang An boat ride. The views are amazing! And you will also not get scammed by either the boatman or people along the way
We enjoyed our boat ride in Ninh Binh which stopped at temples etc and you could get off to see them. It was in a different area to where yours was. We went through caves with coloured ceilings and it lasted 2.5 hours. It was excellent. Wearing the local traditional hat was great at keeping the rain off as unfortunately we had a little light rain shower. Glad we bought them. No sellers annoying us, just a very pleasant time. We had a private guide to organise it for us. A full day out from Hanoi. We went mid March.