While I definitely fall into the plotter camp, I stray from my outline on occasion. Some of this comes from my bad habit of editing as I’m writing the first draft. If I give in to it too much, I’m liable to drop from 1,000 words a day to the double digits.
Other times, I change direction because something I wrote in an earlier chapter has already been flushed out more in a later. Just because I write an outline doesn’t mean I write in order. Scenes come together in my head differently.
As my current work-in-progress (WIP) is first person, my biggest struggles come from scenes where he’s just observing. It’s easy to interrupt the flow of dialogue trying to ensure the MC doesn’t fade into the background. On the other hand, if you have an entire conversation with the narrator reasserting themselves at all (dialogue or otherwise) they can get a bit lost. The edge between has always been difficult for me, and I’m constantly working and reworking those sections.
To combat my off-roading, I maintain a playlist for each book. The music changes depending on the chapter, and by the end, I’ve got a full mental soundtrack. Even if I change the outline, the mood and pacing remain the same as my mental tempo matches the pre-set flow of the songs.