Idea to stop the chain adjusters from moving, lock nut inside.
Just remove the rear tire to do the swing arm bolt switch. I took out the adjusters to see if I had a nut that would fit the threads. So far, no luck.
My idea is to add a nut inside the adjuster on the threads where it screws into the swing arm. You would get in there and loosen it up, adjust as needed, the tighten up the lock nut. It might be tough to get a wrench in there, but it's worth trying.
IDK if it's a common size or not, but I'm not having much luck finding a match so far.