Yup. This is my experience too.While preparing for our upcoming trip I went and did the unthinkable which was to crawl under the truck to look for anything that might be wrong BEFORE we got to camp. I checked the 5MM bolt tightness and found the passenger side tight but the driver's side had backed out about half!!! I don't know what happened as I tightened them both really good and tight almost to the point of bending the Allen wrench.
So you might want to check the bolts now and then.