50 yards? At 50 yards, a varying 15 mph left or right wind can move the 124 grain bullet about an inch. A head or tailwind is not going to do much.

My experience is that the more I pay for a barrel, the better they shoot. You could try and troubleshoot your current barrel or move on to something with a good reputation. A really good bore cleaner is where I would start. I like foaming Wipe Out or Birchwood Casey bore cleaner. Let them soak to do their thing. When the patches stop coming out blue, you should be good to go.

Another thing is to check the fit between the barrel and upper receiver. Remove the barrel from the upper, then check to see if there is play between the upper and the barrel extension. I have used a thin amount of JB Weld , some like Loctite 620, on the extension and re-torqued the barrel nut. Let it sit a day or two before shooting.

Are you using a suppressor or a muzzle device, etc?