Freedom track eating my wheels

If it's rough enough that it's damaging your wheels it needs to be smoothed IMHO. I am not familiar with how a Freedom Track is assembled. Is it possible that it is put together incorrectly? If not, then I would examine each track section and joint and remove any rough edges.
 
I hope that I have it assembled correctly. Pretty hard to screw this up. I just cannot get all he seams to line up. I have a two lane track and one lane is always much faster because I cannot get both lanes smooth. I have contacted Freedom for advice with limited response. Guess I'll take a file after it and taper the down track guides.
 
I would double check each joint prior to tightening the screws to ensure alignment. I've misaligned the track when I've rushed the assembly process.

Worst case if you're concerned about filing the joints, you might try masking or duct tape over the rough areas. Monokote trim also works wonders.

I closely inspect each joint before I tightening things up and raise the track.

Good luck.