I'm gonna go ahead and say that SOME restrictions ARE necessary. Necessary restrictions vary according to region, landforms, wildlife, weather conditions, etc., but some are needed everywhere.
The problem that WE as consumers face is that the restrictions which the authorities devise for the supposed function of mitigating the issues caused by illegitimate users, in stead inconveniences the legitimate users enough that we abandon use altogether, and leave the areas in question to just the illegitimate users.
Of course we all know that this is the opposite of the intended effect, but the legislators look at the resulting destruction and say, "Our restrictions were not enough! Time to restrict the good guys some more, in hopes of stopping the bad guys that we're inviting in!"
This logic is not exclusive to our sport/hobby/lifestyle. It happens to fit nearly all new legislation since about the turn of the millennium.