I've been to three games so far this year. Always move around field level seats as I please. You must have had a bad usher.
That's just shameful. In similar situations, franchises frequently welcome the loyal fans who show up to move down and sit anywhere they would like.
The only real reasons a franchise would want that is to look more full. Generally, ushers just don't care.