Legitimacy vs. authority

"Against Democracy" is a libertarian's argument for epistocracy (as opposed to democracy). I am a socialist, but I am also inclined towards epistocracy. I agree with some of Brennan's arguments, but some of them baffle me.

On page 149-150, Brennan writes:

A government is said to have the right to rule ... when the following conditions obtain: 

- It is morally permissible for it to create and enforce laws ... 

- ... people have moral obligations to abide by the laws ... because the government issued those rules. 

When the first condition obtains, a government i said to be legitimate, When the second condition obtains, a government is said to have authority. 

By definition, a government is legitimate just in case it is permissible for that government to stand and to create, issue, and coercively endorse rules. By definition, a government is authoritative (or "has authority") over certain people just in case those people have a moral obligation to obey that government's laws ... Legitimacy is the thing that is supposed to make it okay for the police to arrest you. Authority is the thing that is supposed to make it wrong for you to resist them ... In short, legitimacy refers to the moral permission to coerce, while authority refers to a moral power that induces in others a duty to submit and obey.

In a footnote, Brennan adds: "In earlier political philosophy, the terms were used in sloppy or non-uniform ways. In the last tens years or so, however, it has become the convention to use the terms exactly as I define them here."

In the following pages, Brennan talks about how competence is a presumptive condition of the right to rule. Without competence, as described by Brennan, there can be neither legitimacy nor authority. But he does not seem to differentiate between these two concepts. Apart from the definition above, his discussion seems to conflate them.

Now, I think I have managed to follow Brennan's subsequent reasoning about competent rule, more or less. And partly because he no longer holds on to the distinctions made above. My biggest quandary is making sense of the distinction itself.

I talked to my wife, and she seems to share my intuition that the distinction seems artificial, somehow: You can't have one without the other.

Then I talked to my youngest son (aged 11). He said something that seemed to square with Brennan's distinction, but that I myself couldn't quite get a hold on: He said he believed that I have, and should have, the right to decide over him. But at the same time, there are instances when doesn't feel obliged to do as I say.

I struggle to make sense of this. Who decides whether a particular government is legitimate and has authority? Does everyone have to agree? Can one and the same person regard a particular government as legitimate but as not having authority, or vice versa? Is it a case-by-case decision?

To me it seems that if a group of people have deemed a certain leadership legitimate (after investigating its competence, among other things), then they have also submitted to its authority. Even if they scrutinize the competence on a case-by-case basis, there should be no exceptions to their obligation to abide by a decision that is made competently.

What do you think?