You have brakes that fail and then kind of sort themselves out again?

Do you mean that sometimes your pedal is really hard but occasionally you don't have to push as hard?

If the answer to the second question is yes I'd be looking at the brake booster - however the fault for the brake booster would probably be described as "it is sometimes really hard to push the brake pedal" <= like you loose power assistance.

Last question => How much brake fluid have you added to the system recently?

 

I agree with Army. Check the Brake Booster, Brake Master and Brake Fluid.

 

If the Brake Pedal goes to the Floor and you Brake Fluid level has always been normal that is likely they Brake Booster.

If the Brake Fluid got too low and you filled it back up to normal, it is possible Air got into the System. This could show itself a Brake Pedal that goes to the Floor but if you pump it you get some of your Braking back.

Of course you could be having more than one problem

If you have inadequate Vacuum for the Power Brake Booster pumping rapid pumping of the Pedal depletes what little vacuum there is and brings on hard Braking effort.

On the Diesel Models there is an O-ring between the Master Cylinder and that Power Brake Booster that seals in Vacuum. It can become a source of a Vacuum Leak.

If you remove the Master Cylinder to inspect or change the Oil ring check inside Booster Cavity to see if there is Brake Fluid inside; indicating that the Rear Master Cylinder Piston Seal is shot (by that time it leaks the internal seals in the Mastercylinder are also most likely shot).

 

At this point, it's safe to assume all components on the main brake system (brake fluid, booster, master cylinder, and vacuum assist) are suspect and must be inspected. If an air leak is found, then the downstream components (rubber brake hoses per wheel and brake calipers) are then suspect too.

When you say the power assist works intermittently, I interpret this as becoming harder to depress and this is why you inspected the hose that provides vacuum assist to the brake booster.