You didn't mention where you lived, but that can be significant. For example, here in NW Arkansas we found it wiser to have one '05 E320 and one ML or GL with AWD. We get snow and ice here in the hills and this occasionally calls for AWD.
Apart from that issue, the '09 has fewer miles (but not significantly fewer), appears to have been properly repaired, and is the same basic size and comfort level. While the SBC and fuel tank / pump issues still face the '04, those aren't an issue on the '09. However, I believe the 350 engine faces the crank sensor and balance shaft issue that the 320 doesn't, but these are less expensive than the SBC and fuel tank / pump.
I love our '05 E-320, but in your situation I'd probably sell the '04 and keep the '09.
Apart from that issue, the '09 has fewer miles (but not significantly fewer), appears to have been properly repaired, and is the same basic size and comfort level. While the SBC and fuel tank / pump issues still face the '04, those aren't an issue on the '09. However, I believe the 350 engine faces the crank sensor and balance shaft issue that the 320 doesn't, but these are less expensive than the SBC and fuel tank / pump.
I love our '05 E-320, but in your situation I'd probably sell the '04 and keep the '09.