Key doesn't turn at first try, and sometimes doesn't start either

Good morning all, I have been having this problem with my 06 Ml350, sometimes I have to try up to 10 times before the key turns in the ignition and starts finely. Sometimes i turn the key all the way to the end and nothing happens, after trying for a couple of times, the car would suprisngly start. The one that annoys me the most is when the key doesn't turn in the ignition, it acts like I inserted the wrong key, sometimes I have to turn the key the other way before it turns and starts fine. After a while the side that was working perfectly would start having the same issue and the other side will begin to work again. since this problem started, I have changed the Aux battery, Alternator, rear Sam, car key, fuel pump, but the issue still keeps coming back after it works for a while. Please I would appreciate if any of you can help me. I hate having to battle with my car key/ ignition for minutes evertyime I want to drive the car, and sometimes I end up not being able to use it. Thanks a lot