2005 E320 CDI engine misfire at idle when fully warm

I have been pulling my hair out trying to solve this problem. Several months ago I noticed my car would sometimes stumble at idle after a long day of driving. It would only happen for a moment or two and it would smooth out. But as the weeks passed, the problem would happen more frequently with less driving, and eventually turned into a full blown misfire at idle. However the problem only happens at idle when the engine is FULLY warmed. If I let the car sit for a few hours and go to start it, the engine purs just fine....until it warms up again. And it only happens purely at idle and when I let my foot off the accelerator. Any throttle at all and the engine totally smooths out and works perfectly fine.

I have tried replacing the injector nozzles, seals, and return line o-rings with no change to the problem whatsoever. I just got done doing a leak off test and all the fuel levels were dead nuts even with each other.

I have replaced the fuel filter, bypassed the EGR valve, and I have all but deleted the swirl flaps. Nothing has even made a change.

I have taken this car to a mercedes shop to see if there were any hidden codes, and with the exception of the egr being bypassed, there are no codes for fuel pressure.

I honestly have no idea where to go from here. I would appreciate any insight you guys might have.