At this age and mileage, I have now had 3 Duramax's that have had similar symptoms with long cranking times that were puzzling to diagnose the cause .... and each one came down to the same fix, even though each one had slightly different signs.
Rebuild your fuel filter head. You can buy the seals online for $30 or so on eBay, and YouTube has videos showing how to do the rebuild so that you get things right. Its not hard to do, its inexpensive, and the only special tools you'll require is the proper sized Torx screw driver ( why you don't already have a set in your toolbox is a sin anyway).
Now, while I won't put big money on this being your fix, I would be willing to put $20 on it, ... if we were friends who lived in the same area. This was the fix for my first 2004 2500 LBZ, my present 2005 LLY 3500, and a neighbour's 2005 2500 LLY. Good Luck!