The Irishman crashed hard during the high-speed run-in to the Col de la Porte, the final climb on Thursday's stage 2.
Martin landed hard on his lower back but got up and finished the stage. However, he was later diagnosed with a non-dislocated sacral fracture and the Israel Start-Up Nation team doctor Ortwin Shafer advised him to head home in the hope of healing up in time for the Tour de France, which is due to start in Nice on Saturday August 29.
"I am really hopeful that I will be ready for the Tour de France," Martin said.
"It's a shame as I felt I was in great shape but I'm a fast healer and I have reason to remain optimistic."
Martin explained how the crash happened.
"It was probably the best I've felt in racing since the lockdown but when you relax, those bad things happen. There was nothing I could do. I was preparing myself for the last climb, the last effort, but the guys crashed in front of me. One of the guys hit a sign that was in the road and I immediately I knew I'd hurt myself," he said.
"It's one of the weirdest feelings, I don’t have a scratch on me really, but I landed full force on the bottom of my back. Unfortunately we have a small fracture down there in my sacrum.
"I rode to the finish and could pedal no problem but it was painful and I could tell something was not quite right, so we got it checked."
Martin was diagnosed before dinner on Thursday and so headed home to Andorra rather than continuing in pain at the the Criterium du Dauphine.
"It's better for me to go home and recover, there's no reason to be here and not be 100 per cent fit," he said.
"We go home, recover and hopefully it heals in time for the Tour. I need to do everything we can to rest up and be as fit as possible for the start of the Tour."
Team ISN medical update regarding Dan Martin:After a crash in the second stage of the #CriteriumduDauphine @dauphine, ISN team rider Dan Martin was diagnosed with a "non-dislocated sacral fracture (S3)." Next: pic.twitter.com/8Eo939U9jlAugust 14, 2020