Silent cave ended before attempt

I had finally made my way to one of the silent cave map locations and I put in a map and loaded it up and before I could even click on the portal the portal collapsed and it said I failed I didn't even get a chance to go in to see it