The nature of Hildifons' journey is not explained. Indeed, we cannot be absolutely sure that Gandalf was involved, though that does seem to be heavily implied. When Gandalf first approached Bilbo in the prelude to his own journey, the Hobbit thought of Gandalf as '...responsible for so many quiet lads and lasses going off into the Blue for mad adventures. Anything from climbing trees to visiting Elves - or sailing in ships, sailing to other shores!' (The Hobbit 1, An Unexpected Party). The 'sailing' here seems to refer to Hildifons' younger brother Isengar, who was said to have gone to sea, so we might take it that Hildifons was the Hobbit involved with visiting the Elves. Beyond that, we can say no more of why he left the Shire, nor why he failed to return.