"you could walk away from the switch."
well, then you're deciding to let it hit the five instead of the one.
"that's the same as doing nothing in the hospital problem."
no, because your actions aren't explicitly responsible for the death of the five patients, who are dying of unrelated disease, like they are for the death of the five walkers. if you walk away from the switch, it's your fault when it hits the five walkers; if you walk away from the hospital, it isn't your fault that those five people die from those five unrelated diseases.
i guess maybe that's something else i'm doing here - i'm converting the problem into an issue of individual responsibility. i guess that philosophers want to talk about broad moral issues, but who cares about that? what's real is what choices a person has in front of them, and what follows as a consequence of their actions.