The one thing that I like about startups more is the camaraderie you have with your teammates. That plus interesting work makes startups a hands down winner over cubicle life.
If your job is to be an advocate for employees at your employer, it is likely better to be around those people, rather than just at the other end of a Hangout. I'm guessing that interacting with people around the office is a critical part of that role.
I think that's company culture friction ... if a group of people primarily works from an office, it takes a serious effort to accommodate remote workers.