When I made my foray into the liberty movement five years ago, I felt like I stood out for a few reasons. I was a woman. I had no knowledge of economic theories. I was a vegan.

Being an outsider for my veganism was nothing new to me. I “quit” animal products of my own volition when I was nine years old, and I don’t need to tell you that kids can be cruel to people they see as different. As far as my peers could tell, I voluntarily became the one-girl branch of my school’s PETA chapter, so I was fair game for ridicule. The name-calling (generally) stopped as I got older, but I was still used to people thinking I was odd for my unusual dietary philosophy.

But exposure to the paleo community within the liberty movement introduced me to being an outsider in whole new sense. The lifestyle was ubiquitous, and though most paleo libertarians and anarchists were perfectly respectful, a surprisingly large minority regarded me with derision or hostility when they found out about my veganism, treating it as some sort of failure of the libertarian-purity test. Some paleo adherents in the liberty community actually seemed angry that I would call myself both a libertarian or anarchist and a vegan.

I found myself wondering: what was the seemingly-strong connection between libertarianism and the paleo diet? A recent Reason interview with John Durant, author of The Paleo Manifesto, shed some light. Libertarians, he says, like paleos, recognize the power of spontaneous order and that decentralized decision-making can produce cohesive systems. Libertarians apply the concept to economics, while paleos apply it to nutrition. Libertarians also tend to be skeptical of government recommended guidelines for anything, including food.

As for the hostility that vegans or vegetarians encounter, Durant suggests that the vegetarian diet brings with it ideological and political “baggage” that libertarians find unappealing. Though he doesn’t say what that baggage is, the connection is obvious. When thinking of the average vegetarian, most people think of someone socially and politically liberal, which is understandable. The vast majority of fellow vegetarians I’ve met have been liberal, and to this day, they usually assume I am too.

But easy as the connections between vegans and liberalism and paleos and Libertarians are to make, they’re superficial, and furthermore, adhering to them can hinder ideological growth. I say this because my conversion experience to the liberty movement was directly influenced by my veganism.

I was introduced to the liberty movement five years ago in college by a guy who was interested in me but first wanted to know if we could be ideologically compatible.* So he gave me the standard rundown: taxation is theft; the state is the gun in the room; the individual is the most persecuted minority. I wasn’t convinced, but I listened, because as a vegan, I knew that even strange ideas could have merit and also how frustrating it was to be dismissed because your argument wasn’t conventional. Then he threw a mind bomb my way: he said, “Look, you’re a vegan, right? Well your taxes go to pay for public school lunches, military rations and prison food, all of which have meat in them. No matter what your feelings are about it, you’re forced to support the meat industry with your money.”

I think my jaw actually dropped, not just because he was right, but because I had raised that same concern years earlier to my parents, and they replied, “That’s just the way it is.” My liberal parents completely wrote off my concerns about being forced to support something I found unethical, yet this Libertarian guy recognized that even though he didn’t agree with my stance on diet and animal welfare, the fact that I couldn’t financially withdrawal my support was wrong.

What I took away from this experience was that Libertarianism isn’t about agreeing on all social issues. I could be a vegan, and others in the movement could be paleos, or Christians, or feminists, as long as we recognized that we didn’t have the right to impose our preferences on one another, either directly or through the state. Libertarianism wasn’t the ideology of white Rand-heads; it was the ideology of individuality.

If we assign certain lifestyle preferences—paleo, atheist, sex-positive—to the liberty movement, we sacrifice that individuality. To be sure, I think most recognize this, at least consciously, though “no true Scotsman” cries do pop up periodically. To me though, Libertarianism is the radical idea that I don’t know what’s best for everyone else, and that includes what they should eat.

*It all worked out; we’re now a happily married vegetarian/libertarian couple.