Because I"M NOT LITVISH EITHER!
OK, deep breaths. Heee, haaaa . . .
Let's play a little game of Jewish geography, OK?
Litvish = Lithuania, a specific area of Russia. Not heimish at all. Different region completely.
Now, chassidus: Invented in the 1700s. Took time to spread. Meaning, the dialect predated chassidus.
Chassidus originated in Ukraine/Poland (moving borders over the years), so there was more of a chassidish influence there. So the heimish from there will sometimes have chassdish accoutrements: a bekechel, maybe a gartel, perhaps peyos behind the ear.
Now, let us move on to Hungary. Hungarian Jewry divided into two primary areas: Oberland, which allied itself with Austria, having a Vein/quasi-Yekkish outlook; and Interland, which while did contain some chassidim (not many, since the chassidus never spread in Hungary the way it did in Poland), also had these others (like my family) who are NEITHER chassidish or litvish (how could they be litvish? To be a litvak, one has to be from Lithuania originally).
Now, these individuals still exist in Jewish communities outside of the US, but here groups tend to identify with strictly 1) chassdish 2) yeshivish.
But there are other groups besides these two! Albeit an endangered species.