I live around the Memphis area and am new to the whole vinyl game. I'm sure there are some other places too, but I only know a few places to get records. Haven't personally been to the last two yet, but I hear they're pretty good.
Spin Street
3484 Poplar Avenue Memphis, TN 38111
Shangri-La
1916 Madison Avenue Memphis, TN
Goner Records
2152 Young Avenue Memphis, TN