Under the arcades near Plac Kościuszki, this vinyl retailer is actually located on the half-floor above/inside the Pod Arkadami bookstore (not much of an English section, but go ahead and ask them to show you). Find crates of used records at good prices and there's even a record player where you can actually listen before buying.
