Here's a (currently very short) list of banks that serve Euros. More will be added later.

Raffeisen Bank - Central Bus Station

Handily, this bank is right at the end of the bus station on the corner of Rruga Adria and Rruga Egnatia.

Raffeisen Bank Location

OTP Bank - Sheshi Liria

Right across from the Great Mosque, although I've found the ATM to be out of action a couple of times.

OTP Bank Location