I have klayout-0.20.2-win32-install.exe installed on both Win XP 32-bit and Win 7 64-bit.
I have a large, complete IC I decided to test the XOR function against (I removed one small piece of metal 1 from the design). On Win XP 32-bit, it fails part way through with an error window that reports:
The process runs using about ~400 MB of private memory. There is 2GB of memory in the machine, and monitoring the system with Process Explorer still shows > 1 GB of physical memory free when it fails.
The same thing run on Win 7 64-bit runs to completion with accurate results.
Running the 64-bit binary of KLayout on Win 64-bit also works to completion just fine.