For almost every architecture out there the transition to 64-bits made existing code run more slowly for the reasons you describe, but on x86 the 64 bit transition also doubled the number of registers, guaranteed that SSE2 was available, and did some other stuff that actually made most code run faster.

However, I believe Google is working on some program to let people use all the goodies in x86-64 while also using 32-bit pointers.

This is one disadvantage, yes. However all registers are now 64 bit, and in general, there are more of them.

Furthermore, the 4GB limit does not just concern physical RAM, it also applies elsewhere, like memory mapped files.

In general, it is unlikely that you will see poorer performance from 64-bit code compared to its 32-bit counterpart.

Most of the memory used by programs are pointers, and if you double your pointer size, it has a negative impact on the efficientcy of the L1 and L2 cache.

Yes, this is exactly what CurtHagenlocher said.

>> Most of the memory used by programs are pointers

This is a huge generalization, may I add.

Yes, I will not argue that cache usage is less efficient, but I am sure you know that register access is faster than cache access, and we double the amount of registers in x64.

To quote Donald Knuth, a 64 bit program would "waste half the memory [and] effectively throw away half of the cache".

You may double the number of registers, but you are still shuffling tons of data between your caches and memory, and that is going to be your constraint for most programs. Losing half of the 4mb or 12mb of your L2 or L3 cache because you use 64 bit pointers has a much bigger impact than doubling the number of registers.