I did just try it. I had to purchase the 4 for my new surface pro 4 because my sp2 keyboard is not yet compatible with the SP4. However I popped the SP4 keyboard on my SP2 and it works great.
Hm.. Was considering picking up a cover 4 for my sp2 (I know it's overly big for it) just because the keyboard and mainly track pad are massively improved, the type cover 2's track pad was absolutely tragic; a stain on Microsoft's design history in my opinion.