My original plan was to interface controls to the Beagleboard through GPIO. It looks like the gpio-keys driver would make this pretty simple. But then there are people with expansion boards that use some of the IO on the expansion header. Also, from looking at the uboot source for Pandora, it seems they use GPIOs that are not available on the Beagleboard header.
In the end I think it would be better to use a USB game pad. Also, since I am using the /dev/input/event* interfaces it would be pretty easy to add in support for any controls that have drivers that use the event system.