The performance you can get from a small desktop computer
A lot of people avoid the small desktop computer because they think the performance such a setup can offer is very limited. While it is a bit limited due to the size of the components (and the cooling systems, if you want to keep it silent), you can get a lot of power from one of these small computers. If it is gaming you have in mind you can easily play any kind of game that has been released in the last period without any kind of problem.
There are components with a lot of power (not the fastest, but they are up there) that have been specifically designed to be installed and used in one of these small desktop computers. Today you can build a quad-core, 4GB Ram and a very efficient GTX 250 or even better without any problem. You can fit all those components in the small case and they will work great. You can also imagine that this kind of setup has no problem running the latest games and applications. In fact, a lot of people use the small desktop computer as a replacement of the game console. They set it up in the living room, they hook it up to an HD TV and they use wireless controllers, just like the ones that you have with your game console. They make this choice because the games are cheaper and a computer offers much more in terms of functionality than a console will ever provide.
In conclusion I can say that if it power and speed that you want from your small desktop computer you can certainly have it. I recommend building a mid-range system because it will be easier to actually find the components and it will provide with enough power and performance to run most applications and games.