Splash

I'm sure your guidebook recommends you Széchenyi Thermal Bath.
In fact, it is the most popular, but the most crowded - and less clean- too.

My favourite is Gellért Bath. The interior is breathtaking if you are about architectural beauty, you have to see it.
The whole building is almost at his original condition.
As a plus, it provides a wave pool service in the outdoor pool, which is real fun.

If you are travelling alone, Rudas Fürdő is highly recommended. It is a magic experience. You'll be out of time and space.

Tuesday is the only women's day here, the rests are for men only and coeducated on weekends (which is not recommended: crowded and loud).

In certain periods, there are late night soaking and parties held in Budapest Spas.
More info

For swimming only, I would rather suggest the Komjádi Uszoda.