Kayaks are made narrow so that they move easily on the water. This means that you need to actively keep the balance. Fortunately the technique is is easy and usually comes naturally.
To stay in balance, you need to try keep your upper body upright. Just relax your stomach and lower back and let the kayak tilt freely when the waves push it. Keeping your eyes on the horizon will also help to keep the balance. Don't hold on the kayak. It will tip you over rather than give you any extra support.
If you loose balance and need to get support from somewhere, just quickly slap the surface of water with the back face of your paddle blade. This technique will give a fairly good support for a second, so you also need to recover your balance immediately.
This and some other techniques to get support from the water surface are described in detail later on the Braces and eskimo roll section.