Is it normal for the birds to sleep in the nest box?

Yes. If you have provided the birds with a nest, they will most likely sleep in it.