If all things are exported then anything in the module might be used in other modules. So none of them can be removed or changed, even ones that other modules should not be using.

There are often components of a module that other modules should not use. Keeping them private ensures that inexperienced or lazy programmers do not use them.