GMail was quite annoying placing all the old emails to the "All Mail" folder.
I don't care if they want to archive them but I do care when my IMAP client downloads all the oldies each time and keeps the huge copy on the local storage.
Praise all gods, that can be fixed. As described in the following article:
In short, just go to the settings, then to the "Labs", and enable the "Advanced IMAP controls". Then go to the "Labels" and uncheck the "All Mail - Show in IMAP" checkbox.
That's it. The mail client won't see that folder anymore!