Finally I found out, that in the conditions for one of the Follow Up Mails there was not chosen whether the User is or is not subscribed to the mentioned list. So the Mail was treated as if there was not any condition set and as soon as anyone was subscribed to anything, he was subscribed to these follow ups, too.
It was not easy to see, because on first sight it seemed as if there was a condition set. It would be good to have an error appearing, if there is no choice made between is and is not.
Anyways - the case is solved!