Well lets check this one below
Remember as far as usergroup permisisons if you have users whom belong to several usergroups and one is not set to have moderation and they can view the cat they will be able to. Only if the user belongs to one or multiple usergroups that can not view and have moderation will this apply. The best possible permissions settings always apply.
If your saying its not working when you set say registered users not to view the category they wont be able to unless if they belong to multiple groups like meantioned above.