There is a far easier way to prevent uninvited guests from joining wich is to use password to be able to connect and allow ssl only but thats just another layer the password should do and also use a non default port to connect to and share with your friends of course so they can connect this should guarantee you peace on your server

I run a test net myself so i tried to come up with all kinds of solutions let me know how it worked out so we may try other solutions as there are various