I added Instagram to the networks in the setup, but then I found that they don't offer an easy way to create an account (other than using the iPhone or Android app) so I decided to delete it again but I couldn't.
Whenever I set Instagram to "no" and save the settings, it will be back with a yes again and also show on the login pages inside WordPress (and obviously not working because I didn't have the nessecasy API keys).
The only way to get rid of it was to delete value "1" directly inside SQL using phpMyAdmin, so I already solved the problem for myself, but I just wanted to report this minor bug in the settings page that doesn't remove Instagram from the list of networks you want to use.