To add a few further details— SSID and password are set via code (in settings) but I tried via the configuration mode as well. It seems like the setting changes aren't taking effect, but I have no idea. In the Arduino IDE, I set it to flash everything (incl. the Wi-Fi settings) not just the sketch.