Ok, so I've followed the steps:
1. Removed Mainboard battery and cleared CMOS by the instructions of my Mainboard
2. Booted and checked that bios is in default configuration. Windows booted as normal, too.
3. Inserted NVMe
4. Booted - stuck on ASRock splash screen again (photo attached)

After shutting the PC down, removing the NVMe and waiting for a couple of minutes, I was able to boot as usual again. I tried inserting the NVMe twice again into two different slots with the same result.

I also noticed that the led on the NVMe adapter is only flashing shortly on startup, then turns off. If I recall correctly, the led was permanently on before the Instant Flash of the modded bios.