Yeah, the fuel system isn't meant to stay pressurised.

First thing you should do is to remove your adjustable fuel pressure regulator and put the stock one on, you have no need for an adjustable one.

Fuel pump does not prime with a factory ECU, it only switches the pump on once you turn the key to START. Well in reality the ECU doesn't turn the pump until the engine has actually started. The circuit opening relay turns the pump on when you're in the START position.

Its quite possible that some dope hasn't wired your circuit opening relay/hasn't used one at all and that only once you've cranked it for a while the ECU then triggers the pumps on and finally the car starts.

To test for this, check that your pump is getting voltage when you're in the start position. You don't even need to have the starter connected. If you disconnect the trigger lead to your starter and just turn the key to start, you should be able to hear the pump switch on.