I would set your FE range a little larger and set your I=0, D=0 and P = .1 or so
to first verify proper feedback direction.
There are a couple of possibilities here for what appears to be runaway, one is feedback direction. the other one is oscillation because the PID loop gains are starting out too high
By setting a low P gain. proper feedback direction can be verified
If this does not work, I would break the feedback loop (say disconnect the drives from the analog out) and verify that the analog outpu does the expected thing when LinuxCNC is started and the motor shafts are rotated away from their starting positions (the analog voltage should be proportional to the distance from the starting position (and the polarity should match the direction of motion from the starting position)