Sometimes it fails to union, but sometimes it unions and fails. I’m taking wedges and dragging them against the slope of other wedges to align them.
Try to move the wedges into each other a little bit so the faces of the bricks don’t occupy the exact same space.
Also don’t forget the repro file 
Just an FYI, the reason it seemingly fails/succeeds at random is because of the order the parts are selected – not due to randomness.