Diverse equilibrium systems with heterogeneous interactions lie at the edge of stability. Such marginally stable states are dynamically selected as the most abundant ones or as those with the largest basins of attraction. On the other hand, systems with non-reciprocal (or asymmetric) interactions are inherently out of equilibrium, and exhibit a rich variety of steady states, including fixed points, limit cycles and chaotic trajectories. How are steady states dynamically selected away from equilibrium? We address this question in a simple neural network model, with a tunable level of non-reciprocity. Our study reveals different types of ordered phases and it shows how non-equilibrium steady states are selected in each phase. In the spin-glass region, the system exhibits marginally stable behaviour for reciprocal (or symmetric) interactions and it smoothly transitions to chaotic dynamics, as the non-reciprocity (or asymmetry) in the couplings increases. Such region, on the other hand, shrinks and eventually disappears when couplings become anti-symmetric.