Large-scale quantum information processing tasks can be realized by interconnecting different quantum processors through photonic links in modular quantum computers, quantum data centers, or quantum networks. Reconfigurable optical routers are needed to facilitate changing demands of connectivity between processors during the execution of protocols. There is a tradeoff in the design of such routers: a deeper router allows for more connectivity, leading to more efficient protocols, but also introduces more loss. In this work, we consider this tradeoff in the design of a router for a multiplexed quantum repeater node with many quantum chips. A deep router allows for dynamic multiplexing policies, in which chips can be assigned for entanglement generation with either of the two end nodes, based on the current entanglement in the system, while with a shallower router this assignment is fixed. We consider a parameter regime where on average less than one entangled link is generated per end-to-end communication cycle, which is the relevant regime for near-term quantum networks. We show that in this regime, the dynamic multiplexing policy can lead to a significant improvement in end-to-end fidelity over a fixed policy, while marginally improving the rate. For limited router losses achievable with current hardware, dynamic policies can still achieve higher secret key rates in the parameter regime studied. Our results suggest that quantum repeater nodes for near-term networks should be equipped with deep routers that support dynamic multiplexing capabilities. Additionally, our work may serve as a benchmark for future investigations into routers for other quantum information processing tasks.