Reinforcement learning (RL) has been developed for mean field games over graphs (G-MFG) in social media and network economics, in which the transition of agents between a node pair incurs an instantaneous reward. However, agents’ en-route choices on edges are largely neglected that incur an experienced reward depending on agents’ actions and population evolution along edges. Here we focus on a broader class of MFGs, named “dual MFG on graphs” (G-dMFG), which models two interacting MFGs, namely, one on edges and one at nodes over a graph. In this setting, agents select travel speed along edges and next-go-to edge at nodes for a minimum cumulative cost, which arises from the congestion effect when many agents compete for the same resource. This has various implications for autonomous driving navigation, spatial resource allocation, and internet packet routing. We establish formally that G-dMFG is a generic G-MFG, encompassing a more complex cost structure (that is nonseparable between states and actions) and with no need to pre-specify a termination time horizon. RL algorithms are designed to solve mean field equilibria (MFE) on large networks.