When you hover the mouse over a totem in the inventory, it automatically moves to the secondary hand.