To display the category tree only on category page (or on the product page that is assigned to it) check the correct options in categories widget:
You should check:
- show hierarchy – to display the categories in a hierarchical list
- show children only on category page – to display the subcategories only for the current category or product that is assigned to it