Pagination widget will show clickable available pages for currently viewed product list. It can be added to any of your widget areas in Appearance > Widgets:
It will show up the same as the default pagination buttons on the bottom of the product list.
The pagination shows up only if additional pages are available.