The product search widget has an option to enable a letter filtering feature.
To enable it, please go to Appearance > Widgets, find the catalog search widget and place it in your selected sidebar (preferably in the product filters bar ).
Check the Show Letter Filter checkbox and click save to enable it. Now the letter filter will show up under the product search field:
As you can see on the screenshot, only the letters available in the current product listing will show up.
When the user clicks on the letter, only the products with a name starting by selected letter will show up.