In the new Windows 10 Creators update there is a change to the “New Tab” button in IE. It now offers you the ability to open a new tab in IE, or a new tab in Edge.
If you want to hide the New tab in Edge button, that capability exists. The setting is in group policy under User Configuration >> Administrative Templates >> Windows Components >> Internet Explorer >> Hide the button that opens Microsoft Edge.
Set this to enabled and the Edge button disappears.