You could play around with some pseudo elements, and create up/down arrows of css content hex codes. The only challange will be to precise the positioning of the arrow, but it may work:
I modified @LcSalazar's answer a bit... it's still not perfect because the background of the default buttons can still be seen in both Firefox, Chrome & Opera (not tested in Safari); but clicking on the arrows still works
Notes:
Adding pointer-events: none; allows you to click through the overlapping button, but then you can not style the button while hovered.
The arrows are visible in Edge, but don't work because Edge doesn't use arrows. It only adds an "x" to clear the input.
I've been struggling with this on mobile and tablet. My solution was to use absolute positioning on the spinners, so I'm just posting it in case it helps anyone else: