If you let them talk directly to your database (for example via SSMS) you need to enter their IP (or you can just whitelist the whole range). Usually they will use your database via your own API, then it's not needed to whitelist their IP addresses.
How will your users be accessing the DB, via a Web App (front end) or directly (I assume you won't give users direct access to your DB?), if its via a Web App (presentation layer) then all you need todo is grant access to this IP address of the presentation layer/service layer (and if hosted in Azure its beside it).
You could either open up all IP address 0.0.0.0 - 255.255.255.255 (not very secure) or come up with more finer grained policies based on the above Database firewall rules.
Login to azure Portal
select your database subscription
click on Tools
Now there is option 'Open in VisualStudio' (click on it)
You can see "Configure Firewall" click on it.
Add you new IP.
Done :)
The only way is to do it is via SQL query. Azure shows only Firewall Server rules to be visible only on the portal but on database level the only way is via SQL.