It’s not a huge deal, no.
If the Tums worries you, try 1/2 tsp of baking soda in a small (4 fluid ounce) glass of water. It’ll taste bad, so chug it fast. Actually, I find it works better than Tums for stomach acid.
Also, try cold brew coffee. I find it much less acidic.