Alright, it's done! Thanks a lot for the advice, guys. It was super easy.
And now, the instructions, since I couldn't find a good one when I was searching online (hence the helpme post), and it would be nice for any other newbies to be able to easily find such a description:
A/C idler pulley replacement on a 6th generation, specifically 1999 Honda Civic:
Step 1: Using a 12 mm wrench or socket, loosen mounting bolts for power steering pump (the topmost piece of equipment with a belt around it) and remove power steering pump, moving it forward and to the left. There is a grounding wire that you'll have to maneuver it under to secure it out of the way. You should not have to unbold the hose or the copper tube affixed to the pump. Move the belt out of the way
Step 2: Using a 14 mm wrench, preferably a ratcheting wrench (a socket wrench will not fit), loosen the nut on the pulley bolt itself. Be careful not to let the washer fall.
Step 3: Use a 10 mm socket and with very long extension (more important later) to screw out the tensioner bolt that is going straight up and down behind the pulley. Be careful not to unscrew it all the way out of the idler pulley axle bolt below. This will result in the belt tension being relieved enough to pull the actual pulley free of the bolt. Be mindful not to lose the other washer/bushing, and also be mindful of the order in which these are placed, as well as the side of the pulley. Additionally, one side of the pulley is hollow and one is basically flat. Make sure to take note of which side is which in relation to the engine. The hollow side should be facing outward, away from the engine.
Step 4: Place the new pulley on the axle and go ahead and tighten the nut all the way (but not too tight).
Step 5: Replace the power steering belt and then replace the two 12 mm bolts on the top/bottom, careful to get the right amount of tension (to test, push down on the belt away from the pulley. You should get less than 1/4 of an inch of play).
Step 6: Use your extension on the 10 mm socket to screw the tensioner bolt on the idler pulley clockwise, adding tension. Your bolt should be clear silver where it's been sitting within the pulley axle itself, so it should give you a general idea when to stop, and then you can fine-tune the tension when you're finished.
All done! Your new pulley has been installed! Start her up, and you'll find that grinding noise is gone! Unless you misdiagnosed the problem and it's another pulley.
Anyway, thanks to everyone here who gave me advice.