I would frown on using emory cloth, and so would my bosses. We use a wire wheel for slight cosmetic imperfections, or just regrind the cam really quick. If you're an Ace cam grinder like my bosses and ouu top grinder of 33 years, Floyd, they can grind ONE lobe without re-indexing the entire cam. It would only take a few millionths of an inch to clear up those blemishes (I know, I'm exaggerating on my fractions there).
As for the valves not touching at that particular point, I would have to (don't hate me!) disagree with ya there. I've not seen a cam yet that where the lifters, rockers, or valves (depending on the cam design) didn't hit the entire circle of the cam (until they get worn out). And although many would assume the top of the lobe is the most critical, in the grinding business it's actually the base circle that is most critical. We have to grind them to within a certain tolerance and keep checking them with a digital caliper. Many manufacturers have exact "base circle minimums" allowed, many don't. Depends mostly on if the cam is solid or hydraulic.
But back to your problem. I just don't believe it is a good idea to use emory cloth on your lobe. Myself, I would go to any local cam grinder and have the lobe reground. This way, it will be microscopically even all the way across the lobe, and there will be no imperfections not viewable to the naked eye.
Just my own, worthless opinion. I'm certainly NOT a camshaft NOR motorcycle expert! Just a cam grinder who happens to ride!
Good luck! And YES, that sucks!
Need professional advice? Call Delta Camshaft in Tacoma, WA and ask for Jon or Scott (owners) or Floyd (top grinder; only does custom work and specializes in Pro Drag Race and Salt Flats grinds). One of these friendly folks can help ya fer sure! You can even tell them Shawn said to call them (they don't know me as Hairball but Jon does from my MySpace). See what THEY suggest before doing anything you'll regret later (like going back to old cams).
1-800-562-5500 between 8am & 4:30pm Mon-Fri PACIFIC time.