You are definitely correct to replace the CPU fan at any sign of trouble. CPU fans are cheap, CPU's are expensive!
1. A good sized heat sink.
2. A fan and heat sink combination that will fit your PC case - a lot of PC cases are very cramped.
3. BTU rating: How much heat can the CPU fan dissipate?
4.
MTFB: How long should this CPU fan run before failing -- on average.
5. Quietness: How annoying is this fan?
6. Fan speed. Is it adjustable?
7. Bearings. A fan which uses bearings will last longer and be more reliable.
Don't forget these other items:
1. Thermal paste: Air space between the CPU and the heat sink will cause heat build up.
2. Case cooling: Keeping the CPU cool starts with a reasonable case temperature.
The CPU fan manufacturer I would recommend is Cool Max Technology.