Not just paying a loan off early, even having the loan eventually drop off the credit report completely. This month the only change was an old, paid-off mortgage and line of credit dropped off my credit report. My credit score dropped substantially.