For starters, you only need to check odd numbers (potential primes) below sqrt(X)√X.
 If A * B == X, then:
-  Either 
A == BandXis a perfect square, so the largest prime dividingAis the largest prime factor, -  Or one of 
AandBis less than the other, and thus less than thesqrt(X)√X. 
 Without loss of generality, say A is less than B. Then B would be greater than the sqrt(X)√X, but the largest prime factor in A or B would be the largest prime factor of X.
 So, you can start testing B, and just like X, you only need to test numbersonly  numbers less than the sqrt(B)√B, and when testing A only those less than the sqrt(A)√A.
 You can keep a list of numbers that divide X, I would always try to find a factor of the largest number that divides X:
- If it is prime, it is the largest prime factor.
 - But if you do find a factor of the largest, get rid of it and replace it with its two factors. Then once again, find the largest factor and prove it is prime or composite.
 
I would also start your loop for finding a factor "from the bottom," not from the top, to play the odds.
1/3⅓ of all numbers are divisible by 3, 1/5⅕ divisible by 5, etc. You can divide by 2 as many times as possible before beginning. Then keep track of the largest odd number you have tried (prime or not, that will include all primes), so once they fail, you don't need to try them again.