Phosphor converted LEDs are the most preferred method to generate white light and can exhibit greater variability of color than other color mixing methods, such as multiple chips color mixing. Even with the same phosphor materials, various color temperatures can be achieved by adjusting the density of the phosphor particles or the thickness of the phosphor binder layer. Color stability over the life time is a key characteristics of LED package, the core of various rapidly emerged solid state illuminations. LED with color shift can be as useless as the ones whose luminous flux output drops below an acceptable threshold. However, the underlying physics that caused the color shift of high power LED packages is unknown. Degradation of color stability may not matter for some applications, but for applications that involve visual demanding, such as museum, medical optics and demonstration lightings, it is as crucial as the demand of luminous flux. LM 80 is the standardized test method for LED packages published by IESNA (illuminating Engineering Society of North American) to measure the luminous flux and color coordinates under several different test conditions (55°C, 85°C and a third temperature) with external biased sources. IESNA also released the TM-21 lumen maintenance projecting method to extend the luminous flux from the test time to desired time. LM 80 test standard only includes the thermal stress and current stress. Considering about the application of automobile lighting and outdoor lighting, relative humidity should also be included in the standardized test. Also, there are no similar method like TM21 that can be used to project the color shift distance forward and estimate the remaining useful life based on color criteria. In this paper, a physics based model of color shift distance is proposed according to the experimental data. In the proposed model, not only temperature and humidity are included, but also the package characteristics, such as phosphor particles diameter, phosphor binder thickness. During the experiment, three different kinds of high power warm white LED packages are used. Each kind of LED package has a representative and unique package structure. During the test, LED packages are placed in a temperature and relative humidity controlled chamber (65°C/90%, 75°C/75%, 85°C/85%) and driven with external current sources (350mA). At each readout time, LED are taken out of the thermal chamber to cool to room temperature for both colorimetric and photometric measurements. In high temperature and high humidity environment, the color stability of higher power LED package starts to degrade at a very high speed. The phosphor particles start to oxidize and the oxidized phosphor particles shift the emission spectrum to the lower energy wavelength. The redistribution and oxidation of phosphor particles inside the phosphor binder will cause dramatically degradation of colorimetric properties of LED package in a very short time. A color shift distance model based on package structure, temperature, humidity is proposed according to the test data. Also, the physics of failure due to material degradation has been correlated with the change in the photometric and colorimetric parameters of the LEDs.