The excess risk of cancer observed in patients with type 2 diabetes (T2DM) may have been influenced by detection bias. The aim of this study was to examine the real association by evaluating time-varying site-specific cancer risks in newly diagnosed T2DM patients. A total of 51,324 registered cancer-free individuals newly diagnosed with T2DM between 2004 and 2014 were linked with the Shanghai Cancer Registry and the Vital Statistics through September 2015. A total of 2920 primary, invasive cancer cases were identified during 325,354 person-years period. Within 1 year following diabetes onset, participants with T2DM had higher risks of total, lung and rectal cancer in men and total, liver, pancreas, thyroid, breast and uteri cancer in women. Thereafter the incidence for overall cancer decreased and then increased along with follow-up time, with the upward trend varying by cancer, suggesting potential detection bias. After the initial 1-year period, standardized incidence ratios (SIR) and 95% CIs for overall cancer were 0.80 (95% CI 0.76–0.85) in men and 0.93 (95% CI 0.88–0.99) in women, but a higher risk of breast and thyroid cancers were observed in women, with SIR and 95% CI being 1.13 (1.01, 1.28) and 1.37 (1.11, 1.63), respectively. Our results suggest that T2DM patients are at higher risk of certain cancers; this risk particularly increases shortly after diabetes diagnosis, which is likely to be due to detection bias caused by increased ascertainment. Prevention of female breast and thyroid cancers should be paid attention in Chinese individuals with T2DM.