Mitochondrial dysfunction has been implicated in the development of insulin resistance (IR); however, a large variety of association and intervention studies as well as genetic manipulations in rodents have reported contrasting results. Indeed, even 39 years after the first publication describing a relationship between IR and diminished mitochondrial function, it is still unclear whether a direct relationship exists, and more importantly if changes in mitochondrial capacity are a cause or consequence of IR. This review will take a journey through the past and summarise the debate about the occurrence of mitochondrial dysfunction and its possible role in causing decreased insulin action in obesity and type 2 diabetes. Evidence is presented from studies in various human populations, as well as rodents with genetic manipulations of pathways known to affect mitochondrial function and insulin action. Finally, we have discussed whether mitochondria are a potential target for the treatment of IR.