Lipids, in particular ceramides and diacylglycerols (DAGs), are implicated in insulin resistance (IR), however their precise roles remain unclear. Here, we leverage natural genetic variation to examine muscle lipids and systemic IR in 399 Diversity Outbred Australia mice fed either chow or a high-fat diet. Adipose tissue mass was significantly associated with 55% of muscle lipid features and whole-body insulin sensitivity, with DAGs as the only lipid class enriched in this association. To disentangle the contribution of adiposity and muscle lipids to whole-body insulin sensitivity, we employed two independent approaches: (1) a linear model correcting muscle lipid features for adipose tissue mass to assess their relationship with insulin sensitivity, and (2) stratifying mice into insulin sensitivity quartiles within adiposity bins. Both revealed that very long-chain ceramides, but not DAGs, were linked to IR. RNA sequencing and proteomics from the same muscles further associated these very long-chain ceramides with cellular stress, mitochondrial dysfunction, and protein synthesis. Meanwhile, DAGs correlated with leptin gene expression in skeletal muscle, suggesting they originate from contaminating adipocytes rather than myocytes per se. We propose that many muscle lipids, including DAGs, associate with muscle and systemic IR due to accumulation of adipose tissue rather than directly influencing muscle insulin sensitivity. By addressing the relationship between adiposity and metabolic state, we identified very long-chain muscle ceramides as being highly associated with IR independently of adiposity.