The gut microbiota plays a crucial role in the human body and has an impact on the physiological function of the host. In particular, changes in the gut microbiota are especially pronounced in patients with type 2 diabetes. There is ample evidence that glucose-lowering drugs exert their therapeutic effects precisely through their interactions with the gut microbiota, but there is a lack of summarization. An overall comparison of the effects of each hypoglycemic agent on the gut flora can provide new inspiration for combinations. In this paper, we selected several representative glucose-lowering drugs, such as metformin, sodium-dependent glucose transporters 2 inhibitors (dapagliflozin), glucagon-like peptide-1 receptor agonists (semaglutide, liraglutide), and the traditional Chinese medicine (berberine), and illustrated how they can affect the disease process by regulating metabolic homeostasis, immune response, and gut barrier. We found that each of these four hypoglycemic agents can have conflicting effects on the gut flora depending on the timing and mode of administration. Meanwhile, the potential impact of gut microbiota on drug safety is explored, and an outlook for the optimization of future type 2 diabetes treatment regimens is presented.