Many outpatient physicians and patients feel that current scheduling systems do not afford enough time for direct patient-physician interaction, leaving patients feeling unheard and physicians feeling demoralized. This dissatisfaction degrades patients' trust in the health care system and contributes to workforce moral injury and burnout. In the hopes of understanding the roots of this time stress and helping to guide future decisions about how to organize physicians' time, this article describes changes in the organization of U.S. outpatient physicians' time, starting from care at home in the late 19th century. It discusses the origins of the appointment system, which was invented to be highly personalized, with assistants adjusting appointment durations to accommodate clinical activities, specific patient needs, and individual physician proclivities. The article then describes how centralization of appointment scheduling became more common as U.S. medicine became increasingly consolidated into larger and larger groups and health systems. This distanced schedulers from the people and care they were organizing and necessitated standardized appointment durations, which did not accommodate individual patient and physician needs. With the rise of managerialism, schedulers became increasingly accountable to administrators rather than patients and physicians. Whereas early appointment systems depended on personal connection between schedulers and the physicians and patients they supported, today's schedulers have few such interactions. The widespread shift to centralized scheduling and standardized time slots has contributed to misalignment among time allocation, patient care, and health care workforce well-being and is likely exacerbating ongoing tensions among patients, physicians, and administrators.