Abstract Objective Pain is the primary limitation to performing hysteroscopy. We aimed to evaluate the predictive factors of low tolerance to office hysteroscopic procedures. Methods Retrospective cohort study of the patients who underwent office hysteroscopy from January 2018 to December 2020 at a tertiary care center. Pain tolerance to office-based hysteroscopy was subjectively assessed by the operator as terrible, poor, moderate, good, or excellent. Categorical variables were compared with the use of the Chi-squared test; an independent-samples t-test was conducted to compare continuous variables. Logistic regression was performed to determine the main factors associated with low procedure tolerance. Results A total of 1,418 office hysteroscopies were performed. The mean age of the patients was 53 ± 13.8 years; 50.8% of women were menopausal, 17.8% were nulliparous, and 68.7% had a previous vaginal delivery. A total of 42.6% of women were submitted to an operative hysteroscopy. Tolerance was categorized as terrible or poor in 14.9% of hysteroscopies and moderate, good, or excellent in 85.1%. A terrible or poor tolerance was more frequently reported in menopausal women (18.1% vs. 11.7% in premenopausal women, p = 0.001) and women with no previous vaginal delivery (18.8% vs. 12.9% in women with at least one vaginal birth, p = 0.007). Low tolerance led more often to scheduling a second hysteroscopic procedure under anesthesia (56.4% vs. 17.5% in reasonable-to-excellent tolerance, p < 0.0005). Conclusion Office hysteroscopy was a well-tolerated procedure in our experience, but menopause and lack of previous vaginal delivery were associated with low tolerance. These patients are more likely to benefit from pain relief measures during office hysteroscopy.