He joined and gained tenure in the History Department at the University of Chicago. Hannah Holborn Gray followed her husband to Chicago, spending a year as a fellow in the Newberry Library, but on the margins of the university that was ultimately to offer her its presidency. She considered going to law school. Finally, the History Department made Hannah Gray an offer, abolishing its nepotism rules to secure its first academic couple. She was granted tenure in 1964.

Asked to redesign the history program, Hannah Gray got her first taste of administration. But it was her service on a faculty committee to review the non-reappointment of a controversial woman faculty member whose dismissal had provoked a major student sit-in that catapulted Hannah Gray into the limelight at Chicago. Demonstrating the powerful combination of rational discourse and steel nerves that was her trademark, Hannah Gray helped to resolve the explosive conflict. In 1972 she became the dean of the College of Arts and Sciences at Northwestern University. There she contributed to planning and to the strengthening of academic programs.

In 1974 the president of Yale, Kingman Brewster, who knew of Gray from her service on the Yale board, named her Yale's first woman provost. Those were dark days for Yale, including a decline in finances and a debate over coeducation. In 1977, when Kingman Brewster became U.S. ambassador to Great Britain, he chose Gray to succeed him as acting president. She never winced at making tough decisions, particularly in paring down a large deficit and taking a hard line on contract negotiations with Yale's striking cafeteria and maintenance workers. Whether it was the effects of the protracted strike which alienated many, or simply her gender, Gray did not receive the offer of the Yale presidency, which went instead to Bartlett Giamotti, with whom she had taught a course in Renaissance history. Almost overnight she was invited to become president of the University of Chicago.

According to Gray, the offer came as a "great surprise." While she considered Yale her "second home," she was helped in her decision to return to Chicago by the fact that Charles Gray felt like a "well-connected anamoly" at Yale. The Grays returned to the University of Chicago, where he held an appointment in history and she presided over a research university of 7,900 students, a faculty of over 1,000, and an endowment of $350 million. She was the first woman to serve as the chief executive of a major coeducational university.

Her presidency spanned a time of economic decline in academe, especially a contraction in the numbers of graduate students, the hallmark of the intellectual life at Chicago. A major achievement of her administration was the formation of a faculty committee to study the future of graduate education in the United States. Gray espoused a concern for limits, but without thinking that "limitations are necessarily negative." Facing a deficit budget, she continued via a successful fundraising campaign to spend money to maintain quality in academic programs and to attract faculty. Once again confronted with a student protest, this one precipitated by the granting of a controversial award to former Secretary of Defense Robert McNamara, Gray held her ground and delivered the award, but then appointed a faculty committee to consider abolishing the prize entirely.

There were sometimes contradictory impressions of Gray—to some she seemed cold and ruthless; to others warm, witty, and effusive—but even her critics recognized her longstanding commitment to academic values. "She has spent her life steeped in them," said one observer.

Gray was a fellow of the American Academy of Arts and Sciences, a member of the American Philosophical Society, and a trustee of Bryn Mawr College. She was a member of the Pulitzer Prize board, the Council on Financial Aid to Education, and the Council on Foreign Relations in Chicago and New York. She served on the board of directors of J.P. Morgan & Company/Morgan Guaranty, Atlantic Richfield Company, and Ameritech. She held honorary degrees from 42 colleges and universities, received the Medal of Liberty in 1986 and the Medal of Freedom in 1991.