Though no longer an observatory, this building was named for one of Packer's oldest friends and influences, Robert Sayre. Sayre was the one to first bring Packer to the Episcopalian Church, a longtime partner in the Lehigh Valley Railroad, and an original member of the Lehigh Board of Trustees. In addition to this building, a group of dormitories is named for him.
In his 1869 speech on the origins and aims of Lehigh University, Bishop Stevens singles out Sayre as one of the most important influences for the school:
"Next to Judge Packer, the University is indebted to him, not only for his deep and thoughtful interest in the institution, but for the gift to it of one of the essential elements of its instruction and success; the Astronomical Observatory... The gift reflects special credit upon the large minded and liberal donor, whose name it will bear as the "Sayre Observatory" as long as the University itself shall stand."