Data Engineer, Temporary

Working at Yale means contributing to a better tomorrow. Whether you are a current resident of our New Haven-based community, eligible for opportunities through the New Haven Hiring Initiative, or a newcomer, interested in exploring all that Yale has to offer, your talents and contributions are welcome. Discover your opportunities at Yale!

Overview

Position Focus: (3000 Characters):

This is a time-limited research position investigating how descriptive metadata might be derived from machine-generated text produced by handwritten text recognition (HTR) and optical character recognition (OCR) applied to digitized library and archival materials. The appointment has a fixed end date and is exploratory. It is not a production role. Work produced during the appointment is not expected to be loaded into any system of record.

The position is a partnership between the Metadata Services Unit, part of Resource Discovery Services, and Digital Collections and Access.

Yale Library is currently experimenting with AI models applied to machine-generated text produced as a byproduct of digitization. How that text can be turned into structured, reliable description is an open question. Current descriptive practice does not capture information at this level of granularity. It is not known which extraction approaches are viable, how accurate they are, what output they produce, or how that output could be represented in a catalog or archival management system. The purpose of this position is to investigate those questions and document what is found, including negative results.

Work will be conducted against exported machine-generated text carrying a known error rate rather than human-corrected text, so findings reflect realistic conditions. All source material is in the public domain.

Main responsibilities include:

Entity extraction and resolution: testing and evaluation (50%)

Identify and test approaches to extracting and resolving entities and concepts from machine-generated text, including names, roles, dates, and places. Design and run structured comparisons across approaches. Establish evaluation criteria, measure accuracy, and characterize error patterns. Build and run Python-based processing pipelines and, where appropriate, use models hosted on Yale computing infrastructure.

Analysis and processing of descriptive output (30%)

Characterize the results of extraction and resolution and assess their reliability by material and text type. Analyze how the resulting data would and would not map to current and emerging bibliographic and archival description standards and to systems such as Alma and ArchivesSpace. Compare extracted values against existing catalog and finding aid records. Evaluate and test semantic storage solutions for project data, including graph and vector databases. Identify where human review would be required before any output could be relied upon.

Documentation and reporting (20%)

Document methods, code, evaluation criteria, and results so the work is reproducible by others. Contribute to documentation of methodological practice. Prepare written summaries of findings and open questions and present results to project stakeholders. Participate in project meetings.

Work is carried out under the direction of the supervisor, in consultation with staff in Resource Discovery Services and Digital Collections and Access.

Required Skills and Abilities

1. Demonstrated proficiency in Python for data processing and analysis, including building and running pipelines to extract, clean, and transform data.

2. Demonstrated ability to design and carry out structured evaluations of computational methods, including defining accuracy measures, benchmarking results, and testing reproducibility.

3. Demonstrated ability to work with incomplete, inconsistent, or erroneous source data, identify patterns in errors, and assess the relative reliability of derived values.

4. Demonstrated ability to work independently with limited supervision on open-ended problems, exercise judgment where no established process or procedure exists, and adapt approach based on findings.

5. Demonstrated ability to document technical methods and results clearly and accurately for both technical and non-technical audiences.

Preferred Education, Experience and Skills: (400 Characters):

Experience with text recognition, OCR, HTR, or natural language processing tools. Experience working in high-performance computing environments. Familiarity with bibliographic or archival description standards. Interest in cultural heritage collections.

Physical Requirements: (1000 Characters):

Sustained concentration at a computer workstation for extended periods. Ability to sit for sustained periods.

Principal Responsibilities

1. Designs, plans, implements and evaluates complex multifaceted research projects/systems, including method development, adaptation and validation. 2. Collaborates with PI to define research endeavors and the development of research hypothesis and approach. 3. Solves complex methodology, protocol, procedural and research problems through design of techniques, procedures and policies that will achieve research goals. 4. Develops long range plans for supplies and equipment to ensure smooth operation of laboratory. 5. Investigates, analyzes and evaluates complex data, data collection systems and methods to reach scientific conclusions and ensures the integrity of research data. 6. Collaborates with PI to coordinate major lab renovations working with Facilities, Health and Safety, and Purchasing to ensure deadlines, safety standards and purchases are according to plans and within budgets. 7. Prepares scientific reports and papers for research proposals and published reports. 8. May perform other duties as assigned. Required Education and Experience Master’s Degree in a scientific discipline and three years of experience or an equivalent combination of education and experience.

Job Posting Date

09/02/2026

Job Category

Professional

Bargaining Unit

NON

Compensation Grade

Clinical & Research

Compensation Grade Profile

Research Associate 2 MS (24)

Salary Range

$68,000.00 - $120,500.00

Time Type

Full time

Duration Type

Temporary / Casual (Fixed Term)

Work Model

On-site

Background Check Requirements

All candidates for employment will be subject to pre-employment background screening for this position, which may include motor vehicle, DOT certification, drug testing and credit checks based on the position description and job requirements. All offers are contingent upon the successful completion of the background check. For additional information on the background check requirements and process visit "Learn about background checks" under the Applicant Support Resources section of Careers on the It's Your Yale website.

Health Requirements

Certain positions have associated health requirements based on specific job responsibilities. These may include vaccinations, tests, or examinations, as required by law, regulation, or university policy.

Posting Disclaimer

Salary offers are determined by a candidate’s qualifications, experience, skills, and education in relation to the position requirements, along with the role’s grade profile and current internal and external market conditions.

The intent of this job description is to provide a representative summary of the essential functions that will be required of the position and should not be construed as a declaration of specific duties and responsibilities of the position. Employees will be assigned specific job-related duties through their hiring department.

The University is committed to basing judgments concerning the admission, education, and employment of individuals upon their qualifications and abilities and seeks to attract to its faculty, staff, and student body qualified persons from a broad range of backgrounds and perspectives. In accordance with this policy and as delineated by federal and Connecticut law, Yale does not discriminate in admissions, educational programs, or employment against any individual on account of that individual’s sex, sexual orientation, gender identity or expression, race, color, national or ethnic origin, religion, age, disability, status as a special disabled veteran, veteran of the Vietnam era or other covered veteran.

Inquiries concerning Yale's Policy Against Discrimination and Harassment may be referred to the Office of Institutional Equity and Accessibility (OIEA).

Note

Yale University is a tobacco-free campus.