Part 2 – Standards and Evaluation (UCAPT)

2.1 Expectations and Standards for Appointment, Promotion, and Tenure

The primary factors considered in appointment, promotion, and tenure decisions are excellence and creativity in both scholarly research and teaching, as documented in the dossier, with outstanding performance required in one—almost always research—and strong performance in the other. The University values scholars who have made important and original contributions, who have had an impact on their field, and whose work shows a clear arc of intellectual and creative development.

Every appointment, promotion, and grant of tenure should meet the national and international standards of the leading institutions, as well as improve the overall stature of the academic unit. A candidate’s scholarly or artistic work should be widely perceived among peers as outstanding, and the candidate should have a strong reputation in his or her field.  The candidate should also be viewed as instrumental in advancing the academic needs of his/her unit.

Each candidate is considered individually. Multiple candidates from the same department or similar disciplines are not compared to each other.  USC does not have quotas restricting the number of tenured appointments.

The University welcomes innovative approaches to scholarship and encourages faculty members to stay at the cutting edge of their field.  It recognizes and supports a variety of styles of scholarship, both independent and collaborative.

Expectations for scholarship do not primarily concern quantity, although the University shares with other leading institutions expectations about productivity. Expectations and metrics for productivity vary by field. For example, candidates in fields that emphasize book production should have a book or books published or in press by a university press or press of equivalent standards and reputation (preferably with published reviews). Candidates in article-producing fields should have a sufficient mass of articles in high-impact journals. Candidates in grant-funded fields should have an independent research program as principal investigator with a sustained record of substantial peer-reviewed external funding from federal agencies. Schools and departments are expected to submit for Provost approval field-specific metrics (see Section 2.3.1).

2.1.1 Candidates for Tenure

A candidate for tenure is expected to have produced significant original scholarly contributions (that are explained in the dossier). He or she should have produced a substantial body of work, given the expectations of the field.  The work should have had a significant impact on the field.  The arc of scholarship should show promise of continued productivity.  Overall, the record of scholarly contributions should be on par with the accomplishments at the tenure stage of the discipline’s leading scholars.

A candidate should have a program of scholarship independent from his or her Ph.D. supervisor or post-doctoral mentor. If the bulk of the candidate’s research is done jointly (especially if it is done with senior and more established scholars), the record should provide evidence of the candidate’s important original contributions.

The University aims to tenure those individuals who show promise of becoming nationally and internationally recognized during their careers. A candidate is expected to be a good teacher and a good university citizen, but it is primarily upon the significance and influence of the candidate’s research, as well as their promise of continued productivity that suitability for tenure will be judged.

2.1.2 Candidates for Promotion to Full Professor

Promotion to full professor is based on achievement rather than promise. The candidate should have compiled a significant record of accomplishment and impact in his or her field, and made substantial contributions beyond those that earned tenure. The post-tenure body of work is examined alongside the pre-tenure body of work to discern the candidate’s career trajectory and to evaluate whether he or she will continue to produce research at a rate and of a quality commensurate with leaders in the field.

The candidate for full professor should have achieved recognition and distinction in his or her field at a national and international level. The candidate’s work should be comparable in significance and impact to the work of newly promoted full professors at leading departments where work of the same type is undertaken.

Candidates for full professor (and tenured faculty members as a group) also have special responsibilities for mentoring junior faculty and for leadership in service and governance on the departmental, school, and university levels. They are expected to excel as teachers and mentors of students, which in many fields includes successful mentoring of Ph.D. students.

In some disciplines, leadership in application of research to societal needs may be an important part of the evidence presented.

Recognizing the University’s support of interdisciplinary and collaborative scholarship, associate professors (and candidates for full professor) are encouraged to take advantage of the freedom afforded by tenure to pursue their scholarly interests whether they fall within or across traditional disciplinary boundaries. This freedom also allows for more risk-taking and creativity in scholarly activities. Although such innovative efforts are not required, they are considered a positive as the University seeks to encourage creative research.

2.2 Time Period

UCAPT considers the individual’s entire body of work.  For candidates already at USC, it looks especially at work completed since the individual was appointed or previously promoted at USC.

In unusual instances, an outstanding new faculty member may be recommended for tenure or promotion during or at the end of his or her first year.  In this case, the original dossier may be resubmitted with clear evidence of continued achievement and collegial activity.

2.3 Assessing Research Quality and Impact

2.3.1 Overview

The most critical factor in appointment, promotion, and tenure cases is the quality and impact of a candidate’s work. UCAPT and reviewers at other levels base their assessments of quality and impact on a variety of factors, including direct reading of the work, the quality of publication venues, the quantity of work, the influence shown by citations, the external peer review expressed in scoring and funding decisions by Federal agencies and organizations known to have high standards, and confidential reviews by external scholars.   Supplemental evidence can consist of editorial appointments or leadership in professional societies, awards and honors, and reviews published in scholarly outlets and important popular media.  There is no formula by which these factors are combined, and assessments of quality and impact cannot be reduced to a number, such as number of publications or citations.  Reviewers consider the record as a whole, giving weight to different factors as is appropriate to the case.

Each school or department is expected to propose school- or department-specific measures and expectations of productivity to be taken into account in appointment, promotion, and tenure decisions. These metrics and expectations should match those of leading institutions. To be official, they must be approved by the dean and Provost, and they must not contradict the UCAPT guidelines. They should be made available to candidates and included in the dossier after formal approval from the dean and Provost.

Internal reviewers at all levels, from a candidate’s department to UCAPT, base their assessment in part on direct examination of the work. For example, in a field where research is typically disseminated through journal articles, internal reviewers read a selection of articles. Most departments and schools utilize a faculty committee to produce a report that summarizes the nature and importance of the candidate’s research.

2.3.2 Quantity and Venue of Publications

All dossiers should include information on the quantity of work produced by a candidate and the venues where that work is published. The nature of the information provided should be appropriate to the field. For example, in fields where research is commonly published in scholarly journals, the number of articles should be reported. In some fields, it is useful to differentiate publications in leading journals from less-respected journals.

Appointment, promotion, and tenure decisions are not a matter of meeting numeric targets. However, contribution and impact generally benefit from cumulative quantity. The arc of productivity is helpful evidence of future promise.  The amount of intellectual output plays a role in tenure and promotion decisions because it is indicative of productivity and stature in the eyes of peer reviewers. If there is less than the usual quantity of work, questions are raised.

Evidence of editorial peer review is highly salient and publication in a field’s most respected venues is an indicator of the quality of work. UCAPT considers the quality and selectivity of journals or publishers in developing a picture of the quality of the intellectual output of a candidate.

While publications by the candidate in the form of book reviews, encyclopedia and review articles, edited volumes, and chapters in edited volumes may add to the candidate’s visibility, they are not regarded by UCAPT as significant evidence of scholarship, and they are usually not the best use of the candidate’s energies unless their special significance is explained.

2.3.3 Citations

In fields where citations are viewed as an indicator of research impact, the dossier should include information on the candidate’s citation frequency, and contextual information on citation norms in the field. This would be the case for most social and natural science fields, as well as many humanities fields. In fields where citations indexes (such as the H-index) are believed to be an indicator of impact, that information is also considered.

2.3.4 Artistic and Creative Work

For candidates in artistic fields, scholarly production often takes the form of creative work. The dossier should demonstrate that the candidate’s creative work is widely perceived among his or her peers as outstanding. In artistic fields, the candidate’s creative products should gain recognition equivalent to the expectations of scholarship in other disciplines.

Additionally, the dossier should detail discipline-specific standards, practices, and measures of impact. Artistic exhibitions and cinema festivals, for instance, typically have their own forms of peer review; departments should supply detailed information about the peer review process.

The dossier should also provide information on the quality, selectivity, and stature of a candidate’s performance venues, where appropriate.

The candidate’s reputation in the field can be documented through invited talks, shows, performances, and the like, as appropriate for the discipline.

Overall, the evidence should show that the candidate’s artistic output is comparable in quality, originality, and stature to those recently granted tenure in similar genres in the top departments in the nation.

2.3.5 Honors and Awards

Most fields honor individuals or specific research contributions. Examples include best paper prizes for journals or conferences, and emerging scholar awards or career awards from professional societies or Federal agencies. Such information can be an important factor in assessing quality and impact of research. In order to put honors and awards in context, the dossier should explain the importance of a candidate’s awards, how exclusive they are, how the winners are selected, and so forth.

2.3.6 Conferences, Patents, and Other Forms of Scholarship

The significance of conferences varies from discipline to discipline. Presenting papers at conferences can be useful in publicizing emerging research, establishing one’s reputation in the field, and other worthwhile goals. Similarly, invitations to present talks to faculty groups at other universities indicate interest in the candidate’s research by outside experts. Computer science is recognized as a special case where many scholars regard published conference papers from top conferences as equivalent to journal articles, but most fields do not rate them as highly.

While patents cannot replace peer-reviewed publications in a candidate’s dossier, they are a sign of impact and productivity and will be considered accordingly.

2.3.7 Impact on Practice and Society

In some disciplines, evaluation of the impact of publications and scholarly work can include not just the impact on other scholars, but also the impact on the practice of the profession, public policy, or the workings of institutions. Sometimes the candidate’s work results in new organizations or new products and services. These activities are not a substitute for peer-reviewed publications, but they can be evaluated as additional measures of the impact of the candidate’s scholarly contribution.   That a candidate’s research was featured or widely discussed in popular media may be documented in the dossier, but in itself may not be useful evidence of impact.

2.3.8 Grants and Fellowships

For faculty members in grant-funded fields, information on the type and amount of external funding can be a useful indicator of the quality of work. The number of dollars awarded in grants and the type of indirect cost recovery are not themselves significant. Rather, it is the rigor of the peer review of federal agencies and similar funding sources that is significant. Thus, it would be significant that a federal grant proposal received an excellent score, even if it was not funded because the agency appropriation was cut. On the other hand, even a very large grant that is awarded without equivalent peer review provides less useful information.

In many fields, of course, grants are not relevant. However, in some areas grants are necessary to provide the resources needed to conduct research. In medical fields, for example, it is usual to consider such questions as: Has the candidate received an R01 (or equivalent grant) as P.I.? Has the grant been awarded a competitive renewal? Has the candidate received a second R01 as P.I.? Has the candidate had consistent federal funding?

Grant expectations vary by field; departments and schools can indicate the typical expectations by field through the dossier cohort analysis, and can explain to UCAPT its significance.

2.3.9 Peer Reviews

All dossiers contain confidential letters from external reviewers who are leading experts in the candidate’s field. See Section 8.8 for information on how those reviewers are selected. These letters are a very important factor in assessing the quality and impact of a candidate’s work, as well as the candidate’s external reputation.

For books and creative work, published reviews in leading outlets can provide useful information on the quality and impact of work. The fact of being reviewed in a leading outlet itself can be an indicator of quality or importance, and the lack of reviews in leading outlets may suggest the work is of limited importance. Most important is the substance of the review.  If books appear late in the probationary period, there may not be enough time to obtain reviews.

2.3.10 Editorial Positions and Leadership in Professional Societies

In many fields, an appointment to serve as editor or on the editorial board of a leading journal indicates that a candidate is viewed by his or her peers as a leading expert. Such information plays a role in assessment of quality and impact of work, and of a candidate’s reputation in the field. Assessments based on such information take into account the prestige of the journal, the selection process, and the precise editorial role.

Similarly, appointment or election to a leadership position in a professional society may also be an indication that a candidate is viewed by peers as a leading expert, and such information can play a role in assessing quality and impact of a candidate’s research insofar as the appointment is based on research accomplishments.

Invitation to present at a Gordon Research Conference is, in many fields, a valuable indication of a candidate’s scholarly reputation.

2.3.11 Digital Scholarship

“Digital scholarship” refers to all forms of research, analysis, and publication that are conducted in digital formats and distributed via the Internet or by other means. No single definition of digital scholarship can encompass all forms of activity. Digital scholarship can range from new ways to publish otherwise traditional texts to “born digital” multimedia and interactive works that are impossible to publish in print form. The term may also cover digital databases or repositories; platforms enabling the conduct or publication of research; the infrastructure enabling access, searching, analysis, and publication; cloud computing; meta-analyses across multiple databases; distance collaborations; and many other forms of scholarship that have been made possible by digital technologies.

UCAPT welcomes innovative approaches to scholarship and strives to evaluate digital scholarship through evidence of contribution, impact, peer review, and creativity. It reviews “born digital” scholarship by viewing the work in its context and taking into account the contribution of the work’s medium or form. Mentors and those responsible for assembling dossiers should ensure that a faculty member’s creativity and impact within the field are demonstrated within the context of the field. For venues where the peer review process and impact factor are not evident, departments should submit such explanatory information as the ratio of submissions to acceptances, the stature of others who publish in that venue, the stature of the reviewers or editors, and any other measures of the influence of the venue. If a digital publication is not itself peer-reviewed, its quality might be evaluated, for instance, through any peer-reviewed funding it receives or its connections with significant publications in peer-reviewed scholarly journals. Other evidence of the work’s impact might be its inclusion in university syllabi, electronic archives, and recognition networks.

2.3.12 General Comment Regarding Explanatory Information in the Dossier

When the significance and impact of items in the candidate’s dossier may not be immediately apparent to UCAPT, the department should supply additional information about these items. For instance, if a candidate’s creative work is selected for a certain prize or festival, the department should supply information as to the ratio of submissions to acceptances, the stature of the judges, and the stature of other winners or participants. If a candidate publishes in non-peer-reviewed venues, the department should detail the ratio of submissions to acceptances, the stature of the editors or reviewers, the stature of other authors in that venue, and measures of the venue’s impact.

Discipline-specific standards and practices should also be explained. The significance of the sequence of authors in collaborative publications, for example, varies by field. In many fields, it is assumed that first and senior authors should receive the most credit; unless specific information is provided, there may be an assumption that other authors have not made major contributions. For candidates who engage in collaborative research, departments should explain the field’s practice for the sequence of authors.

Those preparing the dossier should avoid any temptation to suppress unfavorable information out of concern that UCAPT will not understand it or give it too much weight.  Instead, the information should be presented candidly along with an explanation.

2.4 Assessing Research Independence

Candidates for appointment, promotion, and tenure must demonstrate a program of scholarship independent from their Ph.D. supervisors or post-doctoral mentors, and their record must provide evidence of original intellectual contributions to collaborative projects. The University supports both independent and collaborative work. In some fields collaborative work is the norm. In evaluating a dossier with collaborative work, UCAPT looks to distinguish the intellectual contributions of the candidate.

In some fields, the new faculty member’s early publications will be outgrowths of the Ph.D. dissertation. In such cases, there should be publications that show the candidate’s further intellectual growth.

If the preponderance of a candidate’s research is collaborative, one way that the nature of the candidate’s independent contribution is assessed is through confidential letters from collaborators. The candidate’s personal statement can also play an important role in identifying the nature of the candidate’s independent contribution to joint work. Candidates are encouraged to provide this information in the personal statement if some of their work is collaborative.

It is usually assumed on grants that the intellectual leadership is provided by the principal investigator (or, when explicitly recognized by the granting agency, equal co-principal investigators). The investigator responsible for a separately scored portion of a large grant is typically credited with that portion.

For collaborative work in multidisciplinary teams, a candidate should demonstrate evidence of his or her unique and original contribution to multidisciplinary teams. The National Institutes of Health criteria state that participants in team research can demonstrate this evidence through “independent publication of methodological or seminal contributions to the candidate’s specific research area; where possible, explicit in-print acknowledgment of unique creative contributions in multi-author publications and/or selection for presentation of team findings at national and international scientific conferences; members of research teams should demonstrate peer recognition of their specific contributions and some publications should highlight their distinctive research; creative and unique contributions to team productivity should be documented.” A candidate who conducts collaborative research should make clear in the personal statement and on the CV what his or her specific contributions were to the collaborative work.

2.5 Assessing Research Trajectory

Another factor in appointment, promotion, and tenure decisions is the trajectory of a candidate’s research. The university seeks to appoint, promote, and tenure scholars who will continue to produce high quality and impactful research and maintain a strong professional reputation in the future. There is no required profile for work over time, but large gaps in the production of research may raise questions. A slowing of research activity over time, or a sudden burst just before the tenure decision, may also raise questions. A candidate should use the personal statement to anticipate and address any questions that might arise about his or her research trajectory.

2.6 Assessing Teaching and Mentoring

All candidates for appointment, promotion, and tenure are expected to be strong teachers. The candidate’s teaching must demonstrate commitment to students, and in some fields successful mentoring of doctoral candidates is expected. It is laudable if a candidate for tenure demonstrates excellence and creativity in teaching. On the other hand, devotion to teaching and mentoring should not be allowed to take away the time necessary to publish the expected scholarly work and obtain necessary funding.  For appointments and promotion to full professor, evidence of excellence in teaching and mentoring is expected.

Teaching quality is assessed based on a number of factors.  The best evidence comes from peer assessments, demonstration of students’ learning achievements, utilization of exemplary teaching methods, and inspection of syllabi and class materials.  Other evidence may include when appropriate: student ratings from classes taught, teaching awards and honors, and the candidate’s personal statement. As with other assessments, no single factor is determinative, and assessment involves a thoughtful weighing of multiple factors as appropriate to the case.

2.7 Interdisciplinary Work

2.7.1 Candidates with Joint Appointments

The University welcomes work that spans traditional disciplines. For candidates with greater than zero-percent joint appointments, UCAPT will automatically consider their work to be interdisciplinary.   However, assistant professors on the tenure track are discouraged from having joint appointments of more than zero percent.

It is desirable that the departmental and/or school committees for candidates with greater than zero-percent joint appointments should include one or more appropriate members from the secondary department or school. Similarly, it is desirable that advice be sought from these colleagues on the selection of reviewers from other disciplines, as well as reviewers who share the candidate’s interdisciplinary focus, and that in addition, one or more appropriate senior members in the other discipline be asked to provide letters of evaluation concerning the candidate’s interdisciplinary work. All evaluations from other departments or schools should be included in the dossier before its final consideration by the home department.

The secondary department or school does not vote on the tenure, promotion, or appointment dossier, and the candidate does not have to satisfy the requirements of two departments or schools.

The Faculty Handbook has long provided that tenure is held in the school, and in suitable cases a school may explicitly propose that the award of tenure be in the school rather than any individual department.

2.7.2 Interdisciplinary Candidates without Joint Appointments

If a candidate wishes to be identified as interdisciplinary but does not have a greater than zero-percent joint appointment, either the individual or the home department may send a memo to the dean requesting that he or she be identified as interdisciplinary in the tenure, promotion, or appointment process. This memo should be sent before the beginning of preparation of the dossier. If the dean agrees, he or she should alert the Provost’s Office that the dossier is interdisciplinary.

2.7.3 Evaluating Interdisciplinary Work (Department and School Level)

Department and school committees evaluating interdisciplinary work should strive to value appropriately publications outside the home discipline and its usual journals. In evaluating the candidate’s teaching and mentoring activities, they should consider interdisciplinary graduate teaching and co-teaching, as well as advising or co-advising graduate students outside the home department. The committees should make special effort to understand other disciplines’ customs on co-authorship, sequence of authors, and the use of conferences, journals, or monographs as premiere outlets.

2.7.4 Evaluating Interdisciplinary Work (UCAPT)

UCAPT will use appropriate flexibility in reviewing interdisciplinary dossiers, assigning a dossier to a disciplinary panel, a mixed panel, an ad hoc committee, or using ad hoc members as needed.

2.7.5 Mentoring Interdisciplinary Faculty Members

It is desirable that an interdisciplinary candidate have mentors in all appropriate units, who work together to give the candidate a consistent message about research and publications, as well as guidance on how to avoid excessive burdens of teaching and service. For candidates with appointments in more than one unit, a Joint Appointment Checklist must be approved so that workload expectations are clear. It is also desirable that the ways interdisciplinary excellence will be evaluated (either as set out in the school clarifications or as individually agreed) are made available to the individual at the time of appointment, or early in the candidate’s probationary period.

The mid-probationary period review committee for interdisciplinary candidates should include a member from the other discipline(s) (see section 3.3).

If interdisciplinary work requires a substantially longer start-up time than research in a single discipline, a request may be made to the Provost, early in the probationary period, to consider an extension of that period (see section 3.4). Such a request should include the recommendations of each of the relevant department chairs and deans.

2.8 International Scholarship and Teaching

Department and school committees should consider faculty members’ participation in significant international activities: teaching and research abroad, as well as service to distinguished foreign institutions and students. Work conducted overseas or in conjunction with overseas organizations may be less visible than work done on campus or domestically. Nevertheless, such efforts should be evaluated and accorded reasonable weight in promotion, tenure, and appointment decisions.