Conference Blog


Day 1


Opening Address:  Neuroscience, Inspiration, and Purposeful Lives

Dr. Mary Helen Immordino-YangDr. Mary Helen Immordino-Yang

Dr. Mary Helen Immordino-Yang

Dr. Mary Helen Immordino-Yang kicked off the conference with her talk, “Embodied brains, social minds: The importance of social reflectiveness and emotional awareness in young adult development.” She discussed the social and emotional abilities of young people and suggested that these are critical not only to learning but to the world and our future. She entitled her talk, “Embodied brains, social minds” because our complex, human-developed minds are inherently social minds. The intellect that we have does not stand independent from the rest of lives- we collaborate and can use our knowledge to solve problems that impact us all.

Dr. Immordino-Yang described the brain as an amazingly complex system that works together to make a human being who can be a whole, real person- not just capable of “one off” functions. “We think of the brain as a separate thing from the social world- an independent thing that is inside your head that operates in isolation…. but nothing could be further from the truth.” She described her use of powerful individual stories to gain information about social emotions. Upon hearing a story of an extraordinary person, one recognizes their uniqueness and reflects on their own opportunity to take similar extraordinary actions. According to Immordino-Yang, “Feeling social emotions…involves the neural mechanisms for feeling and regulating your own body and for constructing your own sense of “self”.”

To illustrate the point, Dr. Immordino-Yang showed a video of a subject responding to a powerful story. The subject described sensations in his body- which is a sign of information. There is, however a lot of variability in how people feel and describe what they feel. These signals are indicators of how the person is processing the story or event. Dr. Immordino-Yang highlighted the subject’s pause in his speech- noting that this is an indicator that he is turning off his attention to the world- and then he came back with evidence that he had turned their attention inward, to his own memories, and revealed a new and meaningful lesson that he had built for himself.

The presentation continued with images of the brain and an overview of the functions of different areas. A statistical map superimposed on the brain showed that the blood-flow in certain regions of the brain increased when people indicated that they felt “moved”. She made the connection between the neural platform that allows us to recognize “spoiled food” as the same that makes us “sick to our stomachs” upon hearing something upsetting or repugnant. According to Immordino-Yang, “Emotions (and their primitive counterparts, biological drives) serve to keep organisms alive and living comfortably. Human “survival” has become a complex social and cultural construct.” What makes us survive is the meaning that we make out of our social world and purposefulness.

Dr. Immordino-Yang summarized her presentation with the following, “Learning, in the complex sense in which it happens in schools or the real world, is not a rational or disembodied process; neither is it a lonely one.” She shared the following website as an opportunity for teachers to learn more about neuroscience and the classroom.


Day 2


Keynote Address: Assessing Non-cognitive Variables: Issues and Applications

Dr. William Sedlacek

Dr. William Sedlacek

Dr. William Sedlacek began the morning with an examination of noncognitive variables. He described some of the measurement reasons why we need to look at something else. Specifially, he suggests that we have the statistical problem of restriction of range. We don’t really know how to measure attributes any better. We also have the issue of grade inflation. Some places have almost everyone getting 4.0s on a 4-point system. Even taking more courses in a certain area doesn’t reveal new information- at a certain point, you just top out.

We need to examine a range of things that can be considered. Dr. Sedlacek referenced Sternberg’s work on Intelligence types. Sternberg suggest there re three types of intelligences: Componential- refers to the interpreting information in a well defined and unchanging context (standardized tests), experiential- interpret information in changing contexts (not able to measure with standardized tests), and contextual (ability to adapt to a changing environment and negotiate the system, again, not measured by standardized tests.

Sedlacek decribed additional noncognitive variables, including Self-Concept, Realistic Self-Appraisal, and Handling System, suggesting that these can be measured and developed. Self-Concept means that you think well about yourself and see yourself in higher education. If you don’t have this self-concept, then you will struggle in the higher education environment. Realistic Self-Appraisal suggests that you understand your strengths and weaknesses. A good program lets a program express both what they can and can’t do, this can be measured, and then you can support the areas that need development. Finally, Sedlacek elaborated on the notion of Handling the System/Racism. This suggests that the person can work the system and take an assertive approach to dealing with existing wrongs. You can see that if you were to try to design a test, it would not be practical to try to assess these elements.

Other dimensions include Leadership (in both traditional and non-traditional groups), Long-Range Goals(the longer the better, but it is contextually based- long-range to one may look different to another, but the point is having a plan), Strong Support Person (someone who pushes the student back toward the system), Community (loners have more difficulty), and Non-Traditional Learning (what have you learned from your experiences outside of the academy?).

The advantages of noncognitive variables include: it is a research based approach, there are multiple ways to assess, it is retention related, considers diversity, tested legally, revised to fit situation, no cost, student development, community building, and supported by high school counselors.

Dr. Sedlacek understands the difficulty of taking a new approach, but is open to the challenge and criticism. He quoted Elbert Hubert, “To avoid criticism, say nothing, do nothing, be nothing” and suggested that the criticism can be countered and the ideas should be discussed. He also described several key legal cases that support the use of noncognitive variables and highlighted the recognition of Oregon State as an exemplary program as evidence of support for the approach. After providing resources and references, Dr. Sedlacek left the participants with a challenge, “What don’t you try this?” and cast an invitation to all to implement these measurements as a step toward societal change.


Session 1: Non-cognitive Variables in Action

Noah Buckley

Noah Buckley

Noah Buckley began the session by describing how Oregon State has implemented noncognitive variables in their process and the related research and results. Their tool, the “Insight Resume” was developed with the support of Dr. Sedlacek. It gives the school a way to consider the noncognitive skills that relate to college success in the admissions process. The university launched the process in 2004. The tool includes 6 questions, that align with Sedlacek’s work and assess a students experience and competency in: leadership/group contributions, knowledge in a field/creativity, dealing with adversity, community service, handling systemic change/discrimination, and goals/task commitment. The questions have been the same since the inception of the program.

Each question is scored from 1(low) to 3 (high) by an Insight Resume reader and is blind of all student information. Scores and responses are used to inform some admissions decisions, general scholarship selection process, and at times, “red flag” responses are addressed.

A 2009 analysis by the university aimed to measure the correlation of the Insight Resume scores to retention and graduation and to predict such outcomes. They examined the 2004-2007 cohorts with gender, ethnicity, and Pell status as variables. The results demonstrated that the 2004 cohort had higher odds of retention and graduation with higher IR scores. The 2005 cohort results demonstrated that African Americans and Native Americans showed increased odds of retention with increased IR scores. The 2006 and 2007 cohorts showed no evidence of a relationship between IR scores and graduation and retention. Thus, the conclusion was “In no case does the inclusion of IR scores offer substantive improvements in predications.” GPA was actually a better predictor.

Buckley then moved the focus to today’s applicants and how they are scored and how the results are used in practice. The admissions excepts are scored twice with the goal of finding “the diamond in the rough”. High achieving students who are eligible for scholarships are also scored twice and that is then fed into an equation whereby students are selected for scholarship. Everyone else is scored once. These “red flag” students have demonstrated that they do not have the views and perspectives that would make them a fit on campus. The university has denied admission based on responses to these questions.

In the future, the university realizes they must ask themselves if they should continue to use the IR as a filter. They see the opportunity to use the identification of non-cognitive deficiencies as an opportunity to provide support to build competency through courses for students who are low in various components of the assessment. Other opportunities to support students include early alert monitoring (providing a support system), advisor flagging and training, intentional linking to existing programs and services and new courses to support the development of deficient areas.

Jon Boeckenstedt

Jon Boeckenstedt

Jon Boeckenstedt presented DePaul University’s use of noncognitive variables. He highlighted the title of the conference and emphasized the notion of “Beyond the Usual”, noting that understanding the context of what we can and can’t predict is important to understand. He asked, “What really is the nature of college admissions?” and described how, over the years, it has been considered a Gate-Keeper, Labor Market Sorter, a “cog in one of the greatest inequality producing machines this country has known” (Brooks). Today, Boeckenstedt suggests, we aim to consider college admissions as a process to identify students who have the potential to be successful on a campus. Currently, however, grade point average and test scores are the measures we use.

Boeckenstedt quoted Einstein (below) and noted that there are just such a wide range of factors that contribute to student success, that it is very difficult to develop a formulas.

“When the number of factors coming into play in a phenomenological complex is too large scientific method in most cases fails. One need only think of the weather, in which case the prediction even for a few days ahead is impossible.”

From there, he moved the focus to the practices in play at DePaul. He described the university as the most mission-driven university he has worked with, highlighting demographics that demonstrate the undergraduate diversity. He notes, however, that their movement toward noncognitive variables was not to develop diversity, but rather a reaction to their previous use of used untested and unproven measures of admissions. They realized that some things meant even more in terms of student success than class rank, GPA, and test scores.

The DIAMOND project at DePaul was developed to explore and understand the importance of non-cognitive variables in student success and retention, and extract the characteristics deemed important. Boeckenstedt noted, however, that even if they had a “perfect predictor” of year 1 GPA, the persistence beyond the second year couldn’t be accurately predicted.

The DIAMOND project included many of the same types of questions as the University of Oregon and was scored in the same way. The results so far demonstrate that the DIAMOND scores are less correlated with income and ethnicity- which is a good thing. They are also not correlated with the ACT. Some students show low ACT scores and high DIAMOND scores- these are the students that will experience the biggest bump in the process. Students with higher than average DIAMOND scores have greater retention whether the ACT is above or below average- so there is some promise in the approach, but they still find that high school GPA is still the most significant predictor.

In conclusion, Boeckenstedt emphasized that there is no silver bullet when explaining and predicting human behavior. It is possible to believe the in the power of the variable while realizing that the instruments to measure it may not be as robust as we like.


Session 2: Attributes of Good Students and Good Professionals

Dr. Patrick Kyllonen

Dr. Patrick Kyllonen

Patrick Kyllonen launched the session by framing the key concepts to consider during the panel discussion. He highlighted the title of the conference, Attributes that Matter, and noted that this title begs the question, “What does matter?” To begin to answer this question, Kyllonen pointed to a recent study, Education for Life and Work: Developing Transferable Knowledge and Skills in the 21st Century. This study pointed to three key areas that are needed to be successful: Cognitive, Interpersonal, and Intrapersonal skills. Kyllonen identified the uses, challenges, and approaches related to assessing non-cognitive variables. More specifically, he outlined:

Uses: college admissions, college success, work success

Challenges; noncognitive assessment, “fakability”/coaching, cross-cultural comparability, test-taking motivation, applicant reaction

Approaches: Self-ratings, others’ ratings, situational judgment tests, forced-choice assessments, behavioral interviews, an actionable feedback.

With these in mind, the first panelist, Steve Kappler shared perspectives related the ACT scores, suggesting that there is much more information that can be gained from the results than a simple number. According to Kappler, the scores can actually tell you what a student can do and they can tell a student what they need to work on. He emphasized three domains of college and career readiness that ACT targets for identification and intervention opportunities: Academic Readiness, Academic Behavioral Readiness, and Career and Educational Planning.

From an academic readiness perspective, ACT surveys universities to determine what students need to know and be able to do to be successful in their institutions and surveys high schools to understand how their preparation or curriculum aligns. Kappler noted that excessive grade inflation is an obstacle, noting that 25% of students self-reported in 2012 that they had a GPA of 3.75 or above. This self-reporting is in conflict with the bell curve that ACT scores indicate. Thus, assessments can be used to provide more accurate data. Using the state of Illinois as an example, Kappler pointed to the relationship between ACT benchmarks and student enrollment, re-enrollment, and non-enrollment in college and the predictive validity of ACT composite scores in terms of degree completion within 6 years. He emphasized that there is quite a bit of information in the ACT results that can be used for predictive purposes.

In terms of career and educational planning, Kappler described ACT measurements of expressed vs. measured interest, and academic ability as compared with interests. These are of particular importance, as they point to intervention opportunities, as students can bolster weak areas if they have an interested in a certain area but do not have the academic background to support that path.

Steve Kappler

Steve Kappler

Steve Kappler described Academic Behavior Domains: Motivation, Social Engagement, and Self-Regulation. An analysis of student data found that the more motivated and engaged students were, the higher the GPA. Lower self-regulation related to lower GPA. And from a social engagement perspective, the fewer extracurricular activities a student participated in during high school, the more likely they would drop out.

Sheldon Zedeck described the research at UC Berkeley on predicting lawyer effectiveness with non-cognitive measures and the use in such measures in law admissions decisions. His talk contrasted the predictive ability of tools that are used for admission to law school as compared to their ability to predict success as a lawyer. Berkeley was very interested in identifying lawyer on-the-job effectiveness factors and developing evaluation scales to predict such effectiveness. To reach these goals, they conducted a “job analysis” of lawyering and effectiveness dimensions. Next, they identified other predictors, non-cognitive variables, to explain lawyering effectiveness. Their process included hypothesizing predictors, developing/selecting tests, administering the tests, collecting performance evaluations, and then establishing test-performance statistical relationships. The predictors included UGPA, LSAT, and INDEX.

Sheldon Zedeck

Sheldon Zedeck

Zedeck next described the results of their examination. 26 effectiveness factors were identified across the practice of law. They were curious to know if the LSAT could predict effectiveness as a lawyer and thus developed performance evaluation scales for each of the factors. The 26 factors were organized into 8 categories including: intellectual and cognitive, research and information gathering, communications, planning and organizing, conflict resolution, client and business relationships- entrepreneurship, working with others (other attorneys), and character (passion and engagement, diligence, etc.). The scales became the criterion by which they analyzed the predictive capabilities of the LSAT.

As they developed new predictors, they looked at Snyder’s self-monitoring assessment, carver’s optimism scales, emotional intelligence and a biographical inventory (past experiences) approach, accomplishment and experience records, situational judgment, and a moral responsibility scale. Zedeck continued to describe the many evaluation tools they used to examine the 26 factors, including those already developed and some that were developed in-house.

After identifying the many tools, they began their validity study with 1100 respondents. They found that their new suite of tests did not do better than LSAT in predicting law school performance and there were interesting findings in terms of their ability to predict effectiveness on the job. The LSAT predicted 12 of the 26 dimensions, but 8 were negative. The INDEX predicted 9 of the 26 performance dimensions and 4 were negative. Thus, for some domains, the better you did on the LSAT, the less effective you would be in the workforce in that specific domain. Zedeck found that their personality assessments showed less differences between minorities and Caucasians- and thus the use of these tools supports the identification of a more diverse student body.

Zedeck suggests that this type of process can also be done with undergrads. He did not promote abandoning the cognitive assessments, but to use them within the battery of tools used to identify students with potential for success.


Master Class – Beyond Grutter, Gratz, and Fisher: Legal and Educational Implications of Considerations of Race and Other Options in College Admission

In his presentation, Coleman returned to a central issue associated with admissions, merit, and the policy and legal implications when race or ethnicity enter the picture.

Effective Policy Development

Art Coleman

Art Coleman

Art Coleman began with an overview of effective policy development, noting key spheres of influence on policy development and implementation. These spheres include stakeholder support, research and experience, and law. According to Coleman, one of the myths that is frequently out there is that we have a great disconnect between what law demands and good education requires. He suggests that we should be looking at the question of one fundamental alignment, noting that a university can have the best, legally-based, articulate, policy ever, but without stakeholder support, it will not work. This is not about taking care of the legal questions or figuring out the research foundations- but rather we must be transparent about what is going on behind closed doors in the admissions office. Additionally, he warned that when there are volatile issues, such as race and ethnicity, you must have a compelling story to tell. It is important to pay attention to the laws, but you must also pay attention to the court of public opinion, or the “stakeholder” sphere of influence, and be able to communicate your story in a way that can be understood.

Educational Benefits of Diversity

The “benefits of educational diversity” perspective is primarily the frame by which universities ground their discussions. From the College Board to Medical colleges, etc., it has been suggested that racial diversity is essential for us to succeed. The theory of action recognizes that a diverse learning environment, with an appropriate pedagogical focus, leads to enhanced teaching and learning, and improved outcomes. Students in such environments are better prepared for productive lives in the workplace and society. Coleman suggested that “educationally sound and legally defensible race/ethnicity-conscious practices are the product of a well-designed, institutionally-aligned and integrated process that connects means to ends.”

This is the theory that the University of Michigan, for example, was framed on. This is the framework on which one would identify goals and objectives, and strategies to meet each of those. If you cannot gage the success toward you objectives and goals, you are not in a legally defensible position. This theory is the way we tend to talk about admissions among legal, admissions, and institutional leaders. Coleman suggests, however, that we are missing the central issue, “What is merit?”

Missing in Action: Merit

According to Coleman, we tend to stay within our comfort zone and talk about process. Although there is nothing wrong with that, those who challenge the lawfulness of a university’s approaches tend to focus not on the process but the substance- what are you considering? A simple story: test scores, plus grades, equals merit, can be easily tackled. If it can be proven that someone with lower scores and grades was admitted above someone with higher grades and scores, the university is in trouble. Coleman noted that he finds it remarkable how we are not capturing the intelligence and power behind telling the story. The question of merit, who is admitted, and the composition of the class, is not that simple. The question of merit must be tied to what the university needs, the mission, and what will contribute to the overall needs and experience of the institution.

Coleman then reviewed cases from the perspective of admissions polices and legal outcomes:

1978 Regents of UC vs. Bakke. UC defined merit as anyone over a 2.5, stating clearly that “we will interview no one with a GPA below 2.5”. UC admitted a set-aside group of 16 out of 100 minority students who had GPAs that went as low as 2.11. The university in this case did not stand by the very theory of what merit was for them in practice.

Grutter v. Bollinger (2003). The admissions process at the University of Michigan includes: 1) individual review of all applications, with a focus on academic criteria, 2) likely contribution to intellectual/social life of the institution, and 3) contribution to diversity which can enrich the education of all. The court upheld the admissions policy.

Fisher v. University of Texas (2013). The UT-Austin admissions policy includes a 90% allotment of seats to Texas residents. 81% of entering class was admitted under the top 10% law. The remaining Texas residents compete for admission based on academic and person achievement indices (PAI). Under PAI, SES, Family status, standardized test score as compared to the high school average, and race can be considered.

The implications for action, according to the AAMC, suggest that authentic holistic review should be multi-faceted, involving many student qualities and characteristics (that may include race and ethnicity). So how do we communicate the multiple strands that are part of the admissions process? Coleman suggests that the AAMC model is an effective start.

Coleman stated if there is one key take-away, the one that is as profound as any to think about in conversations about race and ethnicity, is that race is more than just the “check the box”. Coleman suggested that we reflect on the consequences of a court stating that race and ethnicity can never be considered- what does that mean? How is it possible to be blind to race? Coleman concluded that we must be more sophisticated about how we think and talk about race and ethnicity.

Key Questions (summarized):

  • What is Merit?
  • How does race really enter into the equation?
  • Over the course of time, can I document educational success associated with my admissions decision of merit and the ways in which I consider race?

Fisher Overview

Coleman transitioned into an overview of the fisher case with two key questions of relevance: Necessity? And What’s the Impact? The court basically said that if you are using race as a factor of admissions to try to get to some goal, you must show demonstrative evidence that the consideration of race helps you achieve your goal. In the case of Michigan, they succeeded in demonstrating the need for consideration of race. In the Fisher case, Texas suggests that the modest consideration of race does have an impact on the university that is supported by evidence. The plaintiff’s position was that the policies did not have enough of a substantial change in terms of the campus to justify that consideration of race- no matter how limited it is in the equation. The question, then, is what is critical mass? How does that vary based on location? Texas had 20% of minority students, suggesting that race is not necessary. Texas, however, suggests that because they have not achieved critical mass at the classroom level, they have not met their goal.

With this in mind, we wonder what will the future hold? And what it should hold?


Keynote Dinner Address: The College Board: Moving from Here

David Coleman

David Coleman

David Coleman launched his discussion with an explanation of his decision to participate in the CERPP annual conference, noting that it is the last long distance trip that he will take in the next 6 months. He selected this conference to participate because of Dr. Jerry Lucido’s work in helping traditionally underserved students in accessing college. Coleman acknowledged Dr. Lucido’s passion for the notion that dreams must be preserved and described Lucido as someone who continues to find opportunities to preserve the American dream. Thus, he was motivated to join Lucido’s efforts and participate by delivering the dinner address at the conference.

After a standing ovation for Jerry’s work, Coleman shifted his presentation toward his interest with admissions officers. He highlighted their complex work that involves real decisions and noted that they are truly educators. The moment a student engages in college is life-hanging, and thus, the consequences of the work of admissions officers is monumental.

In regards to his role as president of the College Board, Coleman defined recommitting the organization to the ideals of equity and access as one of his primary goals. Additionally, he stated that the mission of the College Board can and must change- it is not enough to connect kids to college, they must complete.

Noting Duckworth’s work on the idea of grit and self-control and Ericsson’s work on deliberate practice, Coleman described himself as interested in the practices that make us excellent. He suggested that people tend to hide their practice, so it looks easy, but most things are by design. He also noted that the ability to work alone is also an important factor- you must be able to work alone to have deliberate practice and you must be able to revise your work.

Coleman moved his focus to testing instruments, including the SAT and ACT, remarking that they do not necessarily add predictive value. There are several hypotheses to be considered:

  • Non-cognitive approaches should help us understand students with high grades and low scores. Such students may be just what we hoped for, or it could also mean they just took easy classes.
  • Students with high SAT scores and low grades are equally interesting. These results could indicate that the student was bored and they should be considered for admission. Or their success on the SAT could just be a one shot. Non-cognitive perspectives would help shed light.
  • There is also the case of the student with both low grades and low SAT scores. Such a student is not a very promising prospect. Grades represent daily stamina, doing work, etc., so the suggestion that non-cognitive only will bubble up promise is unlikely.

Our shared challenge is to get more verifiable and consistent data. We must ask:

What are the kinds of firmer non-cognitive data or patterns that we can collect?

Are they resilient? Did a student receive a D and then rally and ultimately do well. Why?

The College Board will invest resources in the next few years in a search for kids who can go to college, but don’t. Each year 40,000 students do well (top quartile on the SATs), but do not go to college- a notion that Coleman finds unconscionable. Equally upsetting are the lowest income kids who, at the moment of college application, do not apply. Coleman holds the College Board accountable for shoring this up. He said it is our shared work to find the kids who are forgotten.

Coleman described the College Board as having a deep desire to talk to institutions and educators to learn how we can work together. He said he wants to be in a conversation with institutions to create an exam that is what institutions need and asks for. CB will improve, not hurt, differentiation at the high ends. They will get data faster than they did before. In partnership, the College Board and educational institutions can work toward setting the conditions for college opportunity and success for all students.


Day 3


Session 1- Putting it together independently: Rewarding Character and Achievement with Scholarships

Carrie Besnette Hauser

Carrie Besnette Hauser

Carrie Besnette Hauser began the panel with a review of the literature. She notes that the data bout scholarship providers is slim. According to the 2005 IHEP study, there are 5000 private scholarship providers with between 3.1 and 3.3 billion dollars in private scholarships. Roughly 7% of undergraduates receive private scholarships (an average amount of $1982) and 5% of graduate students. 30-50% of recipients are traditionally underrepresented students.

Private scholarships tend to target those who may lip through the cracks and facilitate choice and affordability. Many private scholarships go beyond – it is not just a check, there are other services and supports that go with it. Additionally, many scholarship programs have incorporated interview and essay questions to capture non-cognitive abilities.

Besntte Hauser highlighted several programs, both need and merit-based, that meet the “gold standard” in terms of their design and support. (Please see the presentation for information on these programs.) The common factors in these programs include their interest in supporting students based on their character, resiliency, emotional intelligence and other non-cognitive factors. Additionally, these programs also provide “wrap-around” support- not just funding- including support while in college and connecting students to career opportunities after college. (Please see Dell in the presentation to gain an understanding of a particularly strong support program.)

Charles Lovelace

Charles Lovelace

Charles Lovelace discussed the Morehead-Cain foundation, an organization with 2900 alumni. They offer 50 scholarships a year to the University of North Carolina at Chapel Hill. The program was one of the first full, non-athletic, private scholarship programs in the country. The program is specifically interested in finding student leaders and the question then, is how do you define and find leadership? After a research study 10 years ago, they re-vamped their program with an emphasis on non-cognitive variables.

Lovelace described the “soggy bottom of the barrel”- or students who were underperforming, not necessarily academically, but they were not fulfilling the mission of the organization, which was to maintain the excellence of the university. Students are expected to engage in the university community and make a difference there. The criteria for selection included leadership, scholarship, physical vigor, and moral force of character.

The difficult and challenging task is how to select a leader. Early on the program included participation in clubs and activities, and holding positions of leadership within those clubs, as a major focus of the selection process. He notes, however, this traditional approach to identifying students limits your pool. They decided that they needed to take a new approach. They began to ”grade” their students at graduation in terms of whether they met the program expectations based on their engagement in the university.

Morehead-Cain engaged in a study of 7 classes (352 students) and found that their students, on average, were below the expectations. They then went back and correlated high school activities to outcomes. They found no correlation between performance in the program and SAT scores, nor any relationship between high school activities or positions and program outcomes.

They decided that their 4 criteria were too broad and they needed to drill down to determine what makes a successful scholar. They tried to identify the qualities that they noted in students who did meet or exceed program expectations. They identified achievement drive, independence, commitment, self-knowledge, empathy, and spark as the key characteristics. These became criteria in their selection process.

Other changes included the use of professional readers in the selection process, thresholds for “scholarship” and “physical vigor”, dropped the SATs and transcripts after initial screening. They also completely modified the application. They limited the activities that students could report to 5- honoring the applicant in the sense that they are asking the applicant right upfront what is most important to them. They asked for more stories and examples during the interview process.

The result of these changes is that they have been able to get students that are a better match to their goals. They continue to refine their process, looking at research and opportunities to select the students that match their goals. Lovelace describes his greatest interest now is how to separate merit and privilege.

Larry Griffith

Larry Griffith

Larry Griffith spoke about the GMS program and the application of noncognitive variables. He described watching applications come in earlier this week and that his role is similar to that as an admissions director in terms of the decisions and opportunities that those decisions shape.

The purpose of the Gates Millennium Scholars program is a focus on leaders. They select 1000 candidates a year and provide funding through the doctoral degree level. They consider themselves not an access, but a success program, in that they support students all the way through the college process and experience.

The non-cognitive variables in the selection process aim to identify potentialin the context of the program. These variables include self-concept, realistic self-appraisal, and handling system/racism, leadership (in a wide variety of areas), long-range goals, strong support person, community, and nontraditional learning.

The concept of resiliency was emphasized- how does a student respond when they encounter someone who does not support or believe in them. Are they able to, through their determination and strength of character, push beyond such obstacles? A key question that provides insight is how did you handle a situation that was unfair?

The application includes short-answer questions based on noncognitive variables, a personal statement, letters of recommendation, and demographic, background, and activity questions. Approximately 80% of the weight is placed on the non-cognitive variables.

The results of the program are promising. They have found that persistence was very high with GMS scholars and that they are more likely to go to graduate school. The scholar’s leadership score has a significant relationship with engaging in academic activities in college. The average GPS is 3.4 for undergraduates and the retention rates and 6-year graduation rates are over 90%.


Session 2 – Preparing Community College Students for Transfer: Highlighting the Individual, Institutional, and Policy Attributes that Matter

Stephen J. Handel

Stephen J. Handel

Stephen J. Handel introduced his colleagues and provided an overview of the basics, the need, the desire, and the criticisms related to community colleges. Community college growth has been extraordinary in the last decade. It is the largest system of college education on the planet. The need is clear- we must seek diversity. The desire for incoming students is to transfer to get a baccalaureate degree. We easily dismiss the intentionality of community college students- they want to transfer, but they are not- and this is the big criticism.

Nancy Shulock

Nancy Shulock

Nancy Shulock emphasized the point that too few students reach milestones on the road to degree completion. Latino and black students are less likely to reach those milestones than their Asian and White counterparts. Transfer rates are low overall, and Latino students transfer at half the average rate. Shulock examined the literature to verify success indicators or enrollment patterns that correlate with success. Passing college-level English and Math within the first two years was positively correlated, as was the accumulation of 20 or more credits in the first year. She notes that we know what works, but hardly anyone shows those patterns. The concept of a program of study, then, is a critical success pattern. Those who follow a program of study in the first year are more than twice as likely to complete such a program than those who do not do so.

Among the successful transfers, only 43% ever complete transfer work and only 7% complete a associate degree. The challenge of students transferring without an associate degree is that if they do not complete their 4-year degree they have spent so much time in school with nothing to show for it in terms of degrees.

Lawrence A. Nespoli

Lawrence A. Nespoli

Lawrence A. Nespoli  noted that community colleges simply have a lot of work to do, but that there really is a stronger commitment now to success. Students who make it to and through the point of transfer do very well. Nespoli emphasized the importance of completion. He noted that over ¼ of those who earn bachelor’s degrees started their path at a community college. Thus, there is no way to aim for baccalaureate completion without taking the community colleges into consideration. He notes that many community college students are the first in their families to go to college and most need remediation- they are not college ready when they get to college. Those who do move through and transfer are at-risk for “shock”. When 2-year students land at a 4-year institution, as capable as they are academically, the transfer shock can disrupt their path. He also noted that when all of their credits (as opposed to “some”), of a transfer student are accepted, students are much more likely to persist and complete their degree.

Frank B. Ashley

Frank B. Ashley

Frank B. Ashley spoke to the challenges of faculty who complain that the community college students are not prepared. Thus, they are a “tough sell”. He suggested that partnerships and conversations amongst stakeholders are critical in supporting students and supporting the transfer and success of community college students.

The panel continued their discussion, responding to questions from Handel and the conference participants. Please see the presentations and handouts for additional information and details related to their positions and perspectives.


Closing Session – Leaders Respond

Eric Hoover

Eric Hoover

Eric Hoover opened up the session by praising the conference content but pointing out the lingering “frustrating sense of unease” regarding noncognitive variables (NCV) among conference participants. He quoted some conference participants in admissions as being excited about the research, but unsure of how to implement it, as well as the difficulties of approaching their board to convince them of the value of using noncognitive variables.

Scott Jaschik followed Eric, and reminded us that as a journalist, he always looks for contradictions, and proceeded to enumerate a few: 1) even

Mr. Scott Jaschik

Scott Jaschik

as the conference highlights the importance of transfer students from community colleges to four-year institutions, there is for example, a “civil war” waging in the CUNY system with disagreement on the articulation of policies regarding community college transfer; 2) while the research on NCV has been fascinating to see as it takes off, is the commitment to use NCV by postsecondary institutions real? Scott added, why is it option if it’s meaningful, what are the cognitive assessments people value (like the SAT II), and hearkening back to Art Coleman, why is there not more openness among admission officers?

Both Eric and Scott praised the conference for the high quality and “refreshingly frank” level of discussion, as well as for daring to broach a discussion of the possible ramifications in admissions and higher education in the wake of the Fisher vs. University of Texas Supreme Court case.

Andrea Brownstein

Andrea Brownstein

Andrea Brownstein as a college counselor and as a K-12 representative was asked the first question: Do students need NCV and do they want it?

Andrea answered that what matters in admission should matter in institutional culture. “Especially as college counselors, we spend all of the time that we have to help kids become productive citizens, which involves NCV. NCV will help more students to be more successful.” She also recognized that it takes a long time to change culture, and mentioned that her own organization, MICDS took 25 years of mission assessment to get where it belongs today, with 1/3 of their students representing minority groups.

Gil Villanueva

Gil Villanueva

Gil Villanueva, and Pam Horne were asked the same question. Gil responded that in the spirit of openness, University of Richmond was 13% students of color and is three years later, 20% students of color, with NCV important to the admissions process, particularly for admitting students who were “at the margins.” However, academic quality was not something the university wanted to give up. But what he found was that “top students were attracted to diversity.”

Pam answered that she did not know when we would be able to use NCV nation-wide. Her concern was that the use of resources to use NCV would have to be carefully evaluated. On the other hand, she certainly values the ways in which NCV are already currently being used—in essays, recommendation letters, asking questions about leadership, and training interviewers and readers.

The follow-up question by Scott posed to Gil and Pam was that if you guaranteed a more diverse class by using the Insight Resume or other tool assessing noncognitives, but your institution dropped in ranking, would you still use it?

Pam answered if the return on investment was high and graduation rates were high, yes! Gil responded that his institution evaluated itself on diversity and the academic quality of the incoming pool. Since they’re using NCV already, it’s not as if an assessment of NCV would be the sole tool.

Eric: how could NCV determine the merit of admitting or not admitting a student?

Pam Horne

Pam Horne

Pam answered that they could identify skills and behaviors that can be changed, and thus interventions that could help improve these NCV. Gil answered that as an immigrant himself and in his immigrant students, he saw that certain NCV, including respect for authority, cultural heritage, self-love, etc. were crucial to student success. He recounted the story of a Vietnamese student who lost his mother, but was able to tap into his social connections to get into and stay in college. This student also became a Gates Millennium Scholar.

Eric: What is the next step and what will you carry back to the campus?

Gil: We use NCV on a limited basis. We need a system for socring.

Pam: We need more research and discussion on implementing NCV? How do we definite merit? The press and public have defined it for us. For me, the purpose of merit aid is not to reward high school behavior or achievement, but to help shape our class; thus the student’s promise or potential on campus and society.

Scott: What are the attributes that have mattered more? Wealth! Public schools have been recruiting more out of state and international students, and giving out more merit aid to students who don’t need it but who they want.

Andrea: We want our students to consider places beyond the usual—we don’t want them to be courted by institutions for their relative wealth.

Gil: Our institution meets 100% need and is need-blind. Only 1% of schools can do it, so it’s very expensive. There is a lot of talk in Washington, and hopefully the shift will be from merit to need-based aid.

Pam: This is a matter of balance—70% of high school students and parents expect merit aid, even if the students are not high-achieving. At Purdue, 15% of financial aid is in merit aid to the incoming class, with target populations like small-town pharmacists, for example. We are need-blind in admissions, expect for out of state students.

Eric: How often do admission officers talk about students and their attributes? What do you lack and what do you want more of?

Gil: At mission-centric Brandeis, they were looking for a sense of social justice. At Harvey Mudd, they were looking for female engineers, physicists, and other scientists. They were looking for the kind of student who would look after his/her roommate and was mature.

Pam: Entrepreneurial students. Students good at teamwork.

Andrea: I supposed the question for me is what kind of students do we give you? Money works in terms of resources. So schools with resources can provide students who have NCV we desire. Also, my colleagues spend the fall writing recommendation letters that make the student stand out. Every letter is individualized. NCV are important to help distinguish students from the pack.

Scott: Are NCV vulnerable to coachability and faking it?

Gil: Thank goodness for recommendations. We’re not afraid to call/email colleagues.

Pam: The proportion of students to got this extreme are pretty small. The majority are not “faking it.” However, when assessing NCV, you do have to be careful how you word essay prompts. For example we took the question, “What has been a life-changing event?” off of our application after receiving innumerable essays about life-changing sports injuries. Many students have not experienced something truly life-altering by the time they’re applying for college.

Andrea: Most kids don’t fake it—they don’t have the nerve. In terms of coachability, this is part of the work that we do. Coaching is something needed to bring out the best in a student.

The rest of the panel was devoted to a lively Q&A session with the audience.


Conference Closing

Jerry Lucido

Jerry Lucido

Jerry Lucido, director of the USC Center for Enrollment Research, Policy, and Practice brought the conference to a close, exhorting us to think about the qualities that we care about and how do we bring these objectives to align? Also, how can we use examples to guide us in our work?

Finally, he asked the audience for feedback on the conference and possible topics for next year’s event.