5. Assessment and Research
Assessment and Research is an area of student affairs that is crucial, yet seemingly under-appreciated by graduate students that are on portfolio tracks, such as myself. My instructor for AHE 513, Assessment and Research Methods in Higher Education, noted this dissonance. Thesis-track students, as expected, have a vested interest in learning how to gather information for their original research. Portfolio-track students can have a more difficult time seeing the value in understanding quantitative and qualitative methods. However, I have not necessarily found assessment and research to be troubling, as I come from a scholarly background and may still eventually pursue a Ph.D. in Communication. Luckily, I have found several ways to meet this competency through hands-on projects.
Change in higher education cannot simply derive from intuitive feelings. While perceiving conflicts and emotional impact are important skills in student affairs, it is still necessary to gather and analyze data. Data collection can come from something as simple as a questionnaire distributed after an event. A researcher understands that questionnaires will not yield a 100% rate of return, but that the answers provided are still useful.
For example, my internship at Wenatchee Valley College was primarily designed to cover the community college experience, as well as my interest in new student orientation. While I had originally contacted the school to see if I could complete assessment work for Multicultural Affairs, the experience significantly changed looks when it was decided that an intern in Student Programs was of greater need. The internship still necessitated some assessment work.
My first task was to study the functional area in which I was working. I obtained a copy of the National Orientation Directors Association handbook, which contained many articles and planning resources specifically for orientation. I needed to identify the most relevant resources for my particular needs. I knew that I could not apply an orientation model that worked well at a private liberal arts university on the East Coast when I was working within a two-year community college in rural eastern Washington. “Orientation Programs on the Two-Year Campus” (Cuevas & Timmerman, 2010) was my first source of information. It confirmed my intuition that designing a program to serve the student population would need me to further research which demographics WVC served.
After examining the NODA handbook for other recommendations and guidelines, I next set about orienting myself to the WVC populations. I studied the institution’s Mission, Vision, and Values and found statistics on demographics, as well as the types of degrees and programs offered. What I found was an institution that was in-tune with its community needs. The average age of the WVC student was 29, which was particularly striking because I know from first-hand experience that many traditional-aged college students begin their careers at WVC. This meant I was definitely working with a population that had very diverse needs. My working research question, for the purpose of designing a new program, was along the lines of, “Which aspects are essential to a successful orientation program at WVC?”
After synthesizing the information I found from NODA and WVC statistics, I applied another prong of information to my foundation: successful models from peer institutions. This was a demonstration, of sorts, of theory into practice. While it was not yet my own practice, I was able to see what other institutions (community colleges) were able to take into consideration when designing their orientation programs. Some schools implemented two-day programs, as they had capacity for commuting students and served a more traditional base. Other schools used an intensive one-day model, combining registration, class selection, and general orientation into a day-long event. I knew that WVC’s population was highly comprised of working individuals, individuals that may not be able to take a full day off due to timing, or work and family obligations, and furthermore, a good portion of the student base commuted at least an hour to campus. This would make an overnight stay difficult, as the one residence hall on-campus is small and hotels can make travel undesirable and not affordable.
The solution I developed was to keep certain aspects of orientation intact. Registration and class selection would continue to be handled before orientation by academic advisors. New student orientation would occur on one day, prior to the start of school, and it would occur on a weekday. One change would incorporate an online component which could serve as a pre-orientation “crash course” and a stand-in for students who absolutely could not attend on-campus orientation. Additionally, on-campus orientation would be split into two-half day sessions. A day session would grant students following a traditional and/or flexible schedule the ability to attend the workshops without the need to stay in town overnight. After several hours, staff would reset and welcome the evening session, designed for working students. Again, workshops would be concise, allowing students to obtain useful information while not overloading them with the need to stay incredibly late.
While the internship comprised of much more than research, the development of the new orientation model would not have happened as thoroughly without the use of literature reviews and interviews. As another testament to the power of assessment, I developed a questionnaire for students to share feedback with Student Programs. WVC had not, prior to then, used a questionnaire to find out what was lacking or being done well at orientation. Giving students the ability to share their thoughts after orientation will allow WVC to continue to build a strong orientation program, providing guidance for any changes or shifts that may need to occur in the future. Being a better researcher and using assessment will help institutions provide better services for its students. By listening to students, we listen to the voice of stakeholders. When decisions are made that affect students, I hope that it is because the students gave their input.
There is also valuable cross-over in research. Another internship in which I was engaged, my internship at the University of the West Indies at St. Augustine, comprised mainly of informational interviews. This allowed my colleague and I to make observations about institutional culture, and one of the things I noticed was how they approach orientation. Their program was redesigned circa 2006, and as part of that, a new branch of orientation, “UWI Life Support” (UWI Life is their name for orientation), was developed. UWI Life Support is designed to orient support persons, not necessarily students. Workshops are geared towards family members and other mentors and guardians. The concept seems a bit strange at first; why would an institution spend so much time and effort on a population that is not actually attending school there?
In a nutshell, it is because the culture of Trinidad is more collective than the United States’ own culture. Family members stay very involved with students’ lives, and many students reside with their families as they continue to pursue their degrees. However, as a student affairs professional in the U.S., I do know that there is a current trend of families staying involved with their college students’ lives, too. There is quite a bit of talk surrounding “helicopter parents,” and there is also literature demonstrating that a lack of involvement from families can be harmful, especially in families where the child is the first to attend college. Additionally, as a multicultural country, there are many students whose parents are from collective cultures. I incorporated this knowledge into the development of a WVC orientation workshop, designed for support persons, such as parents, guardians, and other mentors. Considering the population of Wenatchee also has a large Latino presence, I recommended that this support workshop eventually be offered both in English and in Spanish. The idea to develop a support component may not have came to me without doing research in another country. Knowing this, it is critical to the furthering of student affairs that professionals continue to challenge themselves in researching and learning more about other programs and systems.
Without utilizing results from assessment, my orientation course that I taught would not have been more effective. The format changed dramatically in its second year, moving from just guest speakers and presentations from myself to a combination of guest speakers, hands-on activities, and small- and large-group discussion. The feedback received from the second inception of the class–which was delivered one year later from the original course–was much more positive. The final project assigned to the class even served as a vehicle for feedback; students were asked to create a presentation designed to inform administration about what could be done to enhance the transfer process. Significant ideas included bolstering support for commuting students, as well as improving advising delivered at both the community college and university levels.
Assessment can come in many forms, too. If I need to evaluate the effectiveness of a program, I do not need to set up a control group and variable groups. A good student affairs professional assesses constantly. We do it during face-to-face conversations with our students. We do it in engaging in literature reviews, which comprise of a wide range of media: journals, newspapers, posts on Twitter, and so forth.
Additionally, the usage of learning outcomes as a guide helps me to understand what it is my students should get out of a particular experience. In teaching HC 299, the STEM transfer student orientation course, I developed a section based on other academic success courses. The outcomes were as follows, as taken from the HC 299 syllabus:
At the end of the course, the intention is for you to be able to:
Understand the reason behind a structured, ongoing transfer student orientation
Articulate concrete academic study skills that will lead to success at the university
Formulate time management skills
Form academic support cohorts
Identify campus resources designed to help you succeed and recognize their impact on success
Develop a plan for college success that includes effective goal setting, study skills, and personal health and wellness
Using these outcomes was a way to make myself accountable, as it required me to intentionally design a course that addressed specific points. It also gave my students a “road map” of sorts so that they knew what to expect from the course. Furthermore, if students felt that any one of the outcomes was not being properly addressed, or if they felt that an additional outcome was needed, they were able to express that to the teaching assistants and myself. Learning outcomes were also utilized in the redesign of University Honors College overall goals, which in a similar fashion, gave guidance to the type of activities and events our energy focused on, as well as what we wanted to contribute to our students’ experiences. Students were given access to this information, and my student organization advisees edited the outcomes before they were finalized; in this way, staff, faculty, and students were on the same page.
The ability to assess “on the fly” lets me make slight adjustments, whether it is how I communicate with my students to deliver information or how to improve and build upon a program. Flexibility in approaching assessment and research as positives in the workplace will be key to my success in delivering outstanding service to my future students. What they will find is my care and compassion can translate into data-supported changes, and those changes will be for the betterment of the overall college experience.
Over the course of time, I expect to work with many data-driven students. As I expect to work in technical programs within the two-year, community college environment initially, I anticipate being asked why college matters by my students. Pascarella and Terenzini (2005) provide an excellent resource for grounding practice in scientific and formal results. It is also my hope that after working in the field for awhile, analyzing my interests and the current gaps in research literature, that I will contribute to the field in important ways. I am already formulating topics of-interest, including topics regarding mixed-race Asian-American and Pacific Islander (AAPI) students, Filipino-American topics in higher education, spiritual development (especially for students of color), and transfer student services. My foundation is solid; now it is time to focus my assessments and research interests into visible and meaningful contributions.
Cuevas, C. J., & Timmerman, C. (2010). Orientation programs on the two-year campus. Orientation Planning Manual: 2010 Edition. Sedotti, M. A., & Payne, M. J. (Eds.). Minneapolis, MN: National Orientation Directors Association.
Pascarella, E. T., & Terenzini, P. T. (2005). How college affects students: Volume 2; A third decade of research. San Francisco, CA: Jossey-Bass.