The Winona State IPAR Blog
Assessment/Accreditation
Qualtrics Tip: Rich Reporting by Importing Responses
- Qualtrics is a website allowing users to create online surveys, distribute them, and report on the results.
Here at Winona State we have invested significant time and energy in campus-wide assessment efforts for many years. Online surveys are the backbone of those efforts, and Qualtrics (www.qualtrics.com) is our tool of choice for conducting surveys. One strength of the system is the reporting tool set available, and one unobtrusive Qualtrics feature that we have found very useful in combination with that tool set is the ability to import survey responses from a text file straight into a Qualtrics survey. If that doesn’t sound tremendously helpful at first blush, just bear with me.
Imagine being the administrator of two different surveys, both delivered to university sophomores, one in fall, and one in spring. Responses to both surveys have been collected, and the surveys have closed. At that late stage, you realize that it would be very helpful to explore the correlations between a question on Survey A and several questions on Survey B. It would also be helpful to include the college affiliation (e.g., College of Business versus College of Liberal Arts) of the respondents in your reporting. In our office, we frequently find ourselves in similar situations. Obviously, there are many solutions to the problem, but Qualtrics provides a good one.
It is relatively straightforward in Qualtrics to approach the problem this way:
- create a survey, Survey C, that is a hybrid of Survey A and Survey B
- add an Embedded Data element (a Qualtrics term for extra information attached to a survey response) to hold the college affiliation of each respondent
- fill the new survey with selected answers from Survey A and Survey B
- include data from your student information system (SIS), such as the college of each respondent
- quickly put together charts, response statistics, and crosstab reports to analyze the hybrid data within Qualtrics.
I will follow this post with an example of how we’ve used this technique for quick, ad hoc reporting that brings together data from multiple surveys with SIS information using only duct tape and common household items. Stay tuned.
Do students know how much money their parents make?
In a New Entering Freshmen (NEF) survey in Fall 2010, WSU asked its incoming freshmen students what their “Best estimate of parents’ total income” was. They were able to select one of seven options:
- Less than $10,000
- $10,001 to 25,000
- $25,001 to 50,000
- $50,001 to 75,000
- $75,001 to 100,000
- $100,001 to 150,000
- over $150,000
However, WSU also has this data from the FASFA financial aid application form the student/family submitted. Of the 1,633 NEF WSU took in last year, we had FASFA forms on file for 1,393 of the students and we had survey responses from 1,389 students. We had parental income data from both sources for 1,203 of these students.
For these 1,203 students for whom we had data to compare, we contrasted the parental income as reported on the NEF Survey with the Adjusted Gross Parental Income as reported on the FASFA.
Students did not know how much money their parents make.
Overall, 21% of students underestimated their parent’s income and 44% overestimated parental income, leaving only 35% of students that were correct.
Students tended to overestimate parental income, but got (slightly) better the higher their parent’s income was…
It could be that students are estimating their parent’s net income, while the FASFA reflected the parent’s adjusted gross income. However, the percent of under-estimates along with how wildly off some students are in their guess still makes the survey question data, well, questionable.
We would have found a very different frequency distribution of parental income categories had we relied on FASFA data instead of the survey:
So, why do we ask this question on the New Entering Freshmen Survey? That’s a good question, perhaps we won’t anymore.
