ServiceNow Service Management

Overview

Qualtrics is an online survey tool that is available to all UC Davis affiliates.

One possible use for Qualtrics is voting on academic personnel merits and promotions actions.

This guide and the associated videos demonstrate how to use Qualtrics for merits and promotions voting.

Videos

Log In

To begin, visit https://ucdavis.qualtrics.com/, and log in with your UC Davis credentials.

If you have never used Qualtrics before, you may want to go to Help & Feedback, then Visit Qualtrics University. They have articles and videos for every feature of the platform, and it's the first place to go if you what to know how to do something.

When you first log into Qualtrics, you'll be at the My Projects screen, where you will see all of the surveys you've created listed.

Create survey; Set anonymization options

1. To create our first voting survey, begin by clicking "Create Project."
2. Give the survey a name (qq1).
3. Click "Create Project." This brings us to the survey editor.

Now let's create our ballot survey.

Qualtrics has started us off with a blank multiple-choice question. But most department will have some introductory text and instructions to add to the top of the ballot, so let's change this to a Descriptive Text question type.

1. Click under "Change Question Type."
2. Select "Descriptive Text."
3. Click inside the question, and type in your instructions (qq2).

Now that we've got our introductory text, let's move on to our ballot question:

1. Click on the drop-down next to "Create a New Question."
2. Select "Multiple Choice."
3. Click on the question text, and type in the ballot question text (qq3).
4. Click on the first response, then click "Edit Multiple" to edit all of the response options at one time.
5. Select all of the default options, then press Delete to clear them out.
6. Type in your options, one per line. (qq4, qq5, qq6, qq7, qq8)
7. After you have entered all of the options, click "Done."

We want to make this question mandatory, so:

1. Under "Validation Options," select "Force Response."

We now need to add another question to capture comments, so:

1. Click on the drop-down next to "Create a New Question."
2. Hover over the drop-down next to "Text Entry."
3. When the drop-down appears, select "Essay."
4. Click on the question text, and type in the text that you want to use. (qq9)
5. To give lots of room for comments, click on the text box to select it, then click and drag the handle in the lower right corner to make the box bigger.

That's really all we need for our first ballot! Let's scroll back to the top, and click "Preview Survey" to see what it will look like to our voters. This will open a new tab that will show us what the ballot will look like on both a computer and on a mobile device. You can try entering some sample responses to see what will happen.

Note that if you try to submit a response without actually casting a vote, you'll get an error message and be forced to select one of the voting options.

Let's click "Close Preview" in the upper left corner to go back to the survey editor.

Anonymize Votes and Prevent Ballot Box Stuffing

Before we go any farther, let's set some options that are specific to faculty voting.

1. Click on "Survey Options."
2. Scroll to the "Survey Protection" section.
3. Select "By Invitation Only" to ensure that only people who are explicitly invited to vote can do so.
4. Select "Prevent Ballot Box Stuffing" to allow each voter to vote only once.

Most departments have strict rules about confidentiality where you need to be able to track whether or not each faculty member voted, but not how they voted. Qualtrics has an "Anonymize Response" option to support this, but we need to turn it on. To do this:

1. Scroll down to the Survey Termination section.
2. Select "Anonymize Response."
3. Click "Save" to exit out of the Survey Options.

Distribute survey; Cut and paste distribution list from Excel; Minimizing spam filter issues

Now that we've created the survey and set some important options, we need to distribute the survey to our voters.

For most departments, you'll want to be able to track who has already voted and who has not. To do that, we'll need to send the surveys out through the Qualtrics email function.

To start that process:

1. Click the "Distributions" tab.
2. Under Send with Qualtrics, select "Compose Email."

In the Compose Email screen, the first thing we need to do is to put in our voters on the To line. Qualtrics refers to survey recipients as "Contacts." So,

1. Click "Select Contacts."
2. Click "New Contact List."

Most departments have complicated rules about who can vote on any particular merit or promotion action such as only allowing faculty members at the same or higher rank and step as the candidate to vote.

Qualtrics doesn't have a graceful way of managing those voting rules, so I'm going to show you a useful trick, which is to use an Excel spreadsheet.

Here you can see a spreadsheet I have created with the faculty roster for our fictional department. For each faculty member, the spreadsheet has their email address, first name, last name, rank and step.

If I switch back to the Qualtrics contact list, you'll see that those first three columns--email, first name and last name--match up exactly.

Let's say that for this vote, we only want Associate and full Professors. So on my spreadsheet, I'm going to filter out the Assistant Professors.

I also don't want the candidate to be able to vote on their own action, so I'm going to filter them out.

Now that I've got the list of faculty who should be voting on this action, I'm going to:

1. Select the first three columns in Excel, and copy them.
2. Switch back to Qualtrics, and Paste them into our Qualtrics Contact List.
3. Click Create Contact List to save our list.

As you can see, using this faculty roster spreadsheet makes it easy to select just the faculty members you want on your distribution and get them into Qualtrics.

Now that we've got our contact list handled, we can move on to the rest of the email:

1. On the From line, DON'T change the "From Address," you need to leave that as-is.
2. If you want, you can change the From Name and the Reply-To Email address. For example, some departments may want the email to have the Chair's name, or you may want to direct any email replies to a specific person.
3. On the When line, choose when you want to send the message. This is a convenient option to allow you to set up ballots in advance and schedule them to go out at a later time. In this case, I'm going to leave the default of "Send in 1 hour."
4. On the Subject line, put in a reasonable email title (qq1).
5. In the Message section, you can put in the text of the email that will go out to your voters (dd1). Qualtrics starts you off with links to the survey--you'll want to leave those intact. So let's just put in some introductory text at the top.
6. Qualtrics also inserts an "opt out" link at the bottom of the message. Since we're using this for an internal, official ballot rather than a public survey, we can delete that text and add our own signature. (dd2) Note that you will get a warning about the opt out link, but you can safely ignore it.
7. To see what the email will look like, you can choose to send yourself a preview email by clicking on Send Preview Email, entering your own email address, and clicking Send.

Since you'll likely be sending many similar ballots, another trick here is to save your message so that you can re-use it later. To do that:

1. Click "Save As."
2. Under Library, select "My Library."
3. Under Name, give it a good description such as "Merit Action."
4. Then click "Save Message."

In the future, you can reuse this message by clicking in the Load Message box, then going to My Library, then selecting the name of the saved message.

Now that we've prepared our email, the last step is to click "Send."

Monitor Responses

Now that we have distributed our ballot via email, we'll want to monitor the responses.

For the purposes of this demonstration, I have switched over to a different survey that was actually sent out via email.

Still on the Distributions tab, we can see a summary of the responses we have received so far, including how the surveys went out--an email invitation, in this case--how many people received survey emails, and how many have responded, so far.

If we click on "Emails," we can see more detail.

If we click on the original email invitation, we can see the message that went out, the contacts who received it, etc.

If we click on the drop-down menu to the right of the message, we see some interesting options.

The first option we want to explore is "Download History." When we select this option, Qualtrics will give us a CSV file that we can open in Excel.

If we clean this up a bit in Excel, you'll see that it shows you who received the survey messages and specifically who has already responded and when they responded. You can use this information to track down the folks who still need to respond.

Another thing you'll see in the spreadsheet is the unique survey link that's associated with each voter. If someone complains that they either didn't receive or somehow lost the original survey email, you can send them their unique link to allow them to complete the survey.

Switching back to Qualtrics, another item under the drop down menu is "Schedule Reminder." This will let you set up a reminder email that will go out only to voters from the original message who have not yet responded.

One tip to consider here is that you may want to set up the reminder at the same time as you create the original message--just schedule it to go out closer to the voting deadline by using the "When" options--say, for 7 days out.

View results, create report; Reporting comments by vote selection

So now let's say that our voting deadline has passed, and we're ready to begin looking at the results.

We can start that by going to the "Data & Analysis" tab. Here, you can see each of the individual responses that was received.

If you wanted to, you could use this information to tally the results by hand, but that would be tedious.

Instead, we can click on the "Reports" tab. Here, you'll see that Qualtrics has automatically generated a report for us that gives us all of the information we need including a statistical breakout of the votes and a listing of all the comments.

In many cases, this default report may be all you need. If that's the case for you, just go under "Report Options," then "Export Report," then choose your preferred format, such as PDF. Then click "Export Report". Qualtrics will take just a moment to generate the report file, which you can then download and open. As you can see, the report has a bar chart showing the results, the statistical analysis, and all of the comments.

But let's take this one step further.

One common request it to see the comments broken out by each voting option. Qualtrics lets us do that with just a couple of clicks.

Let's go back to the results for our first question, then click "Add Report Breakout," then let's choose our voting option question.

Now if we scroll down in the report, we'll see that the first section remains the same. But when we get down to the second section, we'll see that the comments are categorized by vote, so, for example the comments that were submitted by people who voted for no advancement are shown under that heading.

Additional Tips and Tricks; Password protect; Survey Expiration; Custom Validation to require comment; Downloading results

Now that we've gone through the entire survey process, this video circles back to cover a few more advanced topics that you may want to incorporate into your surveys.

Back in the survey editor, the first two tips I want to cover are found in "Survey Options."

The first is Password Protection. If you want to add an extra layer of security to your voting surveys, you can add a password that your voters will have to enter in order to take the survey. One strategy is to distribute this password separately from the survey emails, such as at a faculty meeting. This can help to address concerns about a hacked email account allowing the hacker to submit a fraudulent vote.

The second option to consider is Survey Expiration. This lets you set exact times for the survey to open and close. Voters won't be able to submit votes before or after. This can help you to ensure that no votes are submitted after the deadline.

The Third tip is found under "Look & Feel." Qualtrics defaults to using a right arrow as the text on the "submit" button. You can customize the "Next Button Text" to be something more definitive, like "Submit Final Vote." Note that you probably only want to do this on a single-page survey like the one we created in this series of videos.

The fourth tip is about custom validation. One requirement that some department have in their voting rules is that "No" votes must be accompanied by a comment, while comments are optional for "Yes" or "Abstain" votes. To meet this requirement in Qualtrics:

1. Select the Comment question.
2. Under Validation Type on the right, select Custom Validation.
3. Note that this gets a bit hairy, but now we specify the logic under which validation will succeed--meaning if these things are true, then let the vote be submitted; if these things are not true, then don't let the vote be submitted.
a. For the first condition, let's select our voting question, then the "No" vote choice, then "Is Selected."
b. Let's click the plus sign to add another condition, the leave the condition as an "And", then select our comment question and response, then "Is Not Empty."
i. So what we've said here is that if the voter has registered a No vote AND the comment field is not empty--meaning that they left a comment--then allow them to proceed.
ii. But we need to add one more bit of logic:
c. Let's click the plus sign again to add another condition. But instead of "and," pick "Move to a new logic set," and change it to an "Or if." Then let's pick our voting question again and the "No" vote option, then change it to "Is not selected."
i. So what we've added here says that we'll also let the vote be submitted if it's a "yes" or "abstain" vote, even if there's no comment.
d. The last thing we need to do is to pick the message we want to send:
i. Click "Load a Saved Message," then "Global Library," then "System Defaults," then "Require Response".
ii. Then finally let's click Save.

Now that we have the custom validation logic in place, let's test it to make sure it does what we want.

1. Click "Preview Survey."
2. Select the "No" vote option.
3. Leave the comment field blank.
4. Click on the submit button.

You'll see a red error message saying that a response is required.

5. So now let's add a comment and try to submit again . . .

And this time it works.

But let's also test the other case:

6. Click "Restart Survey."
7. Pick a "Yes" or "Abstain" option.
8. Leave the comment blank.
9. Click on the submit button.

And viola, it lets us through without leaving a comment.

The Fifth and final tip I want to cover in this video is downloading raw data. Most of the time you probably won't need to do this, but if you ever have someone question the validity of the vote or if they want you to prove that the vote is really anonymous, this can be helpful.

1. Click the "Data & Analysis" tab.
2. Click "Export & Import," and select "Export Data."
3. Click "Export Data with Legacy Format."
4. Leave all the default options, and click "Download."
5. Qualtrics will take a moment to generate the results file. When it's done, the download will automatically start.

The download contains all of the raw data that Qualtrics has collected, including all metadata.

Managing Surveys; Using folders to organize; Copying an existing survey; Sharing surveys; Using groups

In this video, I'm going to cover some good practices for managing your Qualtrics surveys.

Even in a small department, you'll likely be creating many surveys each academic year, which means that things can get unwieldy very quickly.

The first tip is to use folders to organize your surveys. To begin:

1. Click on My Projects to return to a listing of all of your survey projects.
2. On the left you'll see any existing folders.
3. Click on "New Folder" to create a new folder, and give it a name like "Merits & Promotions."

Now let's say that I want to further categorize my surveys by year.

4. Right-click on my new folder, select "Create Subfolder," and give it a name like "2017."
5. Now go back to "All Projects."
6. I'll search for all of my Merit projects using the search in the upper right corner.
7. Then I'll drag each one to the new 2017 folder.

When you've got many surveys, this is a great way to keep them organized.

The second tip is for reusing previous surveys. Since most of your Merits & Promotions surveys will be very similar, you can:

1. Click on the drop-down menu to the right of any existing survey.
2. Select "Copy Project."
3. Update the name and folder, as appropriate.
4. Click "Copy Project."

The copy will include all of the questions, but none of the response data from the original. You just need to edit the survey to update any survey text.

The third tip is about sharing surveys with other Qualtrics users.

As with any critical business process, you should ensure that at least one other person has access to your merits & promotions surveys, in case you're unexpectedly unavailable.

To share a survey:

1. Click on the drop-down menu to the right of any existing survey.
2. Select "Share Project."
3. In the "Type Username or Email" blank, type in the name or email of the person you want to share with. Qualtrics will automatically search based on what you type. Note that the other person must already have a Qualtrics account--meaning that they have logged into Qualtrics at least once.
4. Once you find the correct person, click on their name, then click the Add button.
5. This will send an email notification to the other person, including giving you the option of providing a customized message.
6. Click Ok to complete the share.
7. You can now set the permissions on the share. For example, you can allow the other person to view reports, but not edit or distribute the survey.

The fourth tip is an extension of the sharing options we just covered.

If you're in a complex department where you need to share with multiple people, adding them individually to each survey can be tedious.

Qualtrics also supports groups, and you can share with the entire group instead of individually.

Unfortunately, Qualtrics doesn't allow us to delegate out the ability to manage groups, so requests for new groups or changes to group membership can be sent to IT Express via ithelp@ucdavis.edu.

Once your group has been created, to share surveys with the group:

1. Click on the drop-down menu to the right of any existing survey.
2. Select "Share Project."
3. Click "User and Group Address Book."
4. The groups available to you will be listed at the top of the address book. Choose the relevant group.
5. You can now set the permissions on the share. For example, you can allow the group members to view reports, but not edit or distribute the survey.

One big advantage to using groups is that, if the membership of the group changes, as in the case with staff turnover, removing someone from the group will remove their rights for all surveys shared with that group, and vice versa for adding new group members.