Setting Up Screening Questions and Using AI Check

In this guide, you'll learn how to set up effective screening questions and use our AI Check feature to focus on the most promising applicants instead of spending your valuable time on low-quality responses.

The Golden Rule: Keep It Short

For candidate screening, you want to keep your test to 5 questions maximum, taking no more than 5 to 10 minutes to complete. Why? Because lengthy tests deter quality candidates. Save comprehensive assessments for later stages of your hiring process when candidates are already invested.

Step 1: Start with CV or LinkedIn

The starting point for all screening tests is always asking for a CV or LinkedIn profile. You can ask candidates to provide both options, but we highly recommend choosing just one. This keeps the barrier low for candidates and prevents them from feeling overwhelmed.

Setting Question Time Limits

The question time limit is for indication purposes – it helps candidates understand the total time commitment upfront, but it isn't enforced during the test. Once candidates open your assessment page, they'll see the approximate time it takes to complete it, which sets clear expectations.

Step 2: Add Your Questions

You have two ways to build your screening test: create questions from scratch or use our question library.

Creating Questions from Scratch

Click "Add New Question" and you can format the question however you like. Mark it as required or non-required and choose the appropriate answer type based on what you need to learn about candidates.

Multiple-Choice Questions

1. Go to Screening Tests and select your screening test.
2. Click Manage Questions (or Edit next to Questions).
3. Click + New Question.
4. In Answer Type, choose Multiple Choice.
5. Type your question (e.g. "What is our product's launch year?").
6. Click + Add Option for each choice and enter options like "2018", "2019", "2020", "2021".
7. Mark the correct answer by clicking the circle next to it.
8. Configure settings: Required, Duplicate, Delete, or Reorder.
9. Click Save Question.

Text-Response Questions

1. Click Add Question and choose Text as the answer type.
2. Enter your question (e.g. "What was the average account size you managed?").
3. Toggle on AI Check (ChatGPT) if you want automatic evaluation.
4. Under Prompt, describe both correct and acceptable answers.
5. Save your question.

Video-Response Questions

1. Click Add Question and choose Video as the answer type.
2. Enter your prompt (e.g. "Walk me through how you manage client calls.").
3. Configure settings: Required, Attempts (allow retries), and Hints.
4. Note: Videos are capped at three minutes maximum to respect everyone's time.

Note: For video responses, we cap them at three minutes maximum. It's designed to save your time during review, force candidates to be concise and focused, and respect everyone's time while still getting meaningful responses. Don't worry about candidates getting it perfect on the first try; they can retry their video responses as many times as they need.

Organizing with Sections

For longer assessments, you can use sections to break up your test. You might have initial qualification questions followed by a section introduction like "Now we're moving to the test task. Please watch this video carefully as I describe your use case, then answer the following questions."

To add a section, enter a section name and optionally add a section introduction video. It works the same as the screening test introduction video: allow access to your camera, record your explanation about the new section, stop, save, confirm, and wait while it uploads.

Using the Question Library

Click "Add from Library" to access our extensive collection of questions suitable for all kinds of screening tests, including screening test-specific ones.

Step 3: Enable AI Check for Smart Screening

This is where things get powerful. A lot of questions in our library come with AI Check, which is basically like having a 24/7 virtual assistant who pre-screens candidates for you based on their answers and the context you share with it.

What's meant by context? When you look at a question with AI Check enabled, you'll see fields for "correct answer" and "acceptable answer." This context tells the AI what to look for when evaluating candidate responses. It works for both text answer types and video answer types.

Enabling AI Check on Custom Questions

Want to add AI Check to your own questions? Click "Enable AI Check" when creating or editing a question. Then provide the correct answer context (what constitutes an ideal response) and the acceptable answer context (what meets minimum requirements). Once that's done, the AI will screen candidates for you automatically.

Step 4: Review Candidates with AI-Powered Scoring

When you return to review candidates, you'll see compatibility scores that help you prioritize your time. This is where AI Check really shines.

Here's the reality: you have a fixed amount of time each day for candidate screening. When you have high application volumes – especially for entry-level or customer-facing screening tests – you want to invest that time in the most promising candidates. Instead of spending hours mechanically going through dozens of low-quality responses, you can spot outliers right away and focus your energy where it matters most.

Diving into Individual Candidates

Let's look at how this works for a specific candidate. When you open a candidate profile, you'll see several things:

The AI skims through their resume or LinkedIn profile to assess if it's a match. You'll see the candidate's actual answers alongside the AI summary and scoring. For questions with AI Check enabled, you'll see the context of the correct and acceptable answers, the candidate's response, and the AI's evaluation.

In some edge cases, our AI can make mistakes. But it definitely saves you a lot of time initially, and here's the key: the more you correct it, the more it learns your specific case and becomes more accurate over time.

You can always override the AI's assessment. If you review a response and disagree with the AI scoring, just change it manually and click Save Changes. Your feedback helps the system learn what matters to you.

Developing Your Own Thresholds

As you use the system more, you'll develop a sense for score thresholds. For example, you might learn that candidates scoring below 30% compatibility rarely work out. This insight allows you to skip them entirely and focus on higher-scoring applicants instead, making your screening process even more efficient.

Why You Should Try AI Check

AI Check is available in our free 14-day trial, and we're confident that once you try it, you won't want to live without it. The time savings and improved candidate quality speak for themselves.

Step 5: Finalize Your Questions

Once you have all your questions ready, click Next to move on to publishing your screening test.

Your Next Steps

You've now learned how to set up effective screening questions and leverage AI Check to transform your candidate review process. In the next guide, we'll show you how to publish your position and start receiving applications.