Sit & Mark an Online Recruitment Test in Quelixa

Thirty minutes, one sitting

Lina: Friday, twenty past twelve in Stockholm — twenty past ten, UTC. The paper has been open since nine.

Sofia Lindqvist: Coffee, phone in the other room, thirty minutes. Right.

David: And in Singapore, somebody is watching the register fill up.

Lina: Both sides, in this chapter: sitting the test, and marking it.

Lina: This is chapter seven. The candidates sit the paper from eight countries, the system marks every answer against its key, and the pass mark does the rest.

Ready to start

Lina: Her online tests page. On Tuesday the button said it opened soon; now it says start.

Lina: The window runs until nine tonight, UTC. She can begin whenever she likes inside it.

David: And once she starts, the thirty minutes are hers.

Lina: Hers, even if she starts at eight in the evening. The window only decides when she may begin.

Read before starting

Lina: Before the first question, the rules — the same ones Elena set on the paper, in plain words.

Lina: The clock starts when she presses start and cannot be paused. Every answer is saved as she gives it. And leaving the window is recorded.

David: She has to tick something first.

Lina: She confirms she will answer on her own and that the clock will not stop. Only then does the button wake up.

One question at a time

David: And she is in.

Lina: One question on the screen, a clock in the corner.

Lina: The clock counts down from the server's time, not her laptop's. Changing the time on her computer changes nothing.

David: And the order of the answers?

Lina: Shuffled for her. The person in Tokyo sees the same question with the answers in another order.

Lina: On the right, every question as a square. She can jump anywhere, in any order.

Not sure? Flag it

Lina: Six minutes in, question five. She answers, but she is not certain, so she flags it.

Lina: A dot appears on that square in the navigator — a note to herself, nothing more.

David: Does the recruiter see her flags?

Lina: No. A flag is hers, to find the question again before she hands in.

Halfway

Lina: Fourteen minutes in, question twelve. The progress bar is past halfway, and every answer so far is already saved on the server.

David: So if her connection drops now?

Lina: She signs in again and carries on from question twelve, with the same clock still running.

The last question

Lina: Question twenty, with under three minutes to spare — the clock has turned amber.

Lina: On the last question, the next button becomes hand in.

David: Could she hand in earlier, from the top bar?

Lina: At any point. The same check comes up either way.

One last check

Lina: And one last check before anything is final.

Lina: Twenty of twenty answered, one flagged to revisit, and the time she still has. Once she hands in, nothing can change.

David: So she can still go back to her flagged question.

Lina: She can — keep working is right there. She has already decided, and hands in.

Handed in

Lina: Handed in, at ten to eleven, UTC — twenty-seven minutes after she started.

David: And no score on the screen.

Lina: No score — on purpose.

Lina: That was the paper's rule. Results come with the test result, when the register closes, and everyone hears on the same day.

Three things automatic marking guarantees

Lina: While the others sit it, three things are worth being clear about.

Lina: Every answer is marked against the same key — the one that was reviewed before the question could be used.

Lina: Every candidate gets the same clock: thirty minutes from their own start, kept by the server.

David: And if a mark is ever questioned?

Lina: It traces back to the attempt: which answers, when, and how long it took. A score, not an assertion.

The sitting, live

Lina: Ten past two in the afternoon, UTC, and this is Elena's view of the same sitting.

Lina: One not started, two in progress, five handed in — each scored the moment they handed in.

Lina: And one flag. Tariq left the test window four times, one more than the paper allows.

David: So is he disqualified?

Lina: No. The flag is information for a person, not a verdict by the system. Elena looks at the attempt, the times and the answers, and decides it is not a concern.

Nine at night

Lina: At nine in the evening, UTC, the window closes.

Lina: One action is left: close the test and publish the results.

David: Did all eight hand in?

Lina: All eight. The last one handed in with a minute to spare.

Marked, every row

Lina: And the score sheet fills itself.

Lina: Every row says where its mark came from: the online attempt, sixteen out of twenty for Sofia, converted to the hundred the post asked for. Nobody typed these in.

David: And Tariq's row mentions his tab switches.

Lina: It does — the flag travels with the mark, so nobody forgets it was raised.

Lina: The chart puts all eight on one line, with the pass mark drawn through it. Seven clear it. One does not.

Below the line

Lina: Saturday morning, Elena opens the one application below the line.

David: Forty-five, against a pass of fifty.

Lina: Five marks short.

Lina: And the decision follows the rule — but it still needs a person to make it, with a reason.

David: The system could just reject her automatically.

Lina: It could, if the company switches that on. Northwind prefers a person to press the button, with a reason.

A reason, and a note

Lina: The reason is failed the test, and the note says exactly what happened: forty-five per cent against a fifty per cent pass mark.

David: Does Ana see that?

Lina: She is told, on her own page, with the reason. And she may appeal, like any candidate.

Sofia's result

Lina: And on Sofia's page, the result she was not shown on Friday.

Lina: Eighty per cent. Passed.

Lina: Underneath, how she was assessed — the same procedure James signed off, published to the person it was used on.

David: Does she see anyone else's marks?

Lina: Never. Only her own, and how they were reached.

Chapter seven, in five lines

Lina: Five lines.

Lina: The rules come before the first question, in plain words.

Lina: The server keeps the time, so thirty minutes means thirty minutes everywhere.

Lina: Every answer is marked against one reviewed key.

David: And the flags?

Lina: Inform a person. They do not decide anything on their own.

Lina: And the pass mark does the rest — with a reason for everyone below it.

Seven for the panel

Lina: Monday morning. Seven people passed, and the marks are on the record.

Mei Tan: So now I finally meet them.

Elena Novak: You, Kenji and Ines. On Friday, each of you with your own marking sheet.

Lina: Which is chapter eight.

Lina: Next time the panel interviews the seven, and each member marks on their own sheet before anyone compares notes.

David: See you in chapter eight.