Online Coding Assessment Tracking: Which Data?

Here’s everything about getting tracked during online coding interviews or assessments:

Various types of data are recorded when you take an online coding assessment.

For starters, the code you submit is recorded, often including every version of edits, you might make along the way.

Depending on the assessment, it may also record internet activity, computer activity, and time stamps.

So if you want to learn all about which data get recorded during online coding tests, then this article is for you.

Keep reading!

What Are Online Coding Assessments? (3 Examples)

There’s a lot to cover on this topic.

Before we get into what can or can’t be recorded during an assessment, it makes more sense to talk about what the assessments are and what they try to accomplish.

There are a lot of different aspects of coding assessments, interviews, and online tools.

I don’t have nearly enough time to go through all of it with you in a single article.

Instead, I’m going to focus on mainstream, third-party tools that are used by a lot of companies to hire coders and developers.

These tools basically test coding proficiency, and they’re used as part of the hiring process for many companies that employ software and app developers.

So, if you wanted to get a job writing software for Microsoft, then your coding proficiency will probably be tested along the way.

Now, Microsoft might make their own tests, but that’s not really the point.

The point is that hiring managers will want to verify that you know what you’re doing.

Sure, interviews, resumes, and portfolios will all play into that, but a lot of companies lean on coding tests as part of the process.

These tests are usually administered online.

So, when you’re going through the hiring process, the company will give you time to take your test and probably some login credentials.

You take the test as prescribed, and the company uses that test information as part of their hiring review.

There are more than a few resources for these kinds of tests, so I’m going to focus on three of the biggest in the business: Codility, HackerRank, and LeetCode.

#1 Codility

Let’s start with Codility.

This company is laser-focused on providing a streamlined service for companies.

Codility makes testing tools that help companies review coding and developer applicants.

Even though Codility serves a niche market, the company provides a pretty wide range of tools.

With Codility, companies can customize their own tests.

This means a few things for you if you end up going through a Codility proficiency exam.

First, no two tests are going to be identical.

The custom components allow companies to focus on what matters to them the most, so they can better identify uniquely qualified candidates.

Second, companies can set their own rules on how the tests are scored.

This means that fundamentally, Codility has to measure more than just the final lines of code you put into the testing software.

Codility measures things like time, edits, real-time typing, and even IP traffic.

The platform hasn’t announced everything that it can record, but to provide the services as described, Codility definitely tracks at least some of your computer behavior during the test.

#2 HackerRank

HackerRank takes a different approach.

There are assessment tools available with HackerRank, but that’s more of a peripheral service.

The real goal of this website is to develop professional networks that help developers and companies find each other.

You can almost think of it as an alternative to LinkedIn, catering specifically to the world of software development.

But, many companies feel a strong need to test applicants before making hiring decisions, and HackerRank has tools to help with that.

HackerRank assessments also vary quite a bit.

In general, companies can use the tools to test coding mechanics and aspects of software development.

Since those tests vary in range and application, this is another case where more than just the submitted code is recorded.

Again, HackerRank doesn’t release everything that the software is capable of doing, but it can definitely do more than see what you write.

#3 LeetCode

LeetCode takes a third approach to online coding assessment, mostly because the business model is completely different.

While the previous two tools are focused on providing resources to employers, LeetCode is designed to help developers prepare for online tests and interviews.

It’s kind of like an SAT preparation tool.

LeetCode has a ton of different assessment resources, and users put them to work for the sake of practice.

Now, it is possible to use LeetCode tests as an assessing employer; that’s just not the primary goal for LeetCode developers.

Considering this switch in perspective, LeetCode doesn’t have as much need for additional monitoring.

If you’re taking a practice test, LeetCode doesn’t need to know if you’re using outside resources or trying to cheat or anything else.

The practice test is worth as much as you put into it.

So, I couldn’t find any formal documentation where LeetCode talks about cheat prevention or other monitoring.

These things may exist for employer assessment tests.

It’s unclear. 

What Gets Recorded During Online Coding Assessments? (4 Things)

In my explanation of prominent assessment tools, I began answering this question.

Now, we can tackle it directly.

If you take an online coding assessment exam, is the exam recording more than just the lines of code that you write?

In most cases, it’s safe to assume that the answer is yes.

These tests absolutely have to measure a handful of things in order to make the assessment meaningful.

Additionally, many have cheat prevention measures baked into them.

These measures require that the testing software is able to monitor at least some aspects of computer behavior while you’re taking the test.

With that in mind, I’ll walk you through the most common things that are recorded when you take one of these tests.

#1 Time

This probably won’t surprise you, but time is a big deal for these tests, and a simple example will make that clear.

Let’s say that you hire developers for Google.

You’re working for one of the most prolific and desirable coding companies in the world.

You get thousands of applicants, and a lot of them are pretty darn good at what they do.

But, you can only hire a handful at a time.

So, how do you separate the best from the really good?

If you make every applicant take a standardized assessment test, it’s a good start, but that alone probably wouldn’t be enough.

Odds are that a lot of your applicants will be able to pass your tests.

You need an additional metric to really compare them to each other.

So, you first separate everyone who was able to complete the tasks in the test.

Then, you rank them according to how long it took them to complete the assessment.

Recording time is pretty easy for these testing resources.

Using time to rank people is natural, and it gives you another way to determine who works the best under pressure.

Time won’t be the only thing that matters, but it’s worth recording, and it’s why you can assume that these tests always measure time while they’re running.

#2 Edits

This might seem a little weird, but bear with me.

This is probably the most important aspect of assessment tests.

Generally speaking, assessment tests involve writing code in order to solve a problem that is presented within the test.

The specific problems range wildly from test to test, but writing code is a common theme.

Now, a super basic test would present you with a problem and provide a text box where you could write your code.

When you’re done, you can submit the code, and that’s what the test reviewer gets.

They can see the final form of your code.

The problem with that approach is that it only provides a final result.

Most people hiring developers want to know about your process just as much as they want to see your final code.

So, many of these assessment tools allow testers to see your coding in real time.

If they want to review the test when it isn’t live, the tests can also provide a list of each saved edit in the test.

That means that they’ll see multiple snapshots of your code as you write it in the text box.

This makes it easier for the tester to get an idea of how you go about solving problems.

It also lets them know if you just copied and pasted a bunch of code into the box.

This is why edits are recorded and why they are so important.

#3 Web Searches

Not all assessment tools will do this, but we are talking about online assessments, so the potential is always there.

Basically, since you’re connected to the assessment online, it can see if you’re doing things with the internet while you’re taking the test.

So, if you navigate away from the test while you’re taking it, then the tool knows, and potentially, the person giving you the test also knows.

This isn’t always a bad thing, and I’ll get into it quite a bit more later, but here’s the idea.

If you’re going to cheat on a test, then there’s a chance you’ll use other web-enabled resources in order to cheat.

So, monitoring IP activity can help the assessment look out for cheating.

This doesn’t mean that using the internet at all automatically constitutes cheating.

It just means that monitoring your web activity during the test is common and something you can reasonably expect.

#4 Computer Activity

Then again, it’s possible to cheat without using web resources.

If you really prepared to cheat, then you could have tools already downloaded or installed on your computer, and you might be able to access them without any internet activity.

Because of that, a lot of assessment tools also use anti-cheating software that can monitor at least some of your computer activity.

Again, this doesn’t mean that using your computer outside of the test is always taboo.

The tests vary.

Some of them allow you to use external resources.

Some don’t.

You’ll want to pay attention to the rules so that you don’t accidentally break them.

The monitoring software is common enough that you should assume it’s in place, even if that turns out not to be the case.

Do Only the Online Coding Assessment Results Matter? (3 Scenarios)

I’ve talked about how the assessments work and what kinds of information they like to monitor.

You’ve probably already come to your own conclusion.

Do these tests care about more than just the results?

The answer is a resounding yes.

There are enough testing resources out there that some of them might only focus on results (especially if we’re talking about self-reference tests from LeetCode), but in general, testers care about a whole lot more than just the final code you submit.

Let’s talk about why.

#1 External Resources

When you code for a living, you’re not writing everything from scratch every time.

That’s incredibly inefficient, and every experienced developer already knows this.

An important skill in terms of developing is knowing when you need original code and when you can copy and paste previous work.

Because of this, you’ll run into testers with competing ideas on external resources.

Some will happily let you use any code you have ever written before.

Some will even allow you to use open source code to be more efficient.

These testers are interested in your ability to put together working code.

They’re less interested in some of the more fundamental skills that you presumably have already developed.

On the other hand, there are testers that really want to take a deep look at your personal skill set.

So, they’ll ban external resources of any kind.

They will only accept code that you write from scratch during the test.

It’s important to know which case applies to a test before you take it.

If external resources are allowed, you’re handicapping yourself by writing everything from scratch.

If they aren’t allowed, then you don’t want to accidentally break the rules and tank your job prospect.

#2 Method of Development

This ties to the idea above.

Many testers care a lot more about how you develop than the literal code that you put up in the test.

Real software development isn’t exactly timed (although deadlines are a very real thing). 

Because of this, a lot of testers are more interested in your edit history than the final form of your code.

They want to see the order in which you go through things to reveal how you approach problem-solving in general.

This requires testing tools that record your activity along the way.

#3 Compiling

All of that said, the final result does still matter.

If you have two applicants who appear equal in all other ways, the one who submitted code that actually works is going to get the job.

The results are important.

In most cases, they just aren’t the only thing that matters.