Behaviour tests score your mouse, not just your browser

A behavioral bot detection test scores what you do on the page as well as what your browser is. Incolumitas BotOrNot, the one in our catalogue that does this, watches mouse movement, scrolling and dwell, and prints a score from 0 (bot) to 1 (human) beside thirty-odd checks on the browser's attributes. Load it and touch nothing, and the score is a measurement of your stillness.

We learned that by reporting the wrong number for as long as the row existed.

The row that said "behaviour 0" every time

Our detection suite opens each public detector in a real profile and reads its verdict by machine. For Incolumitas it reported behaviour 0 on every run. That looks like a finding about the profile. It had two causes, and neither was the fingerprint.

The first was that our harness never moved anything. It opened the page, waited, and read. A classifier that rates mouse paths had no mouse path to rate, so zero was the correct answer about the visitor we sent.

The second was worse. The extractor matched /behavioral[^\d]*([\d.]+)/ against the page's visible text, and the first thing that pattern meets is the page's own help sentence, the one explaining that the score "gives a rating between 0 (Bot) and 1 (Human)". It captured that 0, every run. The real score lives in an element of its own, #behavioralScore, and the browser checks render as JSON in #new-tests and #detection-tests. Three rendered values were being ignored in favour of a digit in a sentence. Read the element, never the prose.

How to run a behavioural test so it measures something

The fix for the first cause was to behave, using the same code the profile uses everywhere else. Mosaic's warm-up robot and its page-driving tools send input through one module, src/main/engine/human-input.ts, and the detection suite imports it rather than keeping a second mouse that would drift from the first. For Incolumitas it now does this, in order:

1. Move the cursor to a random spot in the middle 60% of the viewport, then rest 0.6 seconds.
2. Scroll down 420 pixels, move again, rest 0.9 seconds.
3. Scroll back up 260, move, rest 1.2 seconds.
4. Scroll down 700, move, rest 2.5 seconds.

That is about fifteen seconds. Then it polls #behavioralScore every three seconds for up to twenty-four, because the page shows ... until its classifier has something to say, and a fixed six-second wait had reported "no number" about a page that was still thinking.

If you are testing by hand, the same applies: use the mouse the way you would on any page, scroll, wait for the score to change from its placeholder, and read the number the page puts in its result box.

How it works: REST API

What "trusted" input is, and why it is not enough

A page can tell a scripted click from a hand. element.click() dispatches one click event with isTrusted set to false and nothing around it: no pointerover, no mousemove, no pointerdown or mouseup, no focus change. window.scrollBy fires scroll with no wheel in front of it, which a wheel cannot produce. Any script that counts mousemoves before a click separates that from a person on the first sample.

So Mosaic sends input at the browser level, through the DevTools Input.dispatchMouseEvent and Input.dispatchKeyEvent commands, and the page receives the whole sequence with isTrusted true. We checked that neither command flips the CDP-automation check on deviceandbrowserinfo, which Runtime.enable does flip (why we never call it).

Trusted is still not human. A trusted cursor travelling in a straight line at constant speed is a robot with good paperwork. The motion follows the models that describe real pointing:

• Fitts' law sets how long a move takes, so far or small targets take longer, on a logarithmic curve.
• The minimum-jerk profile, 10τ³ − 15τ⁴ + 6τ⁵, sets its shape: slow at both ends, quick in the middle.
• Moves longer than 220 pixels overshoot about 28% of the time and land with a second, small correction.
• Every sample carries sub-pixel tremor.

Each profile also gets a pointing device and a tempo from its seed: a trackpad for 64% of profiles and a notched wheel for the rest, at a speed between 0.82 and 1.24 times the mean. A session that scrolls like a trackpad on one page and a wheel on the next is describing two computers.

The score needs a window someone could be looking at

The suite runs its browsers off-screen, at --window-position=-32000,-32000, so a long run does not cover the desk. Headed, not headless, because a headless Chrome names itself in its User-Agent and loses the real GPU, and we will not grade a browser we do not ship.

Off-screen, Incolumitas's score never left .... We measured fifteen seconds of real minimum-jerk movement, scrolling and dwell, and then forty-five seconds of polling, and got no number. Nothing ever focuses an off-screen window, and their classifier appears to want focus. The attribute checks rendered every time.

So the row is decided by the attribute checks, and the score is reported when it exists and named as unavailable when it does not. Run the suite with --watch and the windows are visible, and the score arrives. The longer list of what unattended runs need is in running 49 fingerprint tests unattended.

Read the two halves separately

On our last full run, all of Incolumitas's attribute tests but one returned OK. The one that did not, WEBDRIVER, expects navigator.webdriver to be absent, which stopped being true when Chrome made it a standard property that is false in an ordinary browser. Our engine and the machine's own Google Chrome failed it byte for byte the same, so it says nothing about the profile, and the row names it as stale instead of dropping it.

That is the general lesson for any behavioural test. The attribute half tells you whether the browser is consistent. The behaviour half tells you whether what drove it looked like a person during those seconds, and it is only meaningful if something drove it. A low behaviour score over a page you never touched is not a finding. Pixelscan has a similar card, and how far we could reproduce its verdict is in Pixelscan's automated behavior card.