My agent refused to put anything in my cart, because it had decided my session had expired. It decided that while looking at my cart page, which Amazon does not serve to anonymous visitors.
The check it used looked perfectly sensible when I wrote it:
@signed_out_markers [
~s(<form name="signin),
~s(id="ap_email"),
"identifiez-vous",
"sign in to your account"
]
A page counted as authenticated when none of those matched. Read it out loud and it still sounds like reasoning: if the page is not asking me to sign in, I must be signed in.
Half these markers are French because the agent drives amazon.fr. They are quoted here exactly as the code has them, since a marker you cannot grep for is no use to anybody.
Amazon ships a hidden sign-in flyout in its navigation, on every page, whether you are signed in or not. The regression test mirrors the real page I was looking at:
<html><body>
<div id="nav-tools">
<a href="/gp/flex/sign-out.html">Se deconnecter</a>
</div>
<div id="nav-flyout-accountList" style="display:none">
<a class="nav-action-signin-button" data-nav-role="signin">Identifiez-vous</a>
</div>
</body></html>
There is a sign-out link, so this page is unambiguously signed in. There is
also a hidden flyout carrying the exact word my check read as proof of the
opposite. display: none means nothing at all to String.contains?/2.
A negative test does not know what it is looking at
The tooltip is the funny half. The instructive half is that this failure has no preferred direction.
A signed-in page that happens to contain one of those strings somewhere harmless is reported as expired, which is what happened to me. And a page I never anticipated at all, a throttle response, a 503, an error template, contains none of the four strings, so it is reported as signed in. The test never had an opinion about which page it was holding. It could only tell me that four particular strings were absent, which is true of very nearly every page on the internet.
The fix inverts it:
@signed_in_markers [~s(/gp/flex/sign-out.html)]
def authenticated?(html) do
normalised = String.downcase(html)
Enum.any?(@signed_in_markers, &String.contains?(normalised, &1))
end
One marker, and the only one I could find that cannot exist without an active
session. nav-link-accountlist and compte et listes were both candidates,
and both lost, because Amazon renders them for anonymous visitors too. An
unrecognised page now fails closed, which for a tool that spends money is the
answer I want.
The third time
I would like to present that as a clean piece of reasoning. It was not. It was the third time I had made this same mistake in this project, and I only recognised the family on the third one.
The 404 that told me to solve a captcha
My anti-bot guard looked for the sentence “to discuss automated access to amazon data”. It sounds like something a page would say to a bot it had just caught. It is footer boilerplate, and Amazon serves it on ordinary error pages. On the French page-not-found template I later recorded as a fixture, it is not even rendered content:
<!--
Pour discuter de l'accès automatique aux données d'Amazon, veuillez contacter
notre équipe à l'adresse api-services-support@amazon.com.
-->
Same boilerplate, one locale over, sitting inside an HTML comment on a page whose title is “Page introuvable”.
So asking for a product that does not exist raised a BotChallenge, whose
message tells the human to go and solve a captcha in a visible browser. There
was no captcha. There was no product either, which was the thing nobody was
being told.
The throttle that looked like markup drift
Dropping that marker fixed the 404 and quietly broke something else.
The same boilerplate appears on Amazon’s 503 throttle page, and that page
carries none of the three real challenge markers. So a throttled request
walked straight past the guard, reached the parser, found no title where a
title should be, and raised SelectorDrift, whose message asserted that
Amazon’s markup had likely changed.
One belief, two wrong answers. I was sent to read CSS selectors at the exact moment the correct move was to wait a few minutes and space out my requests.
Writing the rule where the mistake lives
By the third occurrence the pattern was hard to miss, so the rule went into the module rather than into my memory:
# Authentication must be proven by what is present, not inferred from what
# is absent. A negative test (scanning for "signed-out" markers) fails on
# any unexpected page: a throttle page, a 503, or a signed-in page that
# merely contains the phrase somewhere harmless.
It produces an asymmetry in the guard that I like a great deal more than the
symmetry I had before. A challenge marker is sufficient on its own, because
“enter the characters you see” and the /errors/validatecaptcha path are
challenge UI: they cannot show up on a page that is not one. The throttle page
gets stricter treatment, because the only sentence naming its cause is the
same boilerplate that fooled me on the 404:
@throttle_markers [
"<title>503 service unavailable</title>",
"trop grand nombre de requêtes"
]
def assert_not_throttled(html) do
normalised = String.downcase(html)
if Enum.all?(@throttle_markers, &String.contains?(normalised, &1)) do
Enum.find in one, Enum.all? in the other. That is not a style difference,
it is the quantifier each piece of evidence has earned.
The error that asserted a cause it could not know
The corollary turned out to be worth more to me than the rule itself.
SelectorDrift claiming “Amazon markup likely changed” was not a small
inaccuracy in a message. It was a guess presented as a finding, and it bought
me an afternoon of reading selectors while being rate limited. So the
exception stopped guessing:
@moduledoc """
A required selector was not found. Deliberately silent on why: markup
drift is one possible cause, but so is a page the caller never asked
for (a challenge, a throttle response, ...) slipping past every earlier
guard.
"""
It now carries the little it can actually stand behind:
def with_diagnostics(html, fun) do
fun.()
rescue
error in SelectorDrift ->
reraise %{error | page_title: page_title(html), page_bytes: byte_size(html)}, __STACKTRACE__
end
A page title and a byte count are not a diagnosis. They are what the next occurrence needs in order to be judged, which is a more honest thing for an error to offer than a cause inferred from a missing string. The page itself is never kept past the parse call, so those two fields are the whole record.
Two questions before the next detector
I ask both of these now, before writing any code that decides what a page is.
What is present here that cannot be present on any other page? If the answer is “nothing, but these things are missing”, I do not have a detector. I have a coincidence that has not failed yet.
And what is this error about to assert that it cannot know? An error message is read by a tired human at the worst possible moment. Mine spent three separate occasions sending that human to fix the wrong thing.
The tests ended up reading like the specification the rule turned out to be:
does not treat the automated-access notice alone as a challenge
an ordinary product-not-found page is never mistaken for a throttle page
the automated-access notice alone is not enough to call it throttled
an authenticated page carrying the hidden sign-in tooltip is recognised as signed in
Comments, questions, or just a reaction?
Send an email to ~bounga/bounga.org-discuss@lists.sr.ht. It is public and archived, so other readers can follow along and answer too.
