What software can and cannot buy you

Generated from this site's git history. with Claude means the commit carries a co-author line for an Anthropic model: an LLM took part in that change, not necessarily in every word of it, and git cannot say which. The same history on GitHub →

1. v3 2026-09-12 07:54 posts: inline source cards, and every post cut to the point Joshua Anderton with Claude Opus 5 +182 −572 words

   status: published

   inline_sources: true

   tier: C

   ---

   There is a magnetometer in every phone, sitting there being a compass. The question that got me started was whether disciplined software could push it somewhere it was never meant to go.

   The short version: **software buys you enormously more than I expected, and none of it is sensitivity.** This is a walk through what was actually tried, what the other implementations in this space do, and where the line sits.

   Two months pushing a phone magnetometer past what its hardware should allow. What the discipline bought, and the wall it does not move.

   **Short version: software buys a great deal, and none of it is sensitivity.**

   ## TheWhat mentalitysoftware can do

   The instinct when you have inadequate hardware is to look for the clever trick — the processing step that recovers what the sensor missed. That instinct is mostly wrong, and it took a while to internalise why.

   What software can do, it turns out, is a long list:

   - **Verify rather than assume.** Every parameter you did not measuremeasured is a parameter youbeing are guessing atguessed.

   - **Know its own noise**, in the band that matters rather than across the whole spectrum.

   - **Refuse to believe itself** — surrogate nulls, held-out thresholds, injection tests that establishestablishing what the pipeline would have detected.

   - **Keep everything on one clock**, so that motion can be separatedseparates from field.

   - **Report its own gaps** instead of quietly interpolating them.

   ## What software cannot do

   What software cannot do is manufactureManufacture information that never entered the converter. If the quantisation step is larger than the thing you want to see, no amount of averaging, whitening or filtering invents it. That is the whole lesson and it took building the thing to learn it properly.

   ## The other implementations, and what each one is for

   I did not build this in a vacuum, and the most useful hours were spent reading what other people had already done. Four are worth documenting, because each is built for a different purpose and each is better than mine at its own job.

   Four, each better than this rig at its own job.

   **NOAA's CrowdMag** collects— smartphone magnetometer readings as a distributed geomagnetic survey. Its defining property is that **it validatesValidates every contribution against a field model**, —so it knows whether a reading is *correct*, in absolute terms. Mine does not, and has never checked.

   *correct* in absolute terms. This rig does not, and has never checked.

   **phyphox and/ Physics Toolbox** are— physics-education tools. Their defining property is that they **expose theExpose raw and the calibrated channelchannels side by side**, switchable in two taps. That distinction matters more than it sounds and I will come back to it.

   **TheNASA NASA/Odenwald citizen-science worksmartphone magnetometry** — characterises what a phone magnetometer can do for space-weather monitoring {% include cite.html key="odenwald2022" %}, and is the closest published reference point for the noise performance of this sensor class of sensor.

   {% include source.html key="odenwald2022" %}

   **ADVIO** is— a visual-inertial odometry research dataset —, not a magnetometry project at all, but it ships real multi-sensor phone recordings under an open licence,. whichThat makes it a cross-device control for anyone testing whether an effect isbelongs a property ofto one handset or ofto the platform.

   **How Ithey useare themused here.** CrowdMag is the model for an absolute-accuracy check I have not yet built. phyphox is the reason I knowwhy the dualsingle-channel decision is a defect andrather notthan a preference. ADVIO is the corpus that lets a finding be checked on hardware I do not ownowned. None of that required permission; all of it required reading.

   ## What was actually tried

   **Measure the delivered sample rate, per recording.** Not the requested rate — the delivered one. It is not the same number, theThe difference is systematic, and every frequency-domain result is wrong by that factor if you assume instead of checkassumed. None of the implementations abovecomparators verifies this per record.

   **Measure the noise floor in-band, not broadband.** This one produced the most useful correction of the whole project, in the opposite direction from the one you would fear: a broadband figure **overstated** the noise at the frequency of interest by a large factor, because real recordings are drift-dominated. Carrying the wrong number around declares runnable experiments unrunnable. An instrument that makes you *too* pessimistic kills experiments silently, and nothing in the output tells you it happened.

   **overstated** noise at the frequency of interest, because real recordings are drift-dominated. Carrying the wrong number declares runnable experiments unrunnable. An instrument that makes you too pessimistic kills experiments silently.

   **Build the nulls before the result.** Four surrogate families of surrogate, thresholds taken as a maximum statistic over the entirewhole search rather than per-cell, and a refusal to quote a false-alarm bound above a stated ceiling. The comparators run quality control against a model; none of them runs a null. This is theBiggest single biggest difference, and it is entirely software.

   **Validate by injection.** Put a known signal into the real path and measure what fraction the pipeline recovers. Without this you have no idea whetherit, a null result meanscannot absencebe ordistinguished meansfrom youran analysis that would have missed it anyway.

   **One monotonic timebase across every sensor stream.** Magnetometer, accelerometer, gyroscope, attitude — all on one clock. Almost every useful discriminator for separating "the phone moved" from "the field changed" depends on thisit, and it iscannot impossiblebe to retrofitretrofitted.

   ## Where it is worse, including one real defect

   Being honest about this is most of the value of having done the comparison.

   **The calibrated channel is not logged, and that is a genuine defect.** Recording the raw channel as primary is right — calibration is a filter with unpublished behaviour, and you cannot undobe itundone. But the calibrated channel alongside it is a *free diagnostic*, phyphox does it, and its absence directly caused a multi-day misattribution: a large static field offset was read as an external accessory when it was internal to the device. A diagnostic channel I chose not to record would have shown that immediately. It is the cheapestCheapest fix on mythe list and it; needs an app build.

   **No absolute accuracy validation.** CrowdMag would have caught the same thing a different waydifferently.

   **One operating system, and a private archive.** The cross-device control leans on other people's corpora because mine is not shareable.

   **Dropped samples across long recordings**, reported rather than hidden — and the "zero dropped gaps" figure that circulated in my ownearly notes came from a single short run and does not survive contact with hours of recording.

   ## The wall

   The target for reading cardiac activity magnetically is described in the review literature.

   {% include source.html key="roth2024" %}

   The target for reading cardiac activity magnetically is described in the review literature {% include cite.html key="roth2024" %}, and the gap between it and a phone is not a factor of a few. It is orders of magnitude, and it is a property of the sensor.

   That is the part worth stating plainly, because it is where the mentality I started with runs out. Every item in the "what software can do" list above is real, and I would defend all of it as worth building. Not one of those things moves the floor. They tell you where the floor *is*, they stop you fooling yourself about what is above it, and they make a null result mean something — which is a great deal, and is not the same as sensitivity.

   **The honestHonest summary:.** for what it is actually for —For searching for weak periodic structure with real nulls —, this is a more careful instrument than the comparators, and the gap is not close. For what the comparators are built for, it is worse, and one of those gaps is a defect rather than a trade-off.

   ## What would change my mind

   IfA someone demonstratesdemonstration that a stock phone magnetometer resolves a signal at the amplitude the cardiac literature describes, under a protocol with real nulls and an injection test, then the sensitivity wall I am describing is not where I think it is and this post is wrong. I would want to see theThe null machinery would need to come before the result, which is— the same standard Iapplied am holding myself tohere.

   Commit fb7ef7b on GitHub →

2. v2 2026-09-03 15:17 site: images on every post, bigger thumbnails, CTA moves into the sidebar Joshua Anderton with Claude Opus 5 +50 −0 words

   - odenwald2022

   image: /assets/posts/fluxgate-magnetometer.jpg

   image_alt: >-

   A fluxgate magnetometer under a glass dome in a museum case, its permalloy cores and mounting plates visible.

   image_credit: Daderot

   image_license: CC0

   image_license_url: https://creativecommons.org/publicdomain/zero/1.0/

   image_source_url: https://commons.wikimedia.org/wiki/File:Fluxgate_magnetometer,_of_the_type_installed_in_a_lunar_surface_probe_for_the_Apollo_Project_-_National_Museum_of_Nature_and_Science,_Tokyo_-_DSC07833.JPG

   summary: >-

   Commit cb0df37 on GitHub →

3. v1 2026-09-01 12:19 posts: the magnetometer walkthrough, and the wave-information survey Joshua Anderton with Claude Opus 5 first version

   v1: the original version, as first committed. Read it on GitHub →

   Commit 3f615ec on GitHub →