Diffing across runs

Everything so far was a one-shot probe. A bot that runs every hour has a superpower a human doesn’t: memory. It can remember exactly what the API looked like last time and flag anything new — a fresh endpoint, an extra field in a response, a route that suddenly stopped requiring auth. New code is where new bugs live, and a diff is how you catch it the day it ships.

We’ll build that: fingerprint the API, save it, and on the next run compare against the saved copy. New file, diff.rb:

require 'net/http'
require 'json'
require 'uri'

BASE = 'http://127.0.0.1:4567'
SNAPSHOT = 'snapshot.json'

def get(path)
  Net::HTTP.get_response(URI("#{BASE}#{path}"))
end

Fingerprinting the surface

We don’t want to store whole responses — values change constantly and we’d drown in noise. We want the shape: for each endpoint, its status code and the set of keys it returns. That’s stable when nothing structural changes, and it moves the moment the API grows a field or a route:

ENDPOINTS = %w[/api/markets /api/markets/1 /api/debug /openapi.json]

def fingerprint
  ENDPOINTS.each_with_object({}) do |path, snap|
    res  = get(path)
    body = JSON.parse(res.body) rescue nil
    keys = body.is_a?(Hash) ? body.keys.sort : []
    snap[path] = { 'status' => res.code.to_i, 'keys' => keys }
  end
end

each_with_object({}) walks the endpoints while building up one hash — snap starts empty and we fill it, then it’s returned at the end. For each path we record the status and the sorted top-level keys. rescue nil means a non-JSON body just gives an empty key list instead of crashing.

Save, then compare

Now the memory. If there’s no snapshot yet, this run is the baseline — save it and stop. If there is one, load it and compare, path by path:

current = fingerprint

if File.exist?(SNAPSHOT)
  previous = JSON.parse(File.read(SNAPSHOT))
  changes  = 0

  (previous.keys | current.keys).sort.each do |path|
    was = previous[path]
    now = current[path]
    if was.nil?
      puts "  + NEW      #{path}"
      changes += 1
    elsif now.nil?
      puts "  - REMOVED  #{path}"
      changes += 1
    elsif was != now
      gained = now['keys'] - was['keys']
      note = gained.empty? ? '' : " new keys #{gained.inspect}"
      puts "  ~ CHANGED  #{path}  status #{was['status']}->#{now['status']}#{note}"
      changes += 1
    end
  end

  puts changes.zero? ? '[*] no changes since last run' : "[!] #{changes} change(s) — look here"
else
  puts '[*] no snapshot yet — saving baseline'
end

File.write(SNAPSHOT, JSON.pretty_generate(current))

previous.keys | current.keys unions the two path lists, so we consider every path in either run — that’s how a brand-new endpoint gets noticed. Then three cases: a path only in the new run is NEW, one only in the old is REMOVED, and one in both whose fingerprint changed is CHANGED. For changes, now['keys'] - was['keys'] is the array difference — the keys that appeared.

Run it once to lay down the baseline:

$ ruby diff.rb
[*] no snapshot yet — saving baseline

Run it again with nothing changed, and it stays quiet — no false alarms:

$ ruby diff.rb
[*] no changes since last run

Catching a change

Now simulate the API shipping something. In labapi.rb, add a field to a market — give it a 'volume' => 1_200 — and restart the lab. Run the diff again:

$ ruby diff.rb
  ~ CHANGED  /api/markets/1  status 200->200 new keys ["volume"]
[!] 1 change(s) — look here

The tool noticed a field we never told it about. On a real target that line is a prompt: a new volume on a market is probably harmless, but a new is_admin on a user, or a route that flipped from 401 to 200, is exactly the lead you want to be first to. The bot watches; you investigate only what moved.

Takeaways

• Store a fingerprint, not raw responses: status plus sorted keys is stable against noise and sensitive to structure. That’s the signal you want.
• each_with_object({}) builds a collection while iterating; File.write / File.read with JSON persist it between runs.
• Compare with set logic: a | b (union) to consider every path, a - b (difference) to find what appeared. NEW / REMOVED / CHANGED covers it.
• The payoff is triage: new code surfaces on its own, so your attention goes to what changed instead of re-reading the whole API every time.

Exercises

1. Extend the fingerprint to record response size as well, and flag a path whose body grew or shrank by more than, say, 20% — a cheap signal that a response’s contents changed even when its keys didn’t.
2. Right now a CHANGED only reports keys that were added. Also report keys that were removed (was['keys'] - now['keys']) — a disappearing field can break auth logic and is worth a look.
3. Snapshot the authenticated surface too: fingerprint /api/me and /api/admin/stats with a token, so a permissions change (a 403 becoming a 200) shows up in the diff.

Next — Part 8: the finale. We have eight scripts that each rebuild the same client. We fold them into one Scanner class with a command-line interface, wire in a scope allowlist so it can only ever touch the lab, and have it emit a coordinated-disclosure report — the jump from scripts to a tool you’d actually run.