obsidian-vault/work/tech-design/opaque-url-fragment-detection.md at a7e7cff8c8fa08c2e10dcfed9b68b316008af763

git_admin/obsidian-vault

Fork 0

Files

T

Alexey Martemyanov a7e7cff8c8 [2026-06-30] eagle: work/tech-design/opaque-url-fragment-detection.md

2026-06-30 18:27:24 +06:00

9.2 KiB

Raw Blame History

title, created, tags, status, chosen-approach, asana

title

created

Background

Every navigation through WKWebView goes through DistributedNavigationDelegate, which classifies each WKNavigationAction as a same-document navigation (fragment-only change, e.g. #section) via isSameDocumentNavigation. Misclassification corrupts the navigation state machine.

isSameDocumentNavigation relies on hasFragment and .sameDocument URL equality, both of which must correctly extract the fragment — which is where the bug lives.

Two URL parsers in WebKit

Parser	Used where	Fragment in opaque URLs (`data:`, `about:`, `blob:`)
WTF::URLParser	WebKit internals, navigation decisions	✅ Correctly split at `#`
NSURL / CFURL	Foundation APIs, Swift `URL`	❌ Encodes `#` as `%23`; `.fragment` returns `nil`

When WebKit bridges a WTF::URL to Objective-C it calls:

CFURLCreateAbsoluteURLWithBytes(alloc, rawBytes, len, kCFStringEncodingUTF8, nil, true)

The raw bytes contain the literal #, but CFURL's old-RFC parser does not recognise # as a fragment delimiter in opaque-scheme URLs — it encodes it as %23 and stores the whole thing as the opaque path. Result for data:text/html,hello#anchor:

Property	WTF::URL	NSURL / Swift URL
`.fragment`	`"anchor"`	`nil`
`.absoluteString`	`data:text/html,hello#anchor`	`data:text/html,hello%23anchor`

Problem

A multi-MB data: URL reaches isSameDocumentNavigation on the main thread. The old hashedSuffix implementation called String.firstIndex(of: "#"), iterating grapheme-by-grapheme — O(n) per navigation decision on the main thread. For a large enough URL this blocks the thread >10 s → RunningBoard watchdog SIGKILL (0x8BADF00D).

The fix must:

Make fragment detection fast (no O(n) grapheme scan).
Correctly detect fragments in opaque URLs where NSURL has already encoded # as %23.

Information-Loss Chain

Fragment information is irreversibly lost between WebKit and Swift:

WTF::URL  (m_string = "data:…#anchor")
    ↓  createCFURL()
    ↓  CFURLCreateAbsoluteURLWithBytes(rawBytes = "…#anchor")  ← '#' visible here
    ↓  CFURL re-parses; '#' → '%23'
NSURL  (.absoluteString = "…%23anchor",  .fragment = nil)
    ↓
WKNavigationAction.request.URL          ← what Swift sees

Every downstream Foundation API (URL.fragment, URLComponents, NSURLComponents) operates on the already-broken NSURL — they all return nil for the fragment. No public or private WebKit API exposes the raw WTF::URL string … except one (see Chosen Approach).

Approaches Considered

1. Scan `absoluteString` for `%23` ❌

Since NSURL encodes # as %23, scanning for it recovers the fragment in many cases. Partly working, but fundamentally ambiguous: %23 is also valid payload in data: URLs (SVG fill colors, anchor hrefs, CSS). The two cases produce identical absoluteString and cannot be distinguished from the string alone. A size guard avoids scanning large payloads but causes incorrect equality results.

2. `CFURLCreateAbsoluteURLWithBytes` hook

WebKit calls CFURLCreateAbsoluteURLWithBytes() at the exact moment it converts a WTF::URL to a CFURL. The raw byte buffer at that call site still contains the literal #. Intercepting this C function via the Mach-O __DATA,__interpose section (without method swizzling) gives unambiguous access to the original bytes — scan for 0x23 in a simple C loop, store the byte range as an associated object on the bridged NSURL.

#define DYLD_INTERPOSE(_replacement, _replacee) \
    __attribute__((used)) \
    static struct { const void *replacement; const void *replacee; } \
    _interpose_##_replacee \
    __attribute__((section("__DATA,__interpose"))) = \
    { (const void *)&(_replacement), (const void *)&(_replacee) }

Callers read NSURL.ddg_fragmentByteRange for the unambiguous # position. Could be limited to #ifdef DEBUG so there is zero production overhead.

Downside: adds Objective-C interpose files (CFURLCreateAbsoluteURLWithBytesInterpose.m, NSURL+FragmentByteRange.h/.m), couples to a private CFURL entry point, and requires the associated-object write to happen before any Swift code reads the URL — fragile under future WebKit changes.

2a. `dyld_interpose` at the dyld level ⚠️ (option, not chosen)

Same concept as above but applied via DYLD_INSERT_LIBRARIES + DYLD_INTERPOSE at the dynamic linker level, outside the process binary entirely. More powerful (intercepts calls from any dylib), but more invasive, not available in App Store binaries, and harder to confine to DEBUG.

Chosen Approach: Private WebKit URL String (`_web_originalDataAsString`)

NSURL instances that originate from WebKit (via createNSURL()) carry a private property _web_originalDataAsString which holds the original WTF::URL byte string — before CFURL re-encoded # as %23. Parsing that string with Swift's native URLComponents(string:) gives correct path/query/fragment decomposition for all opaque URLs.

// URLComponentsExtension.swift
extension URLComponents {
    /// Parses a WebKit-origin URL using the original WTF::URL string.
    /// Falls back to URLComponents(url:) for non-WebKit or hierarchical URLs.
    init?(webKitUrl: URL) {
        guard webKitUrl.isOpaque,
              let originalString = (webKitUrl as NSURL).value(forKey: "_web_originalDataAsString") as? String,
              let swiftNativeURLComponents = URLComponents(string: originalString) else {
            self.init(url: webKitUrl, resolvingAgainstBaseURL: false)
            return
        }
        self = swiftNativeURLComponents
    }
}

// URLExtension.swift
public var originalWebKitString: String? {
    (self as NSURL).value(forKey: "_web_originalDataAsString") as? String
}

public var hasFragment: Bool {
    guard let components = URLComponents(webKitUrl: self),
          let fragment = components.fragment else { return false }
    return !fragment.isEmpty
}

public func equals(_ other: URL, by components: EqualityComponents) -> Bool {
    guard let selfParsed  = URLComponents(webKitUrl: self),
          let otherParsed = URLComponents(webKitUrl: other) else { return false }
    // component-by-component comparison …
}

Why this is correct

_web_originalDataAsString holds the WTF::URL's m_string verbatim — # is still a literal character, not %23.
Swift's URLComponents(string:) is WHATWG-aligned and correctly splits at #, so .fragment returns the real fragment.
No ambiguity with %23 payload: we parse the original string, not the NSURL's re-encoded absoluteString.
The property is absent on non-WebKit URLs (URL(string:)-created values) — the initialiser falls back to URLComponents(url:), which is correct for hierarchical URLs.
Pure Swift/Obj-C KVC access; no interpose, no associated objects, no extra files.

Call flow

WTF::URL  (m_string = "data:…#anchor")
    ↓  createNSURL()
NSURL  (.absoluteString = "…%23anchor",  ._web_originalDataAsString = "data:…#anchor")
    ↓
URLComponents(webKitUrl: url)
    → value(forKey: "_web_originalDataAsString") → "data:…#anchor"
    → URLComponents(string: "data:…#anchor")
    → .fragment = "anchor"  ✅

Testing

Unit tests (URLExtensionTests.swift):
- hasFragment and equals(by: .sameDocument / .fuzzyIdentity) on about:blank#section, data:text/html,…#anchor, opaque URLs with query AND fragment.
- Large (20 MB) data: URL: comparison must complete in < 0.1 s.
- Diagnostic timing test (tempOpaqueNSURLComponentTimings) for all Foundation URL APIs on a 20 MB data: URL.
Regression: navigation tests in DistributedNavigationDelegateTests.swift updated to use .equals(by: .fuzzyIdentity) in place of the removed .matches() API.

Notes

WebKit bug: bugs.webkit.org/68089. Chromium fixed # handling in M72 (2019). WTF::URLParser is correct; only the CFURL bridge is broken.
Primary safety fix is the O(n) crash: isSameDocumentNavigation now uses URLComponents(webKitUrl:) which is constant-time relative to URL length (KVC + a single string parse), replacing the old grapheme-scan loop.
%23 ambiguity only affects data: / about:. For blob: and javascript: the opaque path never legitimately contains %23 as encoded payload.
No user data collected or transmitted; all processing is local and in-process.

9.2 KiB Raw Blame History