---
title: 'Tech Design: Opaque URL Fragment Detection for Same-Document Navigation'
created: '2026-06-24'
tags:
  - tech-design
  - apple
  - navigation
  - webkit
  - url
  - crash
status: in-progress
chosen-approach: '_web_originalDataAsString via URLComponents(webKitUrl:)'
asana: >-
  https://app.asana.com/1/137249556945/project/1202406491309510/task/1215403820548743
---
**Author:** Apple team
**Project:** fix-fragment-crash (macOS / iOS browser)
**Sentry:** [APPLE-IOS-DY5A](https://errors.duckduckgo.com/organizations/ddg/issues/APPLE-IOS-DY5A/?project=8) · [DZ58](https://errors.duckduckgo.com/organizations/ddg/issues/APPLE-IOS-DZ58/?project=8) · [DZ5C](https://errors.duckduckgo.com/organizations/ddg/issues/APPLE-IOS-DZ5C/?project=8)

---

## Background

Every navigation through `WKWebView` goes through `DistributedNavigationDelegate`, which classifies each `WKNavigationAction` as a **same-document navigation** (fragment-only change, e.g. `#section`) via `isSameDocumentNavigation`. Misclassification corrupts the navigation state machine.

`isSameDocumentNavigation` relies on `hasFragment` and `.sameDocument` URL equality, both of which must correctly extract the fragment — which is where the bug lives.

### Two URL parsers in WebKit

| Parser | Used where | Fragment in opaque URLs (`data:`, `about:`, `blob:`) |
|--------|-----------|------------------------------------------------------|
| **WTF::URLParser** | WebKit internals, navigation decisions | ✅ Correctly split at `#` |
| **NSURL / CFURL** | Foundation APIs, Swift `URL` | ❌ Encodes `#` as `%23`; `.fragment` returns `nil` |

When WebKit bridges a `WTF::URL` to Objective-C it calls:

```c
CFURLCreateAbsoluteURLWithBytes(alloc, rawBytes, len, kCFStringEncodingUTF8, nil, true)
```

The raw bytes contain the literal `#`, but CFURL's old-RFC parser does not recognise `#` as a fragment delimiter in opaque-scheme URLs — it encodes it as `%23` and stores the whole thing as the opaque path. Result for `data:text/html,hello#anchor`:

| Property | WTF::URL | NSURL / Swift URL |
|----------|----------|-------------------|
| `.fragment` | `"anchor"` | `nil` |
| `.absoluteString` | `data:text/html,hello#anchor` | `data:text/html,hello%23anchor` |

---

## Problem

A multi-MB `data:` URL reaches `isSameDocumentNavigation` on the main thread. The old `hashedSuffix` implementation called `String.firstIndex(of: "#")`, iterating grapheme-by-grapheme — O(n) per navigation decision on the **main thread**. For a large enough URL this blocks the thread >10 s → RunningBoard watchdog SIGKILL (`0x8BADF00D`).

The fix must:
1. Make fragment detection fast (no O(n) grapheme scan).
2. Correctly detect fragments in opaque URLs where NSURL has already encoded `#` as `%23`.

---

## Information-Loss Chain

Fragment information is irreversibly lost between WebKit and Swift:

```
WTF::URL  (m_string = "data:…#anchor")
    ↓  createCFURL()
    ↓  CFURLCreateAbsoluteURLWithBytes(rawBytes = "…#anchor")  ← '#' visible here
    ↓  CFURL re-parses; '#' → '%23'
NSURL  (.absoluteString = "…%23anchor",  .fragment = nil)
    ↓
WKNavigationAction.request.URL          ← what Swift sees
```

Every downstream Foundation API (`URL.fragment`, `URLComponents`, `NSURLComponents`) operates on the already-broken NSURL — they all return `nil` for the fragment. No public or private WebKit API exposes the raw `WTF::URL` string … except one (see Chosen Approach).

---

## Approaches Considered

### 1. Scan `absoluteString` for `%23`  ❌

Since NSURL encodes `#` as `%23`, scanning for it recovers the fragment in many cases. Partly working, but fundamentally ambiguous: `%23` is also valid payload in `data:` URLs (SVG fill colors, anchor hrefs, CSS). The two cases produce identical `absoluteString` and cannot be distinguished from the string alone. A size guard avoids scanning large payloads but causes incorrect equality results.

### 2. `CFURLCreateAbsoluteURLWithBytes` GOT-patch hook

WebKit calls `CFURLCreateAbsoluteURLWithBytes()` at the exact moment it converts a `WTF::URL` to a `CFURL`. The raw byte buffer at that point still contains the literal `#`. The hook intercepts this call using **runtime GOT patching** (same technique as `CxaThrowSwapper` / fishhook) — not a link-time `__DATA,__interpose` section, which would not reach WebKit's internal call from `WebKit.framework` into `CoreFoundation.framework`.

Mechanism:
1. `_dyld_register_func_for_add_image` registers a callback that fires for every Mach-O image loaded into the process (including already-loaded images at registration time).
2. The callback walks the image's **indirect symbol table** (`ImageMap.rebindSymbol`) targeting only the `WebKit` dylib, finds its GOT entry for `CFURLCreateAbsoluteURLWithBytes`, and replaces it with the hook pointer. ARM64e pointer authentication is handled via `BSKStripFunctionPointer` / `BSKSignFunctionPointer`.
3. The hook calls the original via the saved pointer, then scans the raw byte buffer with `memchr` (SIMD-vectorised on ARM64/x86) for `':'` (opaque check) then `'#'`, and stores the resulting `NSRange` as an associated object on the `CFURL` result.
4. `URL.opaqueFragmentAnnotation` reads/writes that associated object; `URL.isOpaqueFragmentScanned` guards against double-scanning the same instance.
5. `uninstallCFURLSwapper()` walks all loaded images and restores the saved original pointer.

```swift
// install: register dyld callback → patch WebKit's GOT entry at runtime
_dyld_register_func_for_add_image(_cfURLProcessMachHeader)

// hook: memchr scan + associated-object annotation
if let hashRaw = memchr(searchBase, Int32(UInt8(ascii: "#")), searchLen) {
    url.opaqueFragmentAnnotation = NSRange(location: hashOff, length: len - hashOff)
}
```

**Downside:** couples to WebKit's dylib name and a private CFURL symbol; requires bypassing page protection to write into a read-only GOT page; depends on dyld image-load ordering; massive and fragile.

### 2a. `__DATA,__interpose` (static link-time interpose)  ⚠️ (option, not chosen)

A related but distinct mechanism: a `__DATA,__interpose` section in the binary redirects calls to the named C function to a replacement at load time, before any code runs. Simpler than GOT patching (no dyld callback, no image walk), and dyld applies it process-wide so it does intercept calls from other frameworks.

**Downside:** the `__DATA,__interpose` section is a well-known jailbreak / hooking fingerprint; App Store review tooling flags it, making approval unlikely.

---

## Chosen Approach: Private WebKit URL String (`_web_originalDataAsString`)

NSURL instances that originate from WebKit (via `createNSURL()`) carry a private property `_web_originalDataAsString` which holds the original `WTF::URL` byte string — **before** CFURL re-encoded `#` as `%23`. Parsing that string with Swift's native `URLComponents(string:)` gives correct path/query/fragment decomposition for all opaque URLs.

```swift
// URLComponentsExtension.swift
extension URLComponents {
    /// Parses a WebKit-origin URL using the original WTF::URL string.
    /// Falls back to URLComponents(url:) for non-WebKit or hierarchical URLs.
    init?(webKitUrl: URL) {
        guard webKitUrl.isOpaque,
              let originalString = (webKitUrl as NSURL).value(forKey: "_web_originalDataAsString") as? String,
              let swiftNativeURLComponents = URLComponents(string: originalString) else {
            self.init(url: webKitUrl, resolvingAgainstBaseURL: false)
            return
        }
        self = swiftNativeURLComponents
    }
}
```

```swift
// URLExtension.swift
public var originalWebKitString: String? {
    (self as NSURL).value(forKey: "_web_originalDataAsString") as? String
}

public var hasFragment: Bool {
    guard let components = URLComponents(webKitUrl: self),
          let fragment = components.fragment else { return false }
    return !fragment.isEmpty
}

public func equals(_ other: URL, by components: EqualityComponents) -> Bool {
    guard let selfParsed  = URLComponents(webKitUrl: self),
          let otherParsed = URLComponents(webKitUrl: other) else { return false }
    // component-by-component comparison …
}
```

### Why this is correct

- `_web_originalDataAsString` holds the `WTF::URL`'s `m_string` verbatim — `#` is still a literal character, not `%23`.
- Swift's `URLComponents(string:)` is WHATWG-aligned and correctly splits at `#`, so `.fragment` returns the real fragment.
- No ambiguity with `%23` payload: we parse the original string, not the NSURL's re-encoded `absoluteString`.
- The property is absent on non-WebKit URLs (`URL(string:)`-created values) — the initialiser falls back to `URLComponents(url:)`, which is correct for hierarchical URLs.
- Pure Swift/Obj-C KVC access; no interpose, no associated objects, no extra files.

### Call flow

```
WTF::URL  (m_string = "data:…#anchor")
    ↓  createNSURL()
NSURL  (.absoluteString = "…%23anchor",  ._web_originalDataAsString = "data:…#anchor")
    ↓
URLComponents(webKitUrl: url)
    → value(forKey: "_web_originalDataAsString") → "data:…#anchor"
    → URLComponents(string: "data:…#anchor")
    → .fragment = "anchor"  ✅
```

---

## Testing

- **Unit tests** (`URLExtensionTests.swift`):
  - `hasFragment` and `equals(by: .sameDocument / .fuzzyIdentity)` on `about:blank#section`, `data:text/html,…#anchor`, opaque URLs with query AND fragment.
  - Large (20 MB) `data:` URL: comparison must complete in < 0.1 s.
  - Diagnostic timing test (`tempOpaqueNSURLComponentTimings`) for all Foundation URL APIs on a 20 MB `data:` URL.
- **Regression**: navigation tests in `DistributedNavigationDelegateTests.swift` updated to use `.equals(by: .fuzzyIdentity)` in place of the removed `.matches()` API.

---

## Notes

- **WebKit bug:** [bugs.webkit.org/68089](https://bugs.webkit.org/show_bug.cgi?id=68089). Chromium fixed `#` handling in M72 (2019). `WTF::URLParser` is correct; only the CFURL bridge is broken.
- **Primary safety fix** is the O(n) crash: `isSameDocumentNavigation` now uses `URLComponents(webKitUrl:)` which is constant-time relative to URL length (KVC + a single string parse), replacing the old grapheme-scan loop.
- **`%23` ambiguity** only affects `data:` / `about:`. For `blob:` and `javascript:` the opaque path never legitimately contains `%23` as encoded payload.
- No user data collected or transmitted; all processing is local and in-process.