Test iOS Vector Search on a Cloud Mac

The same query returned document A yesterday, but today document B ranks first. Before retraining the model, check the retrieval layer: candidate input order, zero vectors, and floating-point overflow can all create apparent regressions in iOS semantic search. Run fixed vectors on a cloud Mac to validate retrieval first, then inspect model output. That narrows the investigation considerably.

Define the retrieval contract before connecting model output

This test covers only exact, in-memory cosine search—not approximate indexes. It takes pregenerated vectors as input and returns document identifiers with scores. Keep model inference in a separate test suite so that a single failure does not implicate both layers at once.

The contract has five rules: vectors must have matching, nonzero dimensions; every element must be finite; zero vectors are rejected; results are sorted by descending score, with ties broken by ascending unique identifier; and the threshold is inclusive. A k of zero returns no results, a negative value raises an error, and a value greater than the candidate count returns all qualifying candidates.

A zero vector does not mean “unrelated to the query.” It has no defined direction and must not be treated like an orthogonal vector with a cosine score of zero.

The example caps the input dimension at 16384. This is a local resource budget, not a model specification. In a real project, tighten the limit to match the model you use, and also limit the number of bytes read from files.

Prevent numerical overflow with scaled normalization

Computing the sum of squares directly can produce an infinite intermediate result even when every element is finite. Dividing by the largest absolute value before calculating the length avoids overflow when squaring large numbers. Do not arbitrarily add a constant to the denominator: that changes the normalization result for vectors with small magnitudes.

Save the complete code below as vector-contract.swift. It uses only modules bundled with the toolchain and runs from the command line on a cloud Mac.

import Foundation

enum VectorError: Error {
    case invalid
}

func unit(_ values: [Double]) throws -> [Double] {
    guard !values.isEmpty,
          values.count <= 16384,
          values.allSatisfy({ $0.isFinite }),
          let scale = values.map({ abs($0) }).max(),
          scale > 0 else {
        throw VectorError.invalid
    }
    let scaled = values.map { $0 / scale }
    let length = sqrt(scaled.reduce(0) { $0 + $1 * $1 })
    return scaled.map { $0 / length }
}

struct Candidate {
    let id: String
    let vector: [Double]
}

struct Hit {
    let id: String
    let score: Double
}

func search(
    query: [Double],
    candidates: [Candidate],
    k: Int,
    threshold: Double
) throws -> [Hit] {
    guard k >= 0, threshold.isFinite,
          (-1.0...1.0).contains(threshold),
          Set(candidates.map(\.id)).count == candidates.count,
          candidates.allSatisfy({ !$0.id.isEmpty }) else {
        throw VectorError.invalid
    }
    let q = try unit(query)
    var hits: [Hit] = []
    for candidate in candidates {
        guard candidate.vector.count == q.count else {
            throw VectorError.invalid
        }
        let v = try unit(candidate.vector)
        let raw = zip(q, v).reduce(0.0) {
            $0 + $1.0 * $1.1
        }
        guard raw.isFinite, abs(raw) <= 1 + 1e-12 else {
            throw VectorError.invalid
        }
        let score = min(1, max(-1, raw))
        if score >= threshold {
            hits.append(Hit(id: candidate.id, score: score))
        }
    }
    hits.sort {
        if $0.score == $1.score { return $0.id < $1.id }
        return $0.score > $1.score
    }
    return Array(hits.prefix(k))
}

let candidates = [
    Candidate(id: "b", vector: [2, 0]),
    Candidate(id: "a", vector: [1, 0]),
    Candidate(id: "c", vector: [0, 1]),
    Candidate(id: "d", vector: [-1, 0])
]
let hits = try search(
    query: [1, 0], candidates: candidates,
    k: 3, threshold: 0
)
precondition(hits.map(\.id) == ["a", "b", "c"])
precondition(hits.map(\.score) == [1, 1, 0])

let reversed = try search(
    query: [1, 0], candidates: Array(candidates.reversed()),
    k: 3, threshold: 0
)
precondition(reversed.map(\.id) == hits.map(\.id))

let huge = try unit([
    Double.greatestFiniteMagnitude,
    Double.greatestFiniteMagnitude
])
precondition(abs(huge.reduce(0) { $0 + $1 * $1 } - 1) < 1e-12)

func rejects(_ body: () throws -> Void) -> Bool {
    do {
        try body()
        return false
    } catch {
        return true
    }
}

precondition(rejects { _ = try unit([0, 0]) })
precondition(rejects { _ = try unit([Double.nan, 1]) })
precondition(rejects {
    _ = try search(
        query: [1], candidates: candidates,
        k: 1, threshold: 0
    )
})

This implementation deliberately validates every candidate before truncating the results. Invalid data therefore raises an error even when k is zero. If the production API should short-circuit instead, define that behavior in a separate contract; do not let tests and production interpret the same contract differently.

Test ranking boundaries with geometric fixtures

Basic fixtures do not need real text. Vectors pointing in the same direction score one, opposite vectors score negative one, and orthogonal vectors score zero. Multiplying a vector by a positive scalar must not change its direction. Candidates a and b in the code both point in the query’s direction. Reversing the input must still produce identifier-based ordering, exposing implementations that depend on iteration order.

Add the following checks:

• Set k to zero, one, the candidate count, and a larger value.
• Set the threshold to negative one, zero, and one, and verify that equality is included.
• Require empty arrays, infinite values, duplicate identifiers, and dimension mismatches to fail.
• Use fixed fixtures to verify similarity symmetry and invariance under positive scaling.

Numerical assertions may use a tolerance, but the sort comparator must not treat scores as equal whenever their difference falls below a tolerance. Approximate equality is not transitive and can violate sorting requirements. If the application needs to rank results by score bands, first define the quantization rules, then compare integer bands.

Run the reference implementation in cloud validation

Run this command in an environment with the command-line developer tools installed:

xcrun swift vector-contract.swift

The script exits normally on success and returns a failure status if an assertion fails or an error goes unhandled. CI must preserve that exit status rather than overwrite it with a later successful command. If you compile before running, use a standard debug build without optimization options that remove precondition checks.

The reference implementation normalizes each vector individually and sorts the complete result set. It is suitable for validating small fixtures, not for high-performance retrieval at scale. If production caches normalized vectors, add tests to verify that the cache matches the version of the source vectors. After switching to batch computation or an index, compare the output against these reference results.

Capture reproducible run evidence

For each run, record the commit identifier, toolchain version, fixture summary, dimensions, threshold, and expected candidate order. Use only synthetic vectors or authorized, de-identified data as fixtures; never include users’ original text in public logs.

Separate algorithm regressions from model drift

If fixed-vector tests pass but real search results change, then inspect the model version, tokenization, and vector preprocessing. If the fixed-vector tests also fail, first investigate the numerical path, sorting, and candidate identities. Do not hide errors by regenerating vectors.

Finally, make the acceptance scope explicit: exact search requires consistent results; approximate indexes need a separate recall criterion; and the least significant floating-point digits may not be bit-for-bit identical across hardware. Establish valid inputs, clear boundaries, and deterministic tie-breaking before increasing the dataset size. Regression reports can then point to problems you can actually fix.

Frequently asked questions