type: user

role: user

content:
I need to design an adaptive vertex reduction strategy for admin boundary polygons used in a reverse geocoder. The current approach uses a fixed max vertex count (e.g., 500) for all polygons via Douglas-Peucker simplification.

Context:
- 459K polygons for admin levels 2-8 (country down to city)
- Total: 105M vertices at max 500, consuming 804 MiB
- Median polygon has 171 vertices, P90 has 499, P99 has 500
- The polygons are used for point-in-polygon containment tests in reverse geocoding
- We want to minimize size while maintaining accuracy at boundaries

Current distribution by level:
  Level 2 (country): 1,170 polygons, avg 265 verts
  Level 3 (region): 741 polygons, avg 201 verts
  Level 4 (state): 18,801 polygons, avg 128 verts
  Level 5 (district): 21,072 polygons, avg 188 verts
  Level 6 (county): 78,061 polygons, avg 256 verts
  Level 7 (municipality): 53,223 polygons, avg 214 verts
  Level 8 (city): 286,573 polygons, avg 235 verts

The key insight: a small rural village boundary that's a simple rectangle doesn't need 500 vertices, while a complex coastline boundary (like London or Sydney) benefits from high detail.

Brainstorm approaches for adaptive/dynamic vertex budgeting that:
1. Allocates more vertices to complex boundaries (coastlines, rivers, irregular shapes)
2. Allocates fewer vertices to simple boundaries (rectangles, smooth curves)
3. Considers that the purpose is point-in-polygon containment, not rendering
4. Could significantly reduce total vertex count without meaningful accuracy loss

Consider metrics like: polygon perimeter/area ratio, vertex angular deviation, boundary complexity measures, area-based budgeting, error-bounded simplification, etc.

Don't read any files - just reason about the problem and propose concrete approaches with estimated impact.

userType: external

entrypoint: cli

gitBranch: main

slug: concurrent-humming-sonnet


type: message

role: assistant

type: text

text:
# Adaptive Vertex Reduction for Admin Boundary Polygons

## Problem Analysis

You have 459K polygons consuming 105M vertices (804 MiB). The fixed cap of 500 vertices is wasteful because most polygons don't need that many -- the median is 171, meaning half the polygons already naturally fall well below the cap. The problem is concentrated in the ~10% of polygons that hit the 500 ceiling (P90=499, P99=500), where you're applying uniform simplification regardless of shape complexity.

For point-in-polygon containment, the key accuracy constraint is different from rendering: you only care about whether a query point ends up on the correct side of the boundary. Interior detail doesn't matter. What matters is the maximum displacement of any simplified edge from the true boundary, because that displacement defines a "confusion zone" where containment results could flip.

---

## Approach 1: Error-Bounded Simplification (Recommended Primary Strategy)

**Instead of a vertex budget, set a maximum displacement tolerance.**

Run Douglas-Peucker with an epsilon (max perpendicular distance) rather than a target vertex count. Choose epsilon based on admin level:

| Level | Epsilon (meters) | Rationale |
|-------|------------------|-----------|
| 2 (country) | 500m | Country borders rarely have ambiguous queries near edges |
| 3 (region) | 200m | |
| 4 (state) | 100m | State/district boundaries matter more for address resolution |
| 5 (district) | 50m | |
| 6 (county) | 30m | |
| 7 (municipality) | 20m | Urban boundaries need precision |
| 8 (city) | 15m | Highest precision needed |

**Why this works well:** A simple rectangular village boundary collapses to 4-5 vertices at any epsilon. A complex coastline retains detail proportional to its actual geometric complexity. The epsilon directly maps to your accuracy guarantee -- you know the worst-case misclassification distance.

**Estimated impact:** Simple polygons (rectangles, smooth curves) that currently use 100-200 vertices would drop to 10-30. Complex polygons that hit 500 might grow to 600-800 where needed, or drop to 300 where the coastline detail was finer than the epsilon. Net estimate: 40-60% total vertex reduction (to ~45-65M vertices, ~350-500 MiB).

---

## Approach 2: Compactness-Ratio Budgeting

**Allocate vertex budgets proportional to boundary complexity.**

Compute the isoperimetric ratio (compactness) for each polygon:

```
compactness = 4 * pi * area / perimeter^2
```

A circle has compactness = 1.0. A long thin gerrymandered district or fjord-heavy coastline has compactness near 0. Use this to set per-polygon budgets:

```
budget = base_vertices + (1.0 - compactness) * max_extra_vertices
```

For example, with base=20 and max_extra=480:
- A compact square polygon (compactness ~0.78): budget = 20 + 0.22*480 = 126 vertices
- A complex coastline (compactness ~0.05): budget = 20 + 0.95*480 = 476 vertices
- A very irregular shape (compactness ~0.01): budget = 20 + 0.99*480 = 495 vertices

**Advantage:** Simple to compute (you already have area and perimeter from the geometry). Predictable total vertex count.

**Disadvantage:** Compactness is a single scalar -- a polygon could be compact but have one highly detailed edge. Also, very small polygons with simple shapes still get over-allocated because the ratio doesn't capture absolute scale.

**Estimated impact:** 25-40% reduction. Less effective than error-bounded because it's a proxy for complexity rather than a direct accuracy measure.

---

## Approach 3: Angular Budget (Vertex Importance Scoring)

**Keep vertices that contribute the most angular deviation.**

For each vertex in the original polygon, compute the turning angle (the angle between the incoming and outgoing edges). Vertices on straight segments have angles near 180 degrees (pi) and are removable. Vertices at sharp corners have small angles and are essential.

Algorithm:
1. Score each vertex by `|pi - turning_angle|` (deviation from straight)
2. Sort by score descending
3. Keep the top N vertices, where N is determined by either a fixed budget or until cumulative angular deviation captured exceeds a threshold (e.g., 95% of total angular deviation)

**Advantage:** Directly preserves shape-defining features (corners, inflection points). A rectangle needs exactly 4 vertices. A smooth curve needs very few.

**Disadvantage:** Doesn't consider the spatial distance of simplification error -- removing a vertex on a long edge causes more displacement than removing one on a short edge. Best used as a refinement on top of Douglas-Peucker, not a replacement.

---

## Approach 4: Area-Weighted Budget Pool

**Allocate from a global vertex budget proportional to polygon importance.**

Set a total budget (e.g., 50M vertices) and distribute:

```
weight_i = sqrt(area_i) * level_importance * complexity_i
polygon_budget_i = max(8, total_budget * weight_i / sum(weights))
```

Where:
- `sqrt(area)` gives larger polygons more vertices (they have more boundary to represent)
- `level_importance` is a multiplier (e.g., level 8 = 1.5x, level 2 = 0.5x -- or reversed depending on your query distribution)
- `complexity_i` is the compactness ratio from Approach 2

**Advantage:** Directly controls total memory usage. You pick your target (e.g., 400 MiB) and the system distributes optimally.

**Disadvantage:** Requires a global pass. If you add polygons, budgets shift. More complex to implement.

---

## Recommended Strategy: Hybrid of Approaches 1 + 2

The strongest design combines error-bounded simplification with a safety cap:

### Algorithm

```
For each polygon:
  1. Compute compactness ratio C = 4*pi*area/perimeter^2
  2. Set epsilon based on admin_level (see table above)
  3. Run Douglas-Peucker with that epsilon
  4. Set max_vertices = base + (1-C) * extra  (e.g., 20 + (1-C)*980 = up to 1000)
  5. If result exceeds max_vertices, re-simplify with vertex cap
  6. Enforce minimum of 8 vertices (preserve basic shape)
```

Step 4 acts as a safety valve so a single extremely complex polygon doesn't consume unbounded memory. But in practice, the epsilon-based simplification in step 3 will be the binding constraint for most polygons.

### Why this is best for point-in-polygon

- The epsilon gives you a direct accuracy guarantee: "no query point more than X meters from the true boundary will be misclassified"
- For containment tests, you don't need uniform detail everywhere -- you need accuracy where boundaries are close to each other, and the epsilon ensures that
- Simple shapes collapse naturally without any special-casing

### Implementation notes

- Douglas-Peucker with an epsilon is the standard mode of the algorithm (it's actually simpler than the vertex-count variant, which requires binary search on epsilon)
- Convert the epsilon from meters to degrees at the polygon's latitude: `epsilon_deg = epsilon_m / (111320 * cos(lat))`
- Or work in projected coordinates if you already do that

### Estimated total impact

| Category | Count | Current avg verts | Projected avg | Reduction |
|----------|-------|-------------------|---------------|-----------|
| Simple shapes (compactness > 0.5) | ~200K | ~180 | ~25 | 86% |
| Moderate complexity (0.1-0.5) | ~200K | ~230 | ~120 | 48% |
| Complex coastlines (< 0.1) | ~59K | ~350 | ~300 | 14% |
| **Total** | **459K** | **229** | **~100** | **~56%** |

Projected: ~46M vertices, ~350 MiB -- roughly a 56% reduction from 105M/804 MiB.

The biggest wins come from the huge number of simple inland administrative boundaries (rectangles, smooth curves following roads) that currently waste 100-200 vertices on what could be 10-30.

---

## Additional Optimization: Shared Boundary Deduplication

This is orthogonal but worth noting: adjacent admin polygons (e.g., two neighboring municipalities) share boundary segments. If you deduplicate shared edges and store them once with references, you could save another 20-30% of vertex storage. This is architecturally more complex (requires an edge index) but the savings compound with the simplification.

---

## Validation Strategy

Before committing to parameters, run this validation:

1. Take a sample of 10K random points near polygon boundaries (within 1km)
2. Test containment against both original and simplified polygons
3. Measure disagreement rate as a function of distance-to-boundary
4. Tune epsilons until disagreement rate at >50m from boundary is < 0.1%

This gives you empirical confidence that the simplification doesn't degrade geocoding quality.

### Critical Files for Implementation
Since you asked me not to read files, I'll list the files most likely relevant based on the project structure and recent commits:

- /root/traccar-geocoder/builder/tools/src/admin.rs - Core admin boundary polygon assembly logic (recent commits reference `add_admin_polygon`)
- /root/traccar-geocoder/builder/tools/src/simplify.rs - Likely contains the current Douglas-Peucker implementation with fixed vertex cap
- /root/traccar-geocoder/builder/tools/src/main.rs - Entry point where `--parallel-admin` and similar flags are parsed; where epsilon/level params would be added
- /root/traccar-geocoder/builder/tools/Cargo.toml - Dependencies; may need a geometry crate if not already present for compactness calculations

service_tier: standard

speed: standard

type: assistant