The file /root/traccar-geocoder/builder/OPTIMIZATION_LOG.md has been updated successfully.

# Build Optimization Log

## Baseline Timings (54m 1.7s total)
Build: `--multi-output --continents --multi-quality` (planet PBF → S3 layout)
- 379 files, 93 GiB output
- 32-core / 174 GiB RAM machine

### Phase Breakdown
| Phase | Time | Notes |
|---|---|---|
| Pass 1: relation scanning | 46.6s | Single-threaded PBF read |
| Pass 2: node processing | 184.4s | 28 threads |
| Pass 2b: way processing | 132.3s | 28 threads |
| Admin assembly (all) | 141.7s | Mixed parallel/sequential |
| S2 cell computation | 251.5s | 28 threads |
| Rebuild cell maps | 224.2s | 2 async tasks (~15 cores) |
| Planet write + qualities | 42.0s | Mixed |
| Continent filter (sum) | 1621.8s | Sequential, ~8 threads each |
| Continent write (sum) | 465.7s | Sequential |
| **Total** | **54m 1.7s** | |

---

## Round 1: Parallel continents + parallel cell map rebuild
**Commit**: `d2901da`

### Changes
1. **Bounded concurrent continents**: Run up to `hardware_concurrency / 8` continents simultaneously (4 on 32-core). Each uses 8 threads for filter_by_bbox.
2. **Parallel cell map rebuild**: Split sorted pairs at cell boundaries into N chunks, each thread builds its own sub-map, then merge with move semantics.

### Results (28m 36.6s total — 47% faster)
| Phase | Baseline | Round 1 | Change |
|---|---|---|---|
| Pass 1 | 46.6s | 91.6s | +45s (disk cache) |
| Pass 2 | 184.4s | 181.7s | -3s |
| Pass 2b | 132.3s | 124.9s | -7s |
| Admin assembly | 141.7s | 142.2s | same |
| S2 computation | 251.5s | 245.6s | -6s |
| Rebuild cell maps | **224.2s** | **169.8s** | **-24%** |
| Planet write + qualities | 42.0s | 51.6s | +10s |
| Continent processing | **2087.5s** | **621.7s** | **-70%** |
| **Total** | **54m 1.7s** | **28m 36.6s** | **-47%** |

### Continent Detail (concurrent, 4 at a time)
| Continent | Baseline filter | Baseline write | Round 1 total |
|---|---|---|---|
| Africa | 162.1s | 16.1s | 19.7s |
| Asia | 234.4s | 93.1s | 107.6s |
| Europe | 277.4s | 121.2s | 124.7s |
| North America | 301.0s | 172.8s | 191.2s |
| South America | 177.3s | 29.6s | 71.6s |
| Oceania | 164.9s | 22.1s | 48.4s |
| Central America | 159.5s | 10.8s | 15.3s |
| Antarctica | 145.2s | 0.0s | 0.3s |
| **Wall clock** | **2087.5s** | | **621.7s** |

---

## Round 2: [Investigating]
### Remaining bottlenecks
1. **Rebuild cell maps**: 169.8s — still the largest single phase. The hash map merge step is sequential.
2. **Pass 1**: 91.6s — inflated by disk cache; normally ~45s. Single-threaded, unavoidable.
3. **S2 covering drain**: 135.9s — fully parallel, hard to optimize.
4. **S2 street ways**: 136.0s — fully parallel, hard to optimize.
5. **Continent filter**: overlapped but each still ~150-400s. Could the filter itself be faster?

-| Pass 1: relation scanning | 46.6s | ~100% | Single-threaded PBF read |

-| Pass 2: node processing | 184.4s | ~2000% | 28 threads, good utilization |

-| Pass 2b: way processing | 132.3s | ~2000% | 28 threads, good utilization |

-| Admin: ring assembly | 2.7s | ~2800% | Fully parallel |

-| Admin: parallel simplify | 1.2s | ~2800% | Fully parallel (uncapped = minimal work) |

-| Admin: append + submit S2 | 2.9s | ~100% | Sequential (append to vectors) |

-| Admin: S2 covering drain | 134.4s | ~2800% | Thread pool processing |

-| S2: street ways | 136.9s | ~2800% | Fully parallel |

-| S2: interp ways | 0.0s | ~2800% | Tiny dataset |

-| S2: sort + group | 8.8s | ~2800% | Parallel tree merge |

-| S2 cell computation total | 251.5s | | |

+| Pass 1: relation scanning | 46.6s | Single-threaded PBF read |

+| Pass 2: node processing | 184.4s | 28 threads |

+| Pass 2b: way processing | 132.3s | 28 threads |

+| Admin assembly (all) | 141.7s | Mixed parallel/sequential |

+| S2 cell computation | 251.5s | 28 threads |

+| Rebuild cell maps | 224.2s | 2 async tasks (~15 cores) |

+| Planet write + qualities | 42.0s | Mixed |

+| Continent filter (sum) | 1621.8s | Sequential, ~8 threads each |

+| Continent write (sum) | 465.7s | Sequential |

-### Dedup + Rebuild Phase (224.2s = 3m 44s)

-| Rebuild cell maps | 224.2s | ~1500% | **2 async tasks (ways + interps), but only uses 2 cores for hash map insertion** |

-| Deduplication | 0.0s | | Included in rebuild timing |

+## Round 1: Parallel continents + parallel cell map rebuild

+1. **Bounded concurrent continents**: Run up to `hardware_concurrency / 8` continents simultaneously (4 on 32-core). Each uses 8 threads for filter_by_bbox.

+2. **Parallel cell map rebuild**: Split sorted pairs at cell boundaries into N chunks, each thread builds its own sub-map, then merge with move semantics.

+### Results (28m 36.6s total — 47% faster)

+| Phase | Baseline | Round 1 | Change |

-| Planet index + qualities | 42.0s | mixed | 3 modes + 7 quality variants |

+| Admin assembly | 141.7s | 142.2s | same |

+| Planet write + qualities | 42.0s | 51.6s | +10s |

-### Continent Processing (sequential, ~34 min)

+### Continent Detail (concurrent, 4 at a time)

+| Continent | Baseline filter | Baseline write | Round 1 total |

-### 1. filter_by_bbox: 1621.8s (30% of total)

-- Scans ALL planet cell maps for each continent (even Antarctica scans all 300M entries)

-- Already has 4 parallel cell scans + 4 parallel remaps (8 threads)

-- But 8 threads on a 32-core machine wastes 24 cores

-- **Fix**: Run 2-3 continents concurrently to use more cores

-### 2. Rebuild cell maps: 224.2s (7% of total)

-- Two async tasks (ways + interps) inserting into hash maps

-- Hash map insertion is inherently sequential per map

-- Only uses ~2 cores effectively despite showing 1500% CPU

-- **Fix**: Partition sorted pairs into N shards, build N smaller hash maps in parallel, merge

-### 3. Continent write phase: 465.7s (8.6% of total)

-- Each continent writes 3 modes + 7 quality variants

-- Quality variant generation re-simplifies vertices for each scale

-- Already parallelized internally (write_index uses threads)

-- **Fix**: Limited room — already using threads within each write

-### 4. S2 covering drain: 134.4s (2.5% of total)

-- Thread pool processes uncapped polygons (365M vertices)

-- With simplified polygons this was ~70s

-### 5. Pass 1: 46.6s (single-threaded PBF scan)

-## Round 1: Parallel continent processing

-**Goal**: Run multiple continents concurrently (2-3 at a time) to better utilize cores during filter phase

-**Expected savings**: ~50% of filter time (1621s → ~810s)

+1. **Rebuild cell maps**: 169.8s — still the largest single phase. The hash map merge step is sequential.

+2. **Pass 1**: 91.6s — inflated by disk cache; normally ~45s. Single-threaded, unavoidable.

+3. **S2 covering drain**: 135.9s — fully parallel, hard to optimize.

+4. **S2 street ways**: 136.0s — fully parallel, hard to optimize.

+5. **Continent filter**: overlapped but each still ~150-400s. Could the filter itself be faster?

# Build Optimization Log

## Baseline Timings (54m 1.7s total)
Build: `--multi-output --continents --multi-quality` (planet PBF → S3 layout)

### PBF Processing Phase (13m 42s)
| Phase | Time | CPU% | Notes |
|---|---|---|---|
| Pass 1: relation scanning | 46.6s | ~100% | Single-threaded PBF read |
| Pass 2: node processing | 184.4s | ~2000% | 28 threads, good utilization |
| Pass 2b: way processing | 132.3s | ~2000% | 28 threads, good utilization |
| Admin: ring assembly | 2.7s | ~2800% | Fully parallel |
| Admin: parallel simplify | 1.2s | ~2800% | Fully parallel (uncapped = minimal work) |
| Admin: append + submit S2 | 2.9s | ~100% | Sequential (append to vectors) |
| Admin: S2 covering drain | 134.4s | ~2800% | Thread pool processing |
| Admin assembly total | 0.5s | | |
| S2: street ways | 136.9s | ~2800% | Fully parallel |
| S2: interp ways | 0.0s | ~2800% | Tiny dataset |
| S2: sort + group | 8.8s | ~2800% | Parallel tree merge |
| S2 cell computation total | 251.5s | | |

### Dedup + Rebuild Phase (224.2s = 3m 44s)
| Phase | Time | CPU% | Notes |
|---|---|---|---|
| Rebuild cell maps | 224.2s | ~1500% | **2 async tasks (ways + interps), but only uses 2 cores for hash map insertion** |
| Deduplication | 0.0s | | Included in rebuild timing |

### Write Phase (2147.9s = 35m 48s)
| Phase | Time | CPU% | Notes |
|---|---|---|---|
| Planet index + qualities | 42.0s | mixed | 3 modes + 7 quality variants |

### Continent Processing (sequential, ~34 min)
| Continent | Filter | Write | Total |
|---|---|---|---|
| Africa | 162.1s | 16.1s | 178.2s |
| Asia | 234.4s | 93.1s | 327.5s |
| Europe | 277.4s | 121.2s | 398.6s |
| North America | 301.0s | 172.8s | 473.8s |
| South America | 177.3s | 29.6s | 206.9s |
| Oceania | 164.9s | 22.1s | 187.0s |
| Central America | 159.5s | 10.8s | 170.3s |
| Antarctica | 145.2s | 0.0s | 145.2s |
| **Total** | **1621.8s** | **465.7s** | **2087.5s** |

## Bottleneck Analysis

### 1. filter_by_bbox: 1621.8s (30% of total)
- Scans ALL planet cell maps for each continent (even Antarctica scans all 300M entries)
- Already has 4 parallel cell scans + 4 parallel remaps (8 threads)
- But 8 threads on a 32-core machine wastes 24 cores
- **Fix**: Run 2-3 continents concurrently to use more cores

### 2. Rebuild cell maps: 224.2s (7% of total)
- Two async tasks (ways + interps) inserting into hash maps
- Hash map insertion is inherently sequential per map
- Only uses ~2 cores effectively despite showing 1500% CPU
- **Fix**: Partition sorted pairs into N shards, build N smaller hash maps in parallel, merge

### 3. Continent write phase: 465.7s (8.6% of total)
- Each continent writes 3 modes + 7 quality variants
- Quality variant generation re-simplifies vertices for each scale
- Already parallelized internally (write_index uses threads)
- **Fix**: Limited room — already using threads within each write

### 4. S2 covering drain: 134.4s (2.5% of total)
- Thread pool processes uncapped polygons (365M vertices)
- With simplified polygons this was ~70s
- Already fully parallel

### 5. Pass 1: 46.6s (single-threaded PBF scan)
- Unavoidable — single PBF reader

---

## Round 1: Parallel continent processing
**Goal**: Run multiple continents concurrently (2-3 at a time) to better utilize cores during filter phase
**Expected savings**: ~50% of filter time (1621s → ~810s)
**Status**: In progress


Now let me look at the next optimizations. The biggest remaining single-threaded bottleneck is the rebuild cell maps merge phase. Let me also look at the dedup phase and continent filter internals more carefully:


  1206→        log_phase("S2 cell computation", _pt);
  1207→        std::cerr << "Deduplicating + sorting for write..." << std::endl;
  1208→        {
  1209→            // Convert addr hash map to sorted pairs.
  1210→            // Sort cell IDs only (30M unique), then build pairs in sorted order.
  1211→            // Much faster than sorting all 160M pairs.
  1212→            auto f2 = std::async(std::launch::async, [&] {
  1213→                // Extract and sort unique cell IDs
  1214→                std::vector<uint64_t> sorted_cells;
  1215→                sorted_cells.reserve(data.cell_to_addrs.size());
  1216→                size_t total_pairs = 0;
  1217→                for (auto& [cell_id, ids] : data.cell_to_addrs) {
  1218→                    sorted_cells.push_back(cell_id);
  1219→                    total_pairs += ids.size();
  1220→                }
  1221→                std::sort(sorted_cells.begin(), sorted_cells.end());
  1222→
  1223→                // Build sorted pairs by iterating sorted cell IDs
  1224→                std::vector<CellItemPair> pairs;
  1225→                pairs.reserve(total_pairs);
  1226→                for (uint64_t cell_id : sorted_cells) {
  1227→                    auto& ids = data.cell_to_addrs[cell_id];
  1228→                    std::sort(ids.begin(), ids.end()); // sort within cell
  1229→                    ids.erase(std::unique(ids.begin(), ids.end()), ids.end());
  1230→                    for (auto id : ids) pairs.push_back({cell_id, id});
  1231→                }
  1232→                data.sorted_addr_cells = std::move(pairs);
  1233→            });
  1234→            auto f4 = std::async(std::launch::async, [&]{ deduplicate(data.cell_to_admin); });
  1235→            f2.get(); f4.get();
  1236→        }
  1237→
  1238→        // Save cache if requested
  1239→        if (!save_cache_path.empty()) {
  1240→            serialize_cache(data, save_cache_path);
  1241→        }
  1242→    }
  1243→
  1244→    // --- Rebuild hash maps from sorted pairs if needed for continent filtering ---
  1245→    if (generate_continents || !save_cache_path.empty()) {

        log_phase("S2 cell computation", _pt);
        std::cerr << "Deduplicating + sorting for write..." << std::endl;
        {
            // Convert addr hash map to sorted pairs.
            // Sort cell IDs only (30M unique), then build pairs in sorted order.
            // Much faster than sorting all 160M pairs.
            auto f2 = std::async(std::launch::async, [&] {
                // Extract and sort unique cell IDs
                std::vector<uint64_t> sorted_cells;
                sorted_cells.reserve(data.cell_to_addrs.size());
                size_t total_pairs = 0;
                for (auto& [cell_id, ids] : data.cell_to_addrs) {
                    sorted_cells.push_back(cell_id);
                    total_pairs += ids.size();
                }
                std::sort(sorted_cells.begin(), sorted_cells.end());

                // Build sorted pairs by iterating sorted cell IDs
                std::vector<CellItemPair> pairs;
                pairs.reserve(total_pairs);
                for (uint64_t cell_id : sorted_cells) {
                    auto& ids = data.cell_to_addrs[cell_id];
                    std::sort(ids.begin(), ids.end()); // sort within cell
                    ids.erase(std::unique(ids.begin(), ids.end()), ids.end());
                    for (auto id : ids) pairs.push_back({cell_id, id});
                }
                data.sorted_addr_cells = std::move(pairs);
            });
            auto f4 = std::async(std::launch::async, [&]{ deduplicate(data.cell_to_admin); });
            f2.get(); f4.get();
        }

        // Save cache if requested
        if (!save_cache_path.empty()) {
            serialize_cache(data, save_cache_path);
        }
    }

    // --- Rebuild hash maps from sorted pairs if needed for continent filtering ---
    if (generate_continents || !save_cache_path.empty()) {


The dedup phase shows as 0.0s but it's actually included in the S2 computation timing. Let me focus on the biggest remaining targets. Looking at the continent filter — each continent scans ALL cell maps even though most cells are outside the bbox. Let me check if we can skip cells early:


    34→    ParsedData out;
    35→
    36→    // Filter cell maps in parallel — each builds an independent ID set
    37→    auto filter_cells = [&](const std::unordered_map<uint64_t, std::vector<uint32_t>>& cell_map,
    38→                            bool mask_interior = false) {
    39→        std::unordered_set<uint32_t> ids;
    40→        for (const auto& [cell_id, cell_ids] : cell_map) {
    41→            if (cell_in_bbox(cell_id, bbox)) {
    42→                for (uint32_t id : cell_ids)
    43→                    ids.insert(mask_interior ? (id & ID_MASK) : id);
    44→            }
    45→        }
    46→        return ids;
    47→    };
    48→
    49→    auto f_ways = std::async(std::launch::async, [&]{ return filter_cells(full.cell_to_ways); });
    50→    auto f_addrs = std::async(std::launch::async, [&]{ return filter_cells(full.cell_to_addrs); });
    51→    auto f_interps = std::async(std::launch::async, [&]{ return filter_cells(full.cell_to_interps); });
    52→    auto f_admin = std::async(std::launch::async, [&]{ return filter_cells(full.cell_to_admin, true); });
    53→
    54→    auto used_way_ids = f_ways.get();
    55→    auto used_addr_ids = f_addrs.get();
    56→    auto used_interp_ids = f_interps.get();
    57→    auto used_admin_ids = f_admin.get();
    58→
    59→    // Remap ways
    60→    std::unordered_map<uint32_t, uint32_t> way_remap;
    61→    { std::vector<uint32_t> sorted_ids(used_way_ids.begin(), used_way_ids.end());
    62→      std::sort(sorted_ids.begin(), sorted_ids.end());
    63→      for (uint32_t old_id : sorted_ids) {

    ParsedData out;

    // Filter cell maps in parallel — each builds an independent ID set
    auto filter_cells = [&](const std::unordered_map<uint64_t, std::vector<uint32_t>>& cell_map,
                            bool mask_interior = false) {
        std::unordered_set<uint32_t> ids;
        for (const auto& [cell_id, cell_ids] : cell_map) {
            if (cell_in_bbox(cell_id, bbox)) {
                for (uint32_t id : cell_ids)
                    ids.insert(mask_interior ? (id & ID_MASK) : id);
            }
        }
        return ids;
    };

    auto f_ways = std::async(std::launch::async, [&]{ return filter_cells(full.cell_to_ways); });
    auto f_addrs = std::async(std::launch::async, [&]{ return filter_cells(full.cell_to_addrs); });
    auto f_interps = std::async(std::launch::async, [&]{ return filter_cells(full.cell_to_interps); });
    auto f_admin = std::async(std::launch::async, [&]{ return filter_cells(full.cell_to_admin, true); });

    auto used_way_ids = f_ways.get();
    auto used_addr_ids = f_addrs.get();
    auto used_interp_ids = f_interps.get();
    auto used_admin_ids = f_admin.get();

    // Remap ways
    std::unordered_map<uint32_t, uint32_t> way_remap;
    { std::vector<uint32_t> sorted_ids(used_way_ids.begin(), used_way_ids.end());
      std::sort(sorted_ids.begin(), sorted_ids.end());
      for (uint32_t old_id : sorted_ids) {


The filter scans the hash maps which iterate randomly. Each `cell_in_bbox` call does an S2 cell center lookup — that's a computation per cell. With 300M+ cells, this is slow even when parallelized across 4 maps.

However, we could use the **sorted pairs** instead — they're sorted by cell_id, so we can binary search for the bbox range. But cell_id is S2 cell ID, not a simple coordinate — S2 cells don't sort geographically in a way that maps to bbox ranges.

A better approach: since the filter phase is dominated by hash map iteration overhead, and we're running 4 continents at a time, each continent's 4 filters run across ~32 threads total. The actual improvement would come from reducing the per-cell work or skipping cells.

Let me instead focus on another bottleneck — the `addr` hash map to sorted pairs conversion (currently in the dedup phase). And also look at whether we can avoid the rebuild entirely by making `filter_by_bbox` work directly with sorted pairs:

Actually, the biggest win might be simpler — the `filter_by_bbox` iterates hash maps but also does the remap phase sequentially. The remap (lines 59-112) copies ways, addresses, interps, and admins one by one. These 4 remap phases are independent and could run in parallel:


    59→    // Remap ways
    60→    std::unordered_map<uint32_t, uint32_t> way_remap;
    61→    { std::vector<uint32_t> sorted_ids(used_way_ids.begin(), used_way_ids.end());
    62→      std::sort(sorted_ids.begin(), sorted_ids.end());
    63→      for (uint32_t old_id : sorted_ids) {
    64→          uint32_t new_id = static_cast<uint32_t>(out.ways.size());
    65→          way_remap[old_id] = new_id;
    66→          const auto& w = full.ways[old_id];
    67→          WayHeader nw = w;
    68→          nw.node_offset = static_cast<uint32_t>(out.street_nodes.size());
    69→          out.ways.push_back(nw);
    70→          for (uint8_t n = 0; n < w.node_count; n++)
    71→              out.street_nodes.push_back(full.street_nodes[w.node_offset + n]);
    72→      } }
    73→
    74→    // Remap addrs
    75→    std::unordered_map<uint32_t, uint32_t> addr_remap;
    76→    { std::vector<uint32_t> sorted_ids(used_addr_ids.begin(), used_addr_ids.end());
    77→      std::sort(sorted_ids.begin(), sorted_ids.end());
    78→      for (uint32_t old_id : sorted_ids) {
    79→          uint32_t new_id = static_cast<uint32_t>(out.addr_points.size());
    80→          addr_remap[old_id] = new_id;
    81→          out.addr_points.push_back(full.addr_points[old_id]);
    82→      } }
    83→
    84→    // Remap interps
    85→    std::unordered_map<uint32_t, uint32_t> interp_remap;
    86→    { std::vector<uint32_t> sorted_ids(used_interp_ids.begin(), used_interp_ids.end());
    87→      std::sort(sorted_ids.begin(), sorted_ids.end());
    88→      for (uint32_t old_id : sorted_ids) {
    89→          uint32_t new_id = static_cast<uint32_t>(out.interp_ways.size());
    90→          interp_remap[old_id] = new_id;
    91→          const auto& iw = full.interp_ways[old_id];
    92→          InterpWay niw = iw;
    93→          niw.node_offset = static_cast<uint32_t>(out.interp_nodes.size());
    94→          out.interp_ways.push_back(niw);
    95→          for (uint8_t n = 0; n < iw.node_count; n++)
    96→              out.interp_nodes.push_back(full.interp_nodes[iw.node_offset + n]);
    97→      } }
    98→
    99→    // Remap admins
   100→    std::unordered_map<uint32_t, uint32_t> admin_remap;
   101→    { std::vector<uint32_t> sorted_ids(used_admin_ids.begin(), used_admin_ids.end());
   102→      std::sort(sorted_ids.begin(), sorted_ids.end());
   103→      for (uint32_t old_id : sorted_ids) {
   104→          uint32_t new_id = static_cast<uint32_t>(out.admin_polygons.size());
   105→          admin_remap[old_id] = new_id;
   106→          const auto& ap = full.admin_polygons[old_id];
   107→          AdminPolygon nap = ap;
   108→          nap.vertex_offset = static_cast<uint32_t>(out.admin_vertices.size());
   109→          out.admin_polygons.push_back(nap);
   110→          for (uint32_t v = 0; v < ap.vertex_count; v++)
   111→              out.admin_vertices.push_back(full.admin_vertices[ap.vertex_offset + v]);
   112→      } }
   113→

    // Remap ways
    std::unordered_map<uint32_t, uint32_t> way_remap;
    { std::vector<uint32_t> sorted_ids(used_way_ids.begin(), used_way_ids.end());
      std::sort(sorted_ids.begin(), sorted_ids.end());
      for (uint32_t old_id : sorted_ids) {
          uint32_t new_id = static_cast<uint32_t>(out.ways.size());
          way_remap[old_id] = new_id;
          const auto& w = full.ways[old_id];
          WayHeader nw = w;
          nw.node_offset = static_cast<uint32_t>(out.street_nodes.size());
          out.ways.push_back(nw);
          for (uint8_t n = 0; n < w.node_count; n++)
              out.street_nodes.push_back(full.street_nodes[w.node_offset + n]);
      } }

    // Remap addrs
    std::unordered_map<uint32_t, uint32_t> addr_remap;
    { std::vector<uint32_t> sorted_ids(used_addr_ids.begin(), used_addr_ids.end());
      std::sort(sorted_ids.begin(), sorted_ids.end());
      for (uint32_t old_id : sorted_ids) {
          uint32_t new_id = static_cast<uint32_t>(out.addr_points.size());
          addr_remap[old_id] = new_id;
          out.addr_points.push_back(full.addr_points[old_id]);
      } }

    // Remap interps
    std::unordered_map<uint32_t, uint32_t> interp_remap;
    { std::vector<uint32_t> sorted_ids(used_interp_ids.begin(), used_interp_ids.end());
      std::sort(sorted_ids.begin(), sorted_ids.end());
      for (uint32_t old_id : sorted_ids) {
          uint32_t new_id = static_cast<uint32_t>(out.interp_ways.size());
          interp_remap[old_id] = new_id;
          const auto& iw = full.interp_ways[old_id];
          InterpWay niw = iw;
          niw.node_offset = static_cast<uint32_t>(out.interp_nodes.size());
          out.interp_ways.push_back(niw);
          for (uint8_t n = 0; n < iw.node_count; n++)
              out.interp_nodes.push_back(full.interp_nodes[iw.node_offset + n]);
      } }

    // Remap admins
    std::unordered_map<uint32_t, uint32_t> admin_remap;
    { std::vector<uint32_t> sorted_ids(used_admin_ids.begin(), used_admin_ids.end());
      std::sort(sorted_ids.begin(), sorted_ids.end());
      for (uint32_t old_id : sorted_ids) {
          uint32_t new_id = static_cast<uint32_t>(out.admin_polygons.size());
          admin_remap[old_id] = new_id;
          const auto& ap = full.admin_polygons[old_id];
          AdminPolygon nap = ap;
          nap.vertex_offset = static_cast<uint32_t>(out.admin_vertices.size());
          out.admin_polygons.push_back(nap);
          for (uint32_t v = 0; v < ap.vertex_count; v++)
              out.admin_vertices.push_back(full.admin_vertices[ap.vertex_offset + v]);
      } }


These 4 remap phases each build their own vector and remap table. They're independent data structures but they all write to different fields of `out`. I can parallelize them by having each thread build its own vectors and remap, then assign to `out`:


Edit

    // Remap ways
    std::unordered_map<uint32_t, uint32_t> way_remap;
    { std::vector<uint32_t> sorted_ids(used_way_ids.begin(), used_way_ids.end());
      std::sort(sorted_ids.begin(), sorted_ids.end());
      for (uint32_t old_id : sorted_ids) {
          uint32_t new_id = static_cast<uint32_t>(out.ways.size());
          way_remap[old_id] = new_id;
          const auto& w = full.ways[old_id];
          WayHeader nw = w;
          nw.node_offset = static_cast<uint32_t>(out.street_nodes.size());
          out.ways.push_back(nw);
          for (uint8_t n = 0; n < w.node_count; n++)
              out.street_nodes.push_back(full.street_nodes[w.node_offset + n]);
      } }

    // Remap addrs
    std::unordered_map<uint32_t, uint32_t> addr_remap;
    { std::vector<uint32_t> sorted_ids(used_addr_ids.begin(), used_addr_ids.end());
      std::sort(sorted_ids.begin(), sorted_ids.end());
      for (uint32_t old_id : sorted_ids) {
          uint32_t new_id = static_cast<uint32_t>(out.addr_points.size());
          addr_remap[old_id] = new_id;
          out.addr_points.push_back(full.addr_points[old_id]);
      } }

    // Remap interps
    std::unordered_map<uint32_t, uint32_t> interp_remap;
    { std::vector<uint32_t> sorted_ids(used_interp_ids.begin(), used_interp_ids.end());
      std::sort(sorted_ids.begin(), sorted_ids.end());
      for (uint32_t old_id : sorted_ids) {
          uint32_t new_id = static_cast<uint32_t>(out.interp_ways.size());
          interp_remap[old_id] = new_id;
          const auto& iw = full.interp_ways[old_id];
          InterpWay niw = iw;
          niw.node_offset = static_cast<uint32_t>(out.interp_nodes.size());
          out.interp_ways.push_back(niw);
          for (uint8_t n = 0; n < iw.node_count; n++)
              out.interp_nodes.push_back(full.interp_nodes[iw.node_offset + n]);
      } }

    // Remap admins
    std::unordered_map<uint32_t, uint32_t> admin_remap;
    { std::vector<uint32_t> sorted_ids(used_admin_ids.begin(), used_admin_ids.end());
      std::sort(sorted_ids.begin(), sorted_ids.end());
      for (uint32_t old_id : sorted_ids) {
          uint32_t new_id = static_cast<uint32_t>(out.admin_polygons.size());
          admin_remap[old_id] = new_id;
          const auto& ap = full.admin_polygons[old_id];
          AdminPolygon nap = ap;
          nap.vertex_offset = static_cast<uint32_t>(out.admin_vertices.size());
          out.admin_polygons.push_back(nap);
          for (uint32_t v = 0; v < ap.vertex_count; v++)
              out.admin_vertices.push_back(full.admin_vertices[ap.vertex_offset + v]);
      } }

    // Remap all 4 data types in parallel — each builds its own vectors and remap table
    std::unordered_map<uint32_t, uint32_t> way_remap, addr_remap, interp_remap, admin_remap;

    // Ways
    auto f_remap_ways = std::async(std::launch::async, [&]() {
        std::vector<uint32_t> sorted_ids(used_way_ids.begin(), used_way_ids.end());
        std::sort(sorted_ids.begin(), sorted_ids.end());
        std::vector<WayHeader> ways;
        std::vector<NodeCoord> nodes;
        ways.reserve(sorted_ids.size());
        nodes.reserve(sorted_ids.size() * 5);
        std::unordered_map<uint32_t, uint32_t> remap;
        remap.reserve(sorted_ids.size());
        for (uint32_t old_id : sorted_ids) {
            remap[old_id] = static_cast<uint32_t>(ways.size());
            const auto& w = full.ways[old_id];
            WayHeader nw = w;
            nw.node_offset = static_cast<uint32_t>(nodes.size());
            ways.push_back(nw);
            for (uint8_t n = 0; n < w.node_count; n++)
                nodes.push_back(full.street_nodes[w.node_offset + n]);
        }
        return std::make_tuple(std::move(remap), std::move(ways), std::move(nodes));
    });

    // Addrs
    auto f_remap_addrs = std::async(std::launch::async, [&]() {
        std::vector<uint32_t> sorted_ids(used_addr_ids.begin(), used_addr_ids.end());
        std::sort(sorted_ids.begin(), sorted_ids.end());
        std::vector<AddrPoint> addrs;
        addrs.reserve(sorted_ids.size());
        std::unordered_map<uint32_t, uint32_t> remap;
        remap.reserve(sorted_ids.size());
        for (uint32_t old_id : sorted_ids) {
            remap[old_id] = static_cast<uint32_t>(addrs.size());
            addrs.push_back(full.addr_points[old_id]);
        }
        return std::make_tuple(std::move(remap), std::move(addrs));
    });

    // Interps
    auto f_remap_interps = std::async(std::launch::async, [&]() {
        std::vector<uint32_t> sorted_ids(used_interp_ids.begin(), used_interp_ids.end());
        std::sort(sorted_ids.begin(), sorted_ids.end());
        std::vector<InterpWay> iways;
        std::vector<NodeCoord> inodes;
        iways.reserve(sorted_ids.size());
        std::unordered_map<uint32_t, uint32_t> remap;
        remap.reserve(sorted_ids.size());
        for (uint32_t old_id : sorted_ids) {
            remap[old_id] = static_cast<uint32_t>(iways.size());
            const auto& iw = full.interp_ways[old_id];
            InterpWay niw = iw;
            niw.node_offset = static_cast<uint32_t>(inodes.size());
            iways.push_back(niw);
            for (uint8_t n = 0; n < iw.node_count; n++)
                inodes.push_back(full.interp_nodes[iw.node_offset + n]);
        }
        return std::make_tuple(std::move(remap), std::move(iways), std::move(inodes));
    });

    // Admins
    auto f_remap_admins = std::async(std::launch::async, [&]() {
        std::vector<uint32_t> sorted_ids(used_admin_ids.begin(), used_admin_ids.end());
        std::sort(sorted_ids.begin(), sorted_ids.end());
        std::vector<AdminPolygon> polys;
        std::vector<NodeCoord> verts;
        polys.reserve(sorted_ids.size());
        std::unordered_map<uint32_t, uint32_t> remap;
        remap.reserve(sorted_ids.size());
        for (uint32_t old_id : sorted_ids) {
            remap[old_id] = static_cast<uint32_t>(polys.size());
            const auto& ap = full.admin_polygons[old_id];
            AdminPolygon nap = ap;
            nap.vertex_offset = static_cast<uint32_t>(verts.size());
            polys.push_back(nap);
            for (uint32_t v = 0; v < ap.vertex_count; v++)
                verts.push_back(full.admin_vertices[ap.vertex_offset + v]);
        }
        return std::make_tuple(std::move(remap), std::move(polys), std::move(verts));
    });

    // Collect results
    {
        auto [wr, ways, nodes] = f_remap_ways.get();
        way_remap = std::move(wr);
        out.ways = std::move(ways);
        out.street_nodes = std::move(nodes);
    }
    {
        auto [ar, addrs] = f_remap_addrs.get();
        addr_remap = std::move(ar);
        out.addr_points = std::move(addrs);
    }
    {
        auto [ir, iways, inodes] = f_remap_interps.get();
        interp_remap = std::move(ir);
        out.interp_ways = std::move(iways);
        out.interp_nodes = std::move(inodes);
    }
    {
        auto [ar, polys, verts] = f_remap_admins.get();
        admin_remap = std::move(ar);
        out.admin_polygons = std::move(polys);
        out.admin_vertices = std::move(verts);
    }


The file /root/traccar-geocoder/builder/src/continent_filter.cpp has been updated successfully.

    // Remap all 4 data types in parallel — each builds its own vectors and remap table
    std::unordered_map<uint32_t, uint32_t> way_remap, addr_remap, interp_remap, admin_remap;

    // Ways
    auto f_remap_ways = std::async(std::launch::async, [&]() {
        std::vector<uint32_t> sorted_ids(used_way_ids.begin(), used_way_ids.end());
        std::sort(sorted_ids.begin(), sorted_ids.end());
        std::vector<WayHeader> ways;
        std::vector<NodeCoord> nodes;
        ways.reserve(sorted_ids.size());
        nodes.reserve(sorted_ids.size() * 5);
        std::unordered_map<uint32_t, uint32_t> remap;
        remap.reserve(sorted_ids.size());
        for (uint32_t old_id : sorted_ids) {
            remap[old_id] = static_cast<uint32_t>(ways.size());
            const auto& w = full.ways[old_id];
            WayHeader nw = w;
            nw.node_offset = static_cast<uint32_t>(nodes.size());
            ways.push_back(nw);
            for (uint8_t n = 0; n < w.node_count; n++)
                nodes.push_back(full.street_nodes[w.node_offset + n]);
        }
        return std::make_tuple(std::move(remap), std::move(ways), std::move(nodes));
    });

    // Addrs
    auto f_remap_addrs = std::async(std::launch::async, [&]() {
        std::vector<uint32_t> sorted_ids(used_addr_ids.begin(), used_addr_ids.end());
        std::sort(sorted_ids.begin(), sorted_ids.end());
        std::vector<AddrPoint> addrs;
        addrs.reserve(sorted_ids.size());
        std::unordered_map<uint32_t, uint32_t> remap;
        remap.reserve(sorted_ids.size());
        for (uint32_t old_id : sorted_ids) {
            remap[old_id] = static_cast<uint32_t>(addrs.size());
            addrs.push_back(full.addr_points[old_id]);
        }
        return std::make_tuple(std::move(remap), std::move(addrs));
    });

    // Interps
    auto f_remap_interps = std::async(std::launch::async, [&]() {
        std::vector<uint32_t> sorted_ids(used_interp_ids.begin(), used_interp_ids.end());
        std::sort(sorted_ids.begin(), sorted_ids.end());
        std::vector<InterpWay> iways;
        std::vector<NodeCoord> inodes;
        iways.reserve(sorted_ids.size());
        std::unordered_map<uint32_t, uint32_t> remap;
        remap.reserve(sorted_ids.size());
        for (uint32_t old_id : sorted_ids) {
            remap[old_id] = static_cast<uint32_t>(iways.size());
            const auto& iw = full.interp_ways[old_id];
            InterpWay niw = iw;
            niw.node_offset = static_cast<uint32_t>(inodes.size());
            iways.push_back(niw);
            for (uint8_t n = 0; n < iw.node_count; n++)
                inodes.push_back(full.interp_nodes[iw.node_offset + n]);
        }
        return std::make_tuple(std::move(remap), std::move(iways), std::move(inodes));
    });

    // Admins
    auto f_remap_admins = std::async(std::launch::async, [&]() {
        std::vector<uint32_t> sorted_ids(used_admin_ids.begin(), used_admin_ids.end());
        std::sort(sorted_ids.begin(), sorted_ids.end());
        std::vector<AdminPolygon> polys;
        std::vector<NodeCoord> verts;
        polys.reserve(sorted_ids.size());
        std::unordered_map<uint32_t, uint32_t> remap;
        remap.reserve(sorted_ids.size());
        for (uint32_t old_id : sorted_ids) {
            remap[old_id] = static_cast<uint32_t>(polys.size());
            const auto& ap = full.admin_polygons[old_id];
            AdminPolygon nap = ap;
            nap.vertex_offset = static_cast<uint32_t>(verts.size());
            polys.push_back(nap);
            for (uint32_t v = 0; v < ap.vertex_count; v++)
                verts.push_back(full.admin_vertices[ap.vertex_offset + v]);
        }
        return std::make_tuple(std::move(remap), std::move(polys), std::move(verts));
    });

    // Collect results
    {
        auto [wr, ways, nodes] = f_remap_ways.get();
        way_remap = std::move(wr);
        out.ways = std::move(ways);
        out.street_nodes = std::move(nodes);
    }
    {
        auto [ar, addrs] = f_remap_addrs.get();
        addr_remap = std::move(ar);
        out.addr_points = std::move(addrs);
    }
    {
        auto [ir, iways, inodes] = f_remap_interps.get();
        interp_remap = std::move(ir);
        out.interp_ways = std::move(iways);
        out.interp_nodes = std::move(inodes);
    }
    {
        auto [ar, polys, verts] = f_remap_admins.get();
        admin_remap = std::move(ar);
        out.admin_polygons = std::move(polys);
        out.admin_vertices = std::move(verts);
    }

#include "continent_filter.h"

#include <algorithm>
#include <cstring>
#include <future>
#include <unordered_set>

#include <s2/s2cell_id.h>
#include <s2/s2latlng.h>

const ContinentBBox kContinents[] = {
    {"africa",            -35.0,  37.5,  -25.0,  55.0},
    {"asia",              -12.0,  82.0,   25.0, 180.0},
    {"europe",             35.0,  72.0,  -25.0,  45.0},
    {"north-america",       7.0,  84.0, -170.0, -50.0},
    {"south-america",     -56.0,  13.0,  -82.0, -34.0},
    {"oceania",           -50.0,   0.0,  110.0, 180.0},
    {"central-america",     7.0,  23.5, -120.0, -57.0},
    {"antarctica",        -90.0, -60.0, -180.0, 180.0},
};

const size_t kContinentCount = sizeof(kContinents) / sizeof(kContinents[0]);

static bool cell_in_bbox(uint64_t cell_id, const ContinentBBox& bbox) {
    S2CellId cell(cell_id);
    S2LatLng center = cell.ToLatLng();
    double lat = center.lat().degrees();
    double lng = center.lng().degrees();
    return lat >= bbox.min_lat && lat <= bbox.max_lat &&
           lng >= bbox.min_lng && lng <= bbox.max_lng;
}

ParsedData filter_by_bbox(const ParsedData& full, const ContinentBBox& bbox) {
    ParsedData out;

    // Filter cell maps in parallel — each builds an independent ID set
    auto filter_cells = [&](const std::unordered_map<uint64_t, std::vector<uint32_t>>& cell_map,
                            bool mask_interior = false) {
        std::unordered_set<uint32_t> ids;
        for (const auto& [cell_id, cell_ids] : cell_map) {
            if (cell_in_bbox(cell_id, bbox)) {
                for (uint32_t id : cell_ids)
                    ids.insert(mask_interior ? (id & ID_MASK) : id);
            }
        }
        return ids;
    };

    auto f_ways = std::async(std::launch::async, [&]{ return filter_cells(full.cell_to_ways); });
    auto f_addrs = std::async(std::launch::async, [&]{ return filter_cells(full.cell_to_addrs); });
    auto f_interps = std::async(std::launch::async, [&]{ return filter_cells(full.cell_to_interps); });
    auto f_admin = std::async(std::launch::async, [&]{ return filter_cells(full.cell_to_admin, true); });

    auto used_way_ids = f_ways.get();
    auto used_addr_ids = f_addrs.get();
    auto used_interp_ids = f_interps.get();
    auto used_admin_ids = f_admin.get();

    // Remap ways
    std::unordered_map<uint32_t, uint32_t> way_remap;
    { std::vector<uint32_t> sorted_ids(used_way_ids.begin(), used_way_ids.end());
      std::sort(sorted_ids.begin(), sorted_ids.end());
      for (uint32_t old_id : sorted_ids) {
          uint32_t new_id = static_cast<uint32_t>(out.ways.size());
          way_remap[old_id] = new_id;
          const auto& w = full.ways[old_id];
          WayHeader nw = w;
          nw.node_offset = static_cast<uint32_t>(out.street_nodes.size());
          out.ways.push_back(nw);
          for (uint8_t n = 0; n < w.node_count; n++)
              out.street_nodes.push_back(full.street_nodes[w.node_offset + n]);
      } }

    // Remap addrs
    std::unordered_map<uint32_t, uint32_t> addr_remap;
    { std::vector<uint32_t> sorted_ids(used_addr_ids.begin(), used_addr_ids.end());
      std::sort(sorted_ids.begin(), sorted_ids.end());
      for (uint32_t old_id : sorted_ids) {
          uint32_t new_id = static_cast<uint32_t>(out.addr_points.size());
          addr_remap[old_id] = new_id;
          out.addr_points.push_back(full.addr_points[old_id]);
      } }

    // Remap interps
    std::unordered_map<uint32_t, uint32_t> interp_remap;
    { std::vector<uint32_t> sorted_ids(used_interp_ids.begin(), used_interp_ids.end());
      std::sort(sorted_ids.begin(), sorted_ids.end());
      for (uint32_t old_id : sorted_ids) {
          uint32_t new_id = static_cast<uint32_t>(out.interp_ways.size());
          interp_remap[old_id] = new_id;
          const auto& iw = full.interp_ways[old_id];
          InterpWay niw = iw;
          niw.node_offset = static_cast<uint32_t>(out.interp_nodes.size());
          out.interp_ways.push_back(niw);
          for (uint8_t n = 0; n < iw.node_count; n++)
              out.interp_nodes.push_back(full.interp_nodes[iw.node_offset + n]);
      } }

    // Remap admins
    std::unordered_map<uint32_t, uint32_t> admin_remap;
    { std::vector<uint32_t> sorted_ids(used_admin_ids.begin(), used_admin_ids.end());
      std::sort(sorted_ids.begin(), sorted_ids.end());
      for (uint32_t old_id : sorted_ids) {
          uint32_t new_id = static_cast<uint32_t>(out.admin_polygons.size());
          admin_remap[old_id] = new_id;
          const auto& ap = full.admin_polygons[old_id];
          AdminPolygon nap = ap;
          nap.vertex_offset = static_cast<uint32_t>(out.admin_vertices.size());
          out.admin_polygons.push_back(nap);
          for (uint32_t v = 0; v < ap.vertex_count; v++)
              out.admin_vertices.push_back(full.admin_vertices[ap.vertex_offset + v]);
      } }

    // Remap cell maps in parallel (each builds an independent output map)
    auto remap_cells = [&](const std::unordered_map<uint64_t, std::vector<uint32_t>>& src,
                           const std::unordered_map<uint32_t, uint32_t>& remap,
                           std::unordered_map<uint64_t, std::vector<uint32_t>>& dst,
                           bool handle_flags = false) {
        for (const auto& [cell_id, ids] : src) {
            if (!cell_in_bbox(cell_id, bbox)) continue;
            std::vector<uint32_t> new_ids;
            for (uint32_t id : ids) {
                uint32_t raw_id = handle_flags ? (id & ID_MASK) : id;
                uint32_t flags = handle_flags ? (id & INTERIOR_FLAG) : 0;
                auto it = remap.find(raw_id);
                if (it != remap.end()) new_ids.push_back(it->second | flags);
            }
            if (!new_ids.empty()) dst[cell_id] = std::move(new_ids);
        }
    };

    {
        auto f1 = std::async(std::launch::async, [&]{ remap_cells(full.cell_to_ways, way_remap, out.cell_to_ways); });
        auto f2 = std::async(std::launch::async, [&]{ remap_cells(full.cell_to_addrs, addr_remap, out.cell_to_addrs); });
        auto f3 = std::async(std::launch::async, [&]{ remap_cells(full.cell_to_interps, interp_remap, out.cell_to_interps); });
        auto f4 = std::async(std::launch::async, [&]{ remap_cells(full.cell_to_admin, admin_remap, out.cell_to_admin, true); });
        f1.get(); f2.get(); f3.get(); f4.get();
    }

    // Rebuild compact string pool
    std::unordered_set<uint32_t> used_offsets;
    for (const auto& w : out.ways) used_offsets.insert(w.name_id);
    for (const auto& a : out.addr_points) { used_offsets.insert(a.housenumber_id); used_offsets.insert(a.street_id); }
    for (const auto& iw : out.interp_ways) used_offsets.insert(iw.street_id);
    for (const auto& ap : out.admin_polygons) used_offsets.insert(ap.name_id);

    const auto& old_sp = full.string_pool.data();
    std::unordered_map<uint32_t, uint32_t> string_remap;
    auto& new_sp = out.string_pool.mutable_data();
    new_sp.clear();

    std::vector<uint32_t> sorted_offsets(used_offsets.begin(), used_offsets.end());
    std::sort(sorted_offsets.begin(), sorted_offsets.end());
    for (uint32_t old_off : sorted_offsets) {
        uint32_t new_off = static_cast<uint32_t>(new_sp.size());
        string_remap[old_off] = new_off;
        const char* str = old_sp.data() + old_off;
        size_t len = std::strlen(str);
        new_sp.insert(new_sp.end(), str, str + len + 1);
    }

    for (auto& w : out.ways) w.name_id = string_remap[w.name_id];
    for (auto& a : out.addr_points) { a.housenumber_id = string_remap[a.housenumber_id]; a.street_id = string_remap[a.street_id]; }
    for (auto& iw : out.interp_ways) iw.street_id = string_remap[iw.street_id];
    for (auto& ap : out.admin_polygons) ap.name_id = string_remap[ap.name_id];

    return out;
}

-    std::unordered_map<uint32_t, uint32_t> way_remap;

-    { std::vector<uint32_t> sorted_ids(used_way_ids.begin(), used_way_ids.end());

-      std::sort(sorted_ids.begin(), sorted_ids.end());

-      for (uint32_t old_id : sorted_ids) {

-          uint32_t new_id = static_cast<uint32_t>(out.ways.size());

-          const auto& w = full.ways[old_id];

-          nw.node_offset = static_cast<uint32_t>(out.street_nodes.size());

-          for (uint8_t n = 0; n < w.node_count; n++)

-              out.street_nodes.push_back(full.street_nodes[w.node_offset + n]);

+    // Remap all 4 data types in parallel — each builds its own vectors and remap table

+    std::unordered_map<uint32_t, uint32_t> way_remap, addr_remap, interp_remap, admin_remap;

-    std::unordered_map<uint32_t, uint32_t> addr_remap;

-    { std::vector<uint32_t> sorted_ids(used_addr_ids.begin(), used_addr_ids.end());

-          uint32_t new_id = static_cast<uint32_t>(out.addr_points.size());

-          out.addr_points.push_back(full.addr_points[old_id]);

+    auto f_remap_ways = std::async(std::launch::async, [&]() {

+        std::vector<uint32_t> sorted_ids(used_way_ids.begin(), used_way_ids.end());

+        std::sort(sorted_ids.begin(), sorted_ids.end());

+        ways.reserve(sorted_ids.size());

+        nodes.reserve(sorted_ids.size() * 5);

+        std::unordered_map<uint32_t, uint32_t> remap;

+        remap.reserve(sorted_ids.size());

+        for (uint32_t old_id : sorted_ids) {

+            remap[old_id] = static_cast<uint32_t>(ways.size());

+            const auto& w = full.ways[old_id];

+            nw.node_offset = static_cast<uint32_t>(nodes.size());

+            for (uint8_t n = 0; n < w.node_count; n++)

+                nodes.push_back(full.street_nodes[w.node_offset + n]);

+        return std::make_tuple(std::move(remap), std::move(ways), std::move(nodes));

-    std::unordered_map<uint32_t, uint32_t> interp_remap;

-    { std::vector<uint32_t> sorted_ids(used_interp_ids.begin(), used_interp_ids.end());

-      for (uint32_t old_id : sorted_ids) {

-          uint32_t new_id = static_cast<uint32_t>(out.interp_ways.size());

-          interp_remap[old_id] = new_id;

-          const auto& iw = full.interp_ways[old_id];

-          niw.node_offset = static_cast<uint32_t>(out.interp_nodes.size());

-          out.interp_ways.push_back(niw);

-          for (uint8_t n = 0; n < iw.node_count; n++)

-              out.interp_nodes.push_back(full.interp_nodes[iw.node_offset + n]);

+    auto f_remap_addrs = std::async(std::launch::async, [&]() {

+        std::vector<uint32_t> sorted_ids(used_addr_ids.begin(), used_addr_ids.end());

+        addrs.reserve(sorted_ids.size());

+        remap.reserve(sorted_ids.size());

+        for (uint32_t old_id : sorted_ids) {

+            remap[old_id] = static_cast<uint32_t>(addrs.size());

+            addrs.push_back(full.addr_points[old_id]);

+        return std::make_tuple(std::move(remap), std::move(addrs));

-    std::unordered_map<uint32_t, uint32_t> admin_remap;

-    { std::vector<uint32_t> sorted_ids(used_admin_ids.begin(), used_admin_ids.end());

-      for (uint32_t old_id : sorted_ids) {

-          uint32_t new_id = static_cast<uint32_t>(out.admin_polygons.size());

-          admin_remap[old_id] = new_id;

-          const auto& ap = full.admin_polygons[old_id];

-          nap.vertex_offset = static_cast<uint32_t>(out.admin_vertices.size());

-          out.admin_polygons.push_back(nap);

-          for (uint32_t v = 0; v < ap.vertex_count; v++)

-              out.admin_vertices.push_back(full.admin_vertices[ap.vertex_offset + v]);

+    auto f_remap_interps = std::async(std::launch::async, [&]() {

+        std::vector<uint32_t> sorted_ids(used_interp_ids.begin(), used_interp_ids.end());

+        iways.reserve(sorted_ids.size());

+        remap.reserve(sorted_ids.size());

+        for (uint32_t old_id : sorted_ids) {

+            remap[old_id] = static_cast<uint32_t>(iways.size());

+            const auto& iw = full.interp_ways[old_id];

+            niw.node_offset = static_cast<uint32_t>(inodes.size());

+            for (uint8_t n = 0; n < iw.node_count; n++)

+                inodes.push_back(full.interp_nodes[iw.node_offset + n]);

+        return std::make_tuple(std::move(remap), std::move(iways), std::move(inodes));

+    auto f_remap_admins = std::async(std::launch::async, [&]() {

+        std::vector<uint32_t> sorted_ids(used_admin_ids.begin(), used_admin_ids.end());

+        std::vector<AdminPolygon> polys;

+        polys.reserve(sorted_ids.size());

+            remap[old_id] = static_cast<uint32_t>(polys.size());

+            const auto& ap = full.admin_polygons[old_id];

+            nap.vertex_offset = static_cast<uint32_t>(verts.size());

+            for (uint32_t v = 0; v < ap.vertex_count; v++)

+                verts.push_back(full.admin_vertices[ap.vertex_offset + v]);

+        return std::make_tuple(std::move(remap), std::move(polys), std::move(verts));

+        auto [wr, ways, nodes] = f_remap_ways.get();

+        out.street_nodes = std::move(nodes);

+        auto [ar, addrs] = f_remap_addrs.get();

+        out.addr_points = std::move(addrs);

+        auto [ir, iways, inodes] = f_remap_interps.get();

+        out.interp_ways = std::move(iways);

+        out.interp_nodes = std::move(inodes);

+        auto [ar, polys, verts] = f_remap_admins.get();

+        out.admin_polygons = std::move(polys);

+        out.admin_vertices = std::move(verts);


Bash

cd /root/traccar-geocoder/builder/build && make -j$(nproc) 2>&1 | tail -5

Build


[ 12%] Building CXX object CMakeFiles/build-index.dir/src/continent_filter.cpp.o
[ 25%] Linking CXX executable build-index
[100%] Built target build-index


cd /root/traccar-geocoder && git add builder/src/continent_filter.cpp builder/OPTIMIZATION_LOG.md && git commit -m "$(cat <<'EOF'
perf: parallel remap phase in filter_by_bbox

The 4 remap phases (ways, addrs, interps, admins) in filter_by_bbox
now run in parallel. Each builds its own vectors and remap tables
independently, then the results are moved into the output ParsedData.

Previously these were sequential, sorting and copying millions of
items one data type at a time.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
EOF
)" && git push origin main 2>&1

Commit and push


[main 5a7f9d7] perf: parallel remap phase in filter_by_bbox
 2 files changed, 158 insertions(+), 120 deletions(-)
To github.com:zp-forks/traccar-geocoder.git
   d2901da..5a7f9d7  main -> main


Now upload and test:


ssh node3 "rm -rf ~/test-opt2" 2>&1 && scp /root/traccar-geocoder/builder/build/build-index node3:~/build-index-opt3 2>&1 && ssh node3 "mkdir ~/test-opt3 && nohup bash -c 'time ~/build-index-opt3 ~/test-opt3 ~/pbf/planet.osm.pbf --multi-output --continents --multi-quality' > /tmp/opt3_build.log 2>&1 &
sleep 3 && head -3 /tmp/opt3_build.log && ps aux | grep build-index-opt3 | grep -v grep | wc -l" 2>&1

Upload and start Round 2 build


Command running in background with ID: bptbtve2q. Output is being written to: /tmp/claude-0/-root-traccar-geocoder/af1c4baa-0f9a-4420-a8b3-ce20dc28c8b9/tasks/bptbtve2q.output


sleep 5 && ssh node3 "head -5 /tmp/opt3_build.log && echo '---' && ps aux | grep build-index-opt3 | grep -v grep | wc -l" 2>&1

Verify started


Simplification: error-bounded, fixed 0.1m epsilon
Using 28 worker threads.
Allocated dense node index: 111GB virtual address space
Processing /home/michtest/pbf/planet.osm.pbf...
  Pass 1: scanning relations...
---
2


Running. While this builds (~28 min expected), let me look ahead at more optimization opportunities by examining the code for other single-threaded sections. Let me check what happens during `write_quality_variant` — the quality simplification loop:


   382→    log_phase("  Write: parallel data files launched", _wt);
   383→    admin_future.get();
   384→    for (auto& f : write_futures) f.get();
   385→}
   386→
   387→void write_quality_variant(const ParsedData& data, const std::string& source_dir,
   388→                           const std::string& output_dir, double epsilon_scale) {
   389→    ensure_dir(output_dir);
   390→
   391→    // Re-simplify admin polygons at the given epsilon scale
   392→    std::vector<AdminPolygon> new_polys;
   393→    std::vector<NodeCoord> new_verts;
   394→    new_polys.reserve(data.admin_polygons.size());
   395→
   396→    // Parallel simplification
   397→    struct SimplifiedPoly {
   398→        std::vector<std::pair<double,double>> verts;
   399→    };
   400→    std::vector<SimplifiedPoly> simplified(data.admin_polygons.size());
   401→
   402→    {
   403→        std::atomic<size_t> idx{0};
   404→        unsigned nthreads = std::thread::hardware_concurrency();
   405→        if (nthreads == 0) nthreads = 4;
   406→        std::vector<std::thread> workers;
   407→        for (unsigned t = 0; t < nthreads; t++) {
   408→            workers.emplace_back([&]() {
   409→                while (true) {
   410→                    size_t i = idx.fetch_add(1);
   411→                    if (i >= data.admin_polygons.size()) break;
   412→                    const auto& ap = data.admin_polygons[i];
   413→
   414→                    // Extract original vertices
   415→                    std::vector<std::pair<double,double>> pts;
   416→                    pts.reserve(ap.vertex_count);
   417→                    for (uint32_t j = 0; j < ap.vertex_count; j++) {
   418→                        const auto& v = data.admin_vertices[ap.vertex_offset + j];
   419→                        pts.emplace_back(v.lat, v.lng);
   420→                    }
   421→
   422→                    if (epsilon_scale > 0) {
   423→                        double eps_m = admin_epsilon_meters(ap.admin_level) * epsilon_scale;
   424→                        double lat = pts.empty() ? 0.0 : pts[0].first;
   425→                        double eps_deg = meters_to_degrees(eps_m, lat);
   426→                        simplified[i].verts = simplify_polygon_epsilon(pts, eps_deg);
   427→                    } else {
   428→                        simplified[i].verts = std::move(pts);
   429→                    }
   430→                }
   431→            });
   432→        }
   433→        for (auto& w : workers) w.join();
   434→    }
   435→
   436→    // Sequential: build new polygon and vertex arrays
   437→    for (size_t i = 0; i < data.admin_polygons.size(); i++) {
   438→        auto& sv = simplified[i].verts;
   439→        if (sv.size() < 3) continue;
   440→
   441→        AdminPolygon np = data.admin_polygons[i];
   442→        np.vertex_offset = static_cast<uint32_t>(new_verts.size());
   443→        np.vertex_count = static_cast<uint32_t>(sv.size());
   444→        np.area = polygon_area(sv);
   445→
   446→        for (const auto& [lat, lng] : sv) {
   447→            new_verts.push_back({static_cast<float>(lat), static_cast<float>(lng)});
   448→        }
   449→        new_polys.push_back(np);
   450→    }
   451→
   452→    std::cerr << "Quality " << epsilon_scale << "x: " << new_polys.size()
   453→              << " polygons, " << new_verts.size() << " vertices ("
   454→              << new_verts.size() * 8 / 1024 / 1024 << " MiB)" << std::endl;
   455→
   456→    // Write admin files
   457→    {
   458→        std::ofstream f(output_dir + "/admin_polygons.bin", std::ios::binary);
   459→        f.write(reinterpret_cast<const char*>(new_polys.data()), new_polys.size() * sizeof(AdminPolygon));
   460→    }
   461→    {

    log_phase("  Write: parallel data files launched", _wt);
    admin_future.get();
    for (auto& f : write_futures) f.get();
}

void write_quality_variant(const ParsedData& data, const std::string& source_dir,
                           const std::string& output_dir, double epsilon_scale) {
    ensure_dir(output_dir);

    // Re-simplify admin polygons at the given epsilon scale
    std::vector<AdminPolygon> new_polys;
    std::vector<NodeCoord> new_verts;
    new_polys.reserve(data.admin_polygons.size());

    // Parallel simplification
    struct SimplifiedPoly {
        std::vector<std::pair<double,double>> verts;
    };
    std::vector<SimplifiedPoly> simplified(data.admin_polygons.size());

    {
        std::atomic<size_t> idx{0};
        unsigned nthreads = std::thread::hardware_concurrency();
        if (nthreads == 0) nthreads = 4;
        std::vector<std::thread> workers;
        for (unsigned t = 0; t < nthreads; t++) {
            workers.emplace_back([&]() {
                while (true) {
                    size_t i = idx.fetch_add(1);
                    if (i >= data.admin_polygons.size()) break;
                    const auto& ap = data.admin_polygons[i];

                    // Extract original vertices
                    std::vector<std::pair<double,double>> pts;
                    pts.reserve(ap.vertex_count);
                    for (uint32_t j = 0; j < ap.vertex_count; j++) {
                        const auto& v = data.admin_vertices[ap.vertex_offset + j];
                        pts.emplace_back(v.lat, v.lng);
                    }

                    if (epsilon_scale > 0) {
                        double eps_m = admin_epsilon_meters(ap.admin_level) * epsilon_scale;
                        double lat = pts.empty() ? 0.0 : pts[0].first;
                        double eps_deg = meters_to_degrees(eps_m, lat);
                        simplified[i].verts = simplify_polygon_epsilon(pts, eps_deg);
                    } else {
                        simplified[i].verts = std::move(pts);
                    }
                }
            });
        }
        for (auto& w : workers) w.join();
    }

    // Sequential: build new polygon and vertex arrays
    for (size_t i = 0; i < data.admin_polygons.size(); i++) {
        auto& sv = simplified[i].verts;
        if (sv.size() < 3) continue;

        AdminPolygon np = data.admin_polygons[i];
        np.vertex_offset = static_cast<uint32_t>(new_verts.size());
        np.vertex_count = static_cast<uint32_t>(sv.size());
        np.area = polygon_area(sv);

        for (const auto& [lat, lng] : sv) {
            new_verts.push_back({static_cast<float>(lat), static_cast<float>(lng)});
        }
        new_polys.push_back(np);
    }

    std::cerr << "Quality " << epsilon_scale << "x: " << new_polys.size()
              << " polygons, " << new_verts.size() << " vertices ("
              << new_verts.size() * 8 / 1024 / 1024 << " MiB)" << std::endl;

    // Write admin files
    {
        std::ofstream f(output_dir + "/admin_polygons.bin", std::ios::binary);
        f.write(reinterpret_cast<const char*>(new_polys.data()), new_polys.size() * sizeof(AdminPolygon));
    }
    {


The quality variant generation already parallelizes the simplification (line 402-434). The sequential part (lines 436-450) builds the final arrays — this is fast since it's just appending. The simplification is the expensive part and it's already parallel. Good.

Let me check how the `write_region` helper uses `write_index` — specifically, do the 3 mode writes run in parallel?


  1278→                        sub[sorted[i].cell_id].push_back(sorted[i].item_id);
  1279→                    }
  1280→                });
  1281→            }
  1282→            for (auto& t : threads) t.join();
  1283→
  1284→            // Merge sub-maps into main map (no conflicts since splits are at cell boundaries)
  1285→            size_t total_cells = 0;
  1286→            for (auto& sub : sub_maps) total_cells += sub.size();
  1287→            map.reserve(total_cells);
  1288→            for (auto& sub : sub_maps) {
  1289→                for (auto& [k, v] : sub) {
  1290→                    map[k] = std::move(v);
  1291→                }
  1292→            }
  1293→        };
  1294→
  1295→        std::cerr << "Rebuilding cell maps for continent filtering..." << std::endl;
  1296→        auto f1 = std::async(std::launch::async, [&]{ rebuild_map_parallel(data.sorted_way_cells, data.cell_to_ways); });
  1297→        auto f2 = std::async(std::launch::async, [&]{ rebuild_map_parallel(data.sorted_interp_cells, data.cell_to_interps); });

                        sub[sorted[i].cell_id].push_back(sorted[i].item_id);
                    }
                });
            }
            for (auto& t : threads) t.join();

            // Merge sub-maps into main map (no conflicts since splits are at cell boundaries)
            size_t total_cells = 0;
            for (auto& sub : sub_maps) total_cells += sub.size();
            map.reserve(total_cells);
            for (auto& sub : sub_maps) {
                for (auto& [k, v] : sub) {
                    map[k] = std::move(v);
                }
            }
        };

        std::cerr << "Rebuilding cell maps for continent filtering..." << std::endl;
        auto f1 = std::async(std::launch::async, [&]{ rebuild_map_parallel(data.sorted_way_cells, data.cell_to_ways); });
        auto f2 = std::async(std::launch::async, [&]{ rebuild_map_parallel(data.sorted_interp_cells, data.cell_to_interps); });


  1305→
  1306→    auto quality_dir_name = [](double scale) -> std::string {
  1307→        if (scale == 0) return "uncapped";
  1308→        char buf[32]; snprintf(buf, sizeof(buf), "q%.4g", scale);
  1309→        return buf;
  1310→    };
  1311→
  1312→    // Write quality variants into a directory that already has admin base files
  1313→    auto write_qualities = [&](const ParsedData& d, const std::string& admin_dir) {
  1314→        for (double scale : quality_scales) {
  1315→            std::string qname = quality_dir_name(scale);
  1316→            std::string qdir = admin_dir + "/" + qname;
  1317→            write_quality_variant(d, admin_dir, qdir, scale);
  1318→        }
  1319→    };
  1320→
  1321→    // Write one region: modes + quality variants
  1322→    auto write_region = [&](const ParsedData& d, const std::string& base_dir) {
  1323→        ensure_dir(base_dir);
  1324→

    auto quality_dir_name = [](double scale) -> std::string {
        if (scale == 0) return "uncapped";
        char buf[32]; snprintf(buf, sizeof(buf), "q%.4g", scale);
        return buf;
    };

    // Write quality variants into a directory that already has admin base files
    auto write_qualities = [&](const ParsedData& d, const std::string& admin_dir) {
        for (double scale : quality_scales) {
            std::string qname = quality_dir_name(scale);
            std::string qdir = admin_dir + "/" + qname;
            write_quality_variant(d, admin_dir, qdir, scale);
        }
    };

    // Write one region: modes + quality variants
    auto write_region = [&](const ParsedData& d, const std::string& base_dir) {
        ensure_dir(base_dir);


  1322→    auto write_region = [&](const ParsedData& d, const std::string& base_dir) {
  1323→        ensure_dir(base_dir);
  1324→
  1325→        if (multi_output) {
  1326→            // Write all 3 modes
  1327→            write_index(d, base_dir + "/full", IndexMode::Full);
  1328→            write_index(d, base_dir + "/no-addresses", IndexMode::NoAddresses);
  1329→            write_index(d, base_dir + "/admin", IndexMode::AdminOnly);
  1330→        } else {
  1331→            write_index(d, base_dir, mode);
  1332→        }
  1333→
  1334→        // Write quality variants under admin/ (or base_dir if not multi_output)
  1335→        if (multi_quality) {
  1336→            std::string admin_dir = multi_output ? base_dir + "/admin" : base_dir;

    auto write_region = [&](const ParsedData& d, const std::string& base_dir) {
        ensure_dir(base_dir);

        if (multi_output) {
            // Write all 3 modes
            write_index(d, base_dir + "/full", IndexMode::Full);
            write_index(d, base_dir + "/no-addresses", IndexMode::NoAddresses);
            write_index(d, base_dir + "/admin", IndexMode::AdminOnly);
        } else {
            write_index(d, base_dir, mode);
        }

        // Write quality variants under admin/ (or base_dir if not multi_output)
        if (multi_quality) {
            std::string admin_dir = multi_output ? base_dir + "/admin" : base_dir;


The 3 mode writes are **sequential**! Full, no-addresses, and admin-only are written one after another. Since they read shared data and write to separate directories, they can run in parallel:


Edit

    auto write_region = [&](const ParsedData& d, const std::string& base_dir) {
        ensure_dir(base_dir);

        if (multi_output) {
            // Write all 3 modes
            write_index(d, base_dir + "/full", IndexMode::Full);
            write_index(d, base_dir + "/no-addresses", IndexMode::NoAddresses);
            write_index(d, base_dir + "/admin", IndexMode::AdminOnly);
        } else {
            write_index(d, base_dir, mode);
        }

        // Write quality variants under admin/ (or base_dir if not multi_output)
        if (multi_quality) {
            std::string admin_dir = multi_output ? base_dir + "/admin" : base_dir;

    auto write_region = [&](const ParsedData& d, const std::string& base_dir) {
        ensure_dir(base_dir);

        if (multi_output) {
            // Write all 3 modes in parallel (they read shared data, write to separate dirs)
            auto wf1 = std::async(std::launch::async, [&]{ write_index(d, base_dir + "/full", IndexMode::Full); });
            auto wf2 = std::async(std::launch::async, [&]{ write_index(d, base_dir + "/no-addresses", IndexMode::NoAddresses); });
            auto wf3 = std::async(std::launch::async, [&]{ write_index(d, base_dir + "/admin", IndexMode::AdminOnly); });
            wf1.get(); wf2.get(); wf3.get();
        } else {
            write_index(d, base_dir, mode);
        }

        // Write quality variants under admin/ (or base_dir if not multi_output)
        if (multi_quality) {
            std::string admin_dir = multi_output ? base_dir + "/admin" : base_dir;