Skip to content

Utility Commands

File system access, spec downloading, and database diagnostics.

TDoc Access

open

Download (if needed), extract, and open the main document of a TDoc with the system default application. The main file is picked from the archive by the configured selector (heuristic by default; see checkout.selector in Configuration) and can be overridden per TDoc with main-file.

3gpp-crawler tdoc open S4-250638
3gpp-crawler tdoc open S4-250638 --selector decision_model   # one-off strategy

--selector accepts heuristic, decision_model, or manual and overrides checkout.selector for that run only. Decision-model selection additionally requires an API key (checkout.systemone_api_key / TDC_CHECKOUT_SYSTEMONE_API_KEY or TYPESAFE_API_KEY); without one, tdoc open logs and falls back to the heuristic. The resulting source is printed (main file: heuristic|decision_model|manual_override).

checkout

Download and extract TDoc(s) to the checkout folder for offline access.

3gpp-crawler tdoc checkout S4-250638
3gpp-crawler tdoc checkout S4-250638 S4-250639
3gpp-crawler tdoc checkout S4-250638 --keep-zip   # keep the downloaded zip next to the extracted files

A TDoc that is not in the local database is looked up by ID first through the public portal download endpoint: that yields the file URL without credentials, so the checkout (or open) works on a cold database. Credentialed portal metadata and WhatTheSpec remain the fallbacks. The parent meeting is resolved from the download URL, and the meetings of the TDoc's subgroup are fetched once when they are missing locally, so fetched records are stored with their meeting linkage instead of as orphans.

A TDoc without any downloadable document (WhatTheSpec knows the ID but returns an empty link, e.g. a numbering gap or a meeting that is not published yet) fails immediately with no download URL; no checkout folder is created.

The command exits with code 1 when any TDoc fails, after printing every failure reason. Withdrawn TDocs are reported as skipped and do not fail the run.

Extraction is configurable per config file ([extract], no CLI flag or env var): flatten_single_root = true (default) moves the contents of a single top-level wrapper folder into the TDoc folder root, so an archive containing S4-262108/S4-262108.docx is extracted as <tdoc-folder>/S4-262108.docx. exclude holds gitignore-style patterns that are skipped during extraction (defaults: .git/, .hg/, .svn/); excluded members are never written, and the built-in clutter rules (__MACOSX/, .DS_Store, …) keep applying. See Configuration.

files

List the files inside a TDoc's zip archive. The first call reads the listing from the server (HTTP range reads, no full download) and stores it; later calls use the stored listing. --refresh re-reads it.

3gpp-crawler tdoc files S4-250638
3gpp-crawler tdoc files S4-250638 --refresh

main-file

Manage manual main-document overrides. An override is a stored archive member path and takes precedence over the configured selector when opening a TDoc. Use the exact member path printed by tdoc files; the value must be relative to the archive root (.. and absolute paths are rejected), and tdoc open fails loudly if the stored member is no longer inside the archive.

tdoc open also records the resulting classification in the database — member path, source (manual_override, decision_model, heuristic), selector, confidence — so a later run can replay it. show displays both the override and that stored record.

3gpp-crawler tdoc main-file set S4-250638 "Cover sheet.pdf" --note "real contribution"
3gpp-crawler tdoc main-file show S4-250638      # override + stored classification
3gpp-crawler tdoc main-file list                # every override
3gpp-crawler tdoc main-file clear S4-250638     # remove the override

Setting checkout.selector = "manual" (or TDC_CHECKOUT_SELECTOR=manual) disables the heuristic and the decision model, so only overrides are used.

main-file reclassify

Re-run classification for TDocs that are already checked out, without downloading anything. Use this after changing the selector, configuring the decision model, or upgrading the heuristic.

3gpp-crawler tdoc main-file reclassify S4-250638 S4-250639
3gpp-crawler tdoc main-file reclassify --all
3gpp-crawler tdoc main-file reclassify --all --selector decision_model

# Overwrite manual choices too (deletes those overrides):
3gpp-crawler tdoc main-file reclassify --all --include-manual --confirm-overwrite

TDocs without a stored classification or without a checkout are reported as missing; manual choices are skipped unless --include-manual is given. The decision model works best when configured once in the global config (~/.config/3gpp-crawler/config.toml); see Configuration.

Specifications

open

Download and open the latest document for a specification.

3gpp-crawler spec open 23.501
3gpp-crawler spec open 23.501 --release 17

checkout

Batch download specification documents to the checkout folder.

Features:

  • Progress bar for multi-spec operations.
  • Range syntax support (e.g. 26130-26140).
  • --release selector (latest or e.g. 18.0).
3gpp-crawler spec checkout 26130-26140
3gpp-crawler spec checkout 26260 --release 18.0

files

List the files inside a specification's zip archive. Stored from checkout; if nothing is stored yet, the listing is read from the server (HTTP range reads) and kept.

3gpp-crawler spec files 26260 --release 18.0
3gpp-crawler spec files 26260 --refresh

Diagnostics

stats

Display high-level database statistics.

3gpp-crawler tdoc stats

config

Manage configuration — see the Configuration guide for init, show, validate, and docs subcommands.