An AI patent-mapping pipeline keeps its decisions open to inspection

Patent documents describe proposed technologies in specialized language, making large collections difficult to navigate. Automated tools can classify inventions and group related documents, but the resulting map depends on choices made throughout the process. A mistaken classification can change which documents are grouped together; an appealing topic name can conceal an uncertain interpretation.

Sercan Ozcan, Enes Ozgozler and C. Okan Sakar address that problem in an arXiv preprint posted on 6 November 2026. Their contribution is not a new language model or clustering algorithm. It is a workflow that connects established tools while retaining records of how each stage produced its outputs. The study demonstrates that the system runs at scale, rather than establishing that its judgments are accurate.

Three models vote, with an important fallback

The researchers began with 46,375 patent documents retrieved from Lens.org using terms associated with local government, infrastructure and services. Three language models—Kimi K2, DeepSeek-V3.1 and gpt-oss-120b—separately assessed relevance. Documents supported by at least two models entered the final collection of 30,964 records.

These are documents judged relevant to public-sector functions, not necessarily government-owned inventions or technologies adopted by municipalities. The collection includes applications, granted patents and utility models.

The same three models then classified each document along two dimensions. One concerned novelty: whether the disclosed invention appeared radical or incremental. The other concerned its form: product, process or service. These type labels could overlap, allowing an invention to receive more than one.

Each type required two supporting votes. The novelty rule was asymmetric: two Radical votes were required for that label, but every other outcome defaulted to Incremental. Consequently, an Incremental assignment did not always mean two models explicitly agreed on it. Unparseable responses counted as no vote, and the system retained individual decisions so fallback cases could be identified.

The models labeled 1,749 documents, or 5.65%, Radical. They assigned Product to 93.93%, Process to 11.05% and Service to 1.91%; 2,111 documents received multiple type labels. These figures describe automated judgments within this selected collection, not the distribution of innovation across the public sector. A Radical label also does not establish subsequent adoption or technological impact.

From documents to topics

The workflow next discovered topics separately within the five classification groups. PatentSBERTa converted abstracts into numerical representations intended to capture technological meaning. UMAP simplified those representations, and HDBSCAN grouped nearby documents into clusters while allowing some to remain outliers.

The researchers adjusted candidate settings to collection size, from 591 Service documents to 29,215 Incremental documents. They selected configurations using a weighted combination of document coverage, cluster separation and distinctiveness of topic vocabulary. This was a tuning rule, not an independent test of quality.

A deliberate separation governed text processing. Abstracts entered the document-embedding model without stopword removal. Common, low-information patent language was filtered only when clusters were represented as ranked keywords, using a method called class-based term frequency–inverse document frequency. This preserved wording and context for the first model while limiting boilerplate in topic descriptions.

A second embedding model, Qwen3-Embedding, supported comparisons and refinement of topics. Finally, GPT-5-mini received each topic’s ten highest-ranked keywords and ten representative patents to generate a short label, rationale and confidence category. Those inputs and outputs were retained for review.

High coverage is not proven accuracy

The five separately modeled collections produced 74 category-specific topic instances. These are not 74 mutually exclusive themes in one global map: documents can appear in overlapping type groups and are also analyzed by novelty.

Coverage—the share assigned to non-outlier topics—ranged from 94.4% to 100%. Its unweighted average across the five category models was 98.1%, not a single coverage rate for unique documents. Topic diversity, measuring the proportion of unique words among topics’ leading terms, ranged from 0.900 to 1.000. Neither measure establishes that the topics are substantively correct.

More than 99% of classification outputs were parseable. All 74 topic-label outputs met the required format, and the labeling model rated every one HIGH confidence. Uniform confidence does not distinguish trustworthy labels from questionable ones, and successful formatting is not successful interpretation.

A reviewable starting point

The authors have not tested classifications against expert coding or benchmarked the pipeline against alternative topic-modeling approaches. Models can share errors, so majority agreement is not proof of correctness. The benefits of individual design choices also remain unisolated.

Data scope adds further limits. Analysis used English-language titles and abstracts rather than full descriptions and claims. The initial collection was heavily concentrated in Chinese publications and utility models; its composition should not be assumed to match the retained collection. Related documents from the same patent family could also remain.

The preprint’s practical result is therefore an inspectable starting point: analysts can trace labels back to votes, keywords and documents. Whether those labels provide a reliable technology map still requires independent evaluation.