Open source · Methodology

How the directories are built.

How Shadowfetch Linux ranks public GitHub repositories and Hugging Face models and datasets, what is excluded, and how the snapshots are refreshed.

What this section is.

The GitHub and Hugging Face lists are machine-collected popularity directories. A listing is not a review, an endorsement, or a claim about quality, safety, or licensing terms. Studio projects on /open-source/projects are a separate, hand-written catalog of Shadowfetch Studio software.

GitHub.

The list comes from the public GitHub REST search API (/search/repositories), called without an account or token. A refresh searches AI-related topics and a few name or description phrases for repositories with at least 500 stars, merges the results, and ranks them by stars (ties alphabetical). GitHub returns at most 1,000 results per search.

These are left out: forks; archived or disabled repositories; curated link lists; courses, tutorials, books and interview prep; leaked or jailbreak prompt collections; deepfake and face-swap tools; and repositories with no AI signal in the name, description, or topics. A short reviewed denylist covers what the automatic rules miss. Catalog extras — repositories named from Studio project pages — stay even when they fall below the published rank cutoff (the first 1,000 by stars).

Slugs are owner--repository, assigned on the full collected list before the published set is trimmed, so a later refresh keeps the same URL when a project stays in the list. Collision suffixes are -2, -3, in rank order.

Hugging Face.

Models and datasets come from the public Hub API (/api/models and /api/datasets), sorted by likes, without an account or token. Models and datasets are ranked separately. Likes measure Hub popularity; downloads are the Hub's last-30-day file request counts.

These are left out: private, disabled, or removed items; items tagged not-for-all-audiences or whose name marks them as explicit; deepfake, face-swap, and undressing tools; fine-tunes that remove safety training (names such as uncensored or abliterated); and jailbreak or leaked-prompt collections. Detectors and evaluation sets stay. A reviewed denylist covers the rest. Catalog extras stay below the published cutoffs (400 models, 100 datasets).

Slugs use the same author--name rule as GitHub, assigned on the full list and then trimmed.

Refresh.

Snapshots live in src/data/open-source/. Run node scripts/refresh-open-source.mjs to fetch the public APIs again. The script uses no secrets, identifies itself as this site, and is not scheduled. GitHub's unauthenticated search limit is low; a full run pauses between queries and may stop early, keeping what it has. After a refresh, rebuild the site so the pages match the new snapshot.

Pages that drop out of the published set are not kept as thin stubs. The Worker sends an unknown /open-source/github/…, /open-source/huggingface/…, or /open-source/projects/… path to that section's index.