GitHub Research

GitHub Research is an open-source tool for researching GitHub trending repositories, newly emerging projects, and keyword-driven repository activity. It uses Decodo's Web Data API to retrieve public GitHub pages and your chosen LLM provider to turn normalized repository data into structured research reports.

Unlike leaderboard-style trending trackers, GitHub Research can start from a keyword or concept and return recurring themes, notable repositories, and historical changes instead of only a ranked list.

GitHub Research is not affiliated with or endorsed by GitHub, Inc.

_{Choose Trending, Emerging, or Keyword research from the main screen.}

Why GitHub Research?

GitHub Research turns public repository activity into a structured research report instead of stopping at a leaderboard.

Three research modes. Explore GitHub Trending, discover newly created repositories that meet traction filters, or start from a keyword or concept.
LLM-synthesized findings. Get a research summary, recurring themes, notable repositories, and deterministic repository metrics in one report.
Longitudinal monitoring. Save any research configuration as a recurring monitor and compare later snapshots with earlier runs.
Verifiable outputs. Keep GitHub source links and repository-level data visible while deterministic metrics remain separate from LLM interpretation.
Portable reports. Export completed research as Markdown or JSON, with every run stored in History for later review.

_{Example Emerging report generated on September 18, 2026.}

How it works

Choose a research mode. Select Trending, Emerging, or Keyword and configure the relevant filters.
Build the GitHub research request. The backend converts the configuration into GitHub Trending or repository-search URLs.
Retrieve public GitHub pages. Decodo Web Data API retrieves the page HTML used for the research run.
Normalize repository data. Deterministic parsers extract repository metadata into a shared repository-snapshot format.
Generate the research report. Your selected LLM provider receives the normalized current dataset, deterministic metrics, and relevant previous snapshots to produce themes and notable findings.
Store the run. MongoDB stores the configuration, repository snapshots, and generated report for History and later comparisons.
Review or export the result. Open the report in the UI or download it as Markdown or JSON.

The current research flow does not use the GitHub REST API as its data source. It retrieves public GitHub pages through Decodo Web Data API.

Prerequisites

Before installing the project, make sure you have:

Bun 1.2.5 or newer
Docker Desktop with Compose, Podman Desktop with Compose, or an existing MongoDB server
A Decodo Web Data API key – create a Decodo account
At least one supported LLM provider API key:
- Anthropic
- OpenAI
- Google Gemini

If you've just installed Bun, open a new terminal window or reload your shell configuration before running bun install.

Examples:

zsh: source ~/.zshrc

bash: source ~/.bashrc

Installation

1. Clone the repository

git clone https://github.com/Decodo/github-research.git
cd github-research

2. Install dependencies

bun install

3. Configure environment variables

Create a local .env file from the included example:

cp .env.example .env

Files beginning with a dot may be hidden by default. If you don't see .env in the project folder:

On macOS Finder, press Cmd+Shift+.
On most Linux file managers, press Ctrl+H
On Windows, use View → Show → Hidden items if needed

Open .env, select your LLM provider, and add its API key. You can configure the Decodo Web Data API key here or enter it later on the Settings page.

# Backend
PORT=5002
PUBLIC_API_BASE_URL=http://localhost:5002
PUBLIC_FRONTEND_URL=http://localhost:5274

# Database
MONGO_PORT=27018
MONGODB_URI=mongodb://localhost:27018/github-research

# Decodo Web Data API
# Optional: paste the key without the "Bearer" prefix, or enter it in Settings.
DECODO_WEB_DATA_API_KEY=
DECODO_SCRAPER_ENDPOINT=https://data.decodo.com/v1/scrape

# LLM provider: claude | openai | gemini
LLM_PROVIDER=claude
LLM_MODEL=

ANTHROPIC_API_KEY=
OPENAI_API_KEY=
GEMINI_API_KEY=

Only the API key for the selected LLM_PROVIDER is required. LLM_MODEL is optional and can be left empty to use the provider default configured by the application.

4. Start MongoDB

Start Docker Desktop or your Podman machine, with Compose support enabled, then start the local MongoDB container:

bun db:up

The command automatically detects a running Docker or Podman engine and a compatible Compose provider. You do not need the legacy docker-compose command. For Podman Desktop, install the Compose extension through Settings → Resources → Compose → Setup if it is missing. Use CONTAINER_RUNTIME=podman bun db:up to explicitly prefer Podman when both engines are installed. The matching bun db:down command stops the container.

If you already have MongoDB running, you can skip bun db:up and set MONGODB_URI to its connection string. MongoDB remains the required database; the container runtime is interchangeable. If port 27018 is occupied, stop the conflicting service or change MONGO_PORT together with MONGODB_URI.

5. Build the application

bun run build

6. Start GitHub Research

bun dev

Keep the terminal window open while using the application. Closing it stops the local frontend and backend. After startup, open Settings to save a Web Data API key if you did not configure one in .env.

Frontend:

http://localhost:5274

Backend API:

http://localhost:5002

Find trending GitHub repositories by language and timeframe

Trending mode researches GitHub's own Trending page and turns the current ranking into a structured research report.

You can configure:

Timeframe. Daily, weekly, or monthly
Programming language. Optional
Spoken language. Optional, using GitHub Trending's spoken-language filter

For each repository, GitHub Research can collect the repository name, description, programming language, total stars, forks, and the period-specific star count exposed by GitHub Trending. The report then identifies supported themes, notable repositories, the most represented language, and the top period-star mover.

Period-star counts come from GitHub Trending itself. GitHub Research does not attempt to reproduce or reverse-engineer GitHub's Trending ranking algorithm.

Discover emerging repositories before they trend

Emerging mode uses GitHub repository search to find recently created repositories that already meet the thresholds you choose.

Available filters include:

Created within. Past day, past 7 days, or past 30 days
Programming language. Optional
Minimum stars
Minimum forks. Optional
Exclude forked repositories. Enabled by default

The resulting report focuses on current star counts, languages, repository topics, recurring clusters, and standout new projects.

In this tool, emerging means newly created repositories that match the selected filters. It is an operational research category, not a prediction that a repository will become broadly popular.

Run keyword research across GitHub repositories

Keyword mode starts from a term or concept such as MCP, RAG, or browser agent and researches matching GitHub repositories.

The main search always keeps a current relevance landscape. You can optionally select a 7-day, 30-day, or 90-day recent repository window, which adds repositories created during that period instead of replacing the broader landscape with creation-only results.

Keyword research can use:

Repository names and descriptions
Programming languages
GitHub topics exposed in repository search
Current star counts
An optional minimum-star threshold
Recently created matching repositories when a recent window is selected

The LLM analyzes the normalized repository data and returns a research summary, recurring themes, notable repositories, and historical comparisons when previous matching runs exist.

Research accuracy improvements

Evidence-backed themes. Each reported theme must have exact quotes from at least two current repositories and name its supporting repositories. The tool returns up to three themes and may return none rather than inventing connections.
Standout Trending activity. Deterministic checks highlight dominant rolling period-star figures and unusually large changes in the rolling metric between saved snapshots. These are not the number of new stars earned between runs.
Recurring name clusters. Terms shared by at least three repository names can be reported independently of the LLM, without claiming a shared cause.
More complete source data. GitHub search attempts a second page when the first is full. Missing topics, descriptions, and fork counts can be filled by scraping a bounded subset of repository pages, while failures do not prevent the main report.
More dependable comparisons. Historical runs are matched by research configuration, and overlapping repository counts and measured deltas are computed outside the LLM. Short comparison windows are described as diagnostic, not durable trends.
Richer exports. The UI and Markdown reports include the standalone signals, name clusters, supported themes, and historical comparisons.

Metadata enrichment can add up to six Web Data API requests per run by default. Set GITHUB_METADATA_ENRICH_LIMIT=0 to disable it, or use a value between 1 and 12 to control the maximum number of additional repository requests. Pagination may add one more request in search-based modes.

Track GitHub repository research over time

Any research configuration can be saved as a recurring monitor.

Supported schedules include:

Custom intervals in hours
Daily
Weekly
Monthly

The local NestJS backend checks for due monitors every 15 seconds. Automatic runs occur only while the backend and MongoDB are running, so the included scheduler is intended for local or always-on deployments rather than offline execution.

The Monitors view shows saved configurations, next and previous run times, and the latest run status. Monitors can be run manually, paused or resumed, and deleted. Pausing a monitor stops future scheduled runs; it does not cancel research that is already in progress.

Each completed run is stored in History. When a matching previous snapshot exists, GitHub Research compares shared repository metrics and repository composition across runs. Comparison windows under six hours are treated as diagnostic snapshots rather than evidence of a durable trend.

How this compares to GitHub Trending, OSS Insight, and Star History

GitHub Research is designed as a research layer rather than a replacement for every GitHub analytics tool.

	GitHub Research	GitHub Trending	OSS Insight	Star History
Primary workflow	Research trending, emerging, or keyword-based repository activity	Browse repositories currently trending on GitHub	Query and analyze large-scale GitHub event data	Explore repository star growth and star-based trends
Keyword or concept research	Yes, with LLM-synthesized themes across matching repositories	No dedicated keyword-research workflow	Natural-language analytics queries over its GitHub dataset	Primarily repository and star-history oriented
LLM-synthesized repository themes	Yes	No	Analytics answers and visualizations rather than this report format	No
Emerging repository workflow	Yes, using creation windows and user-defined star/fork filters	No dedicated newly-created-repository mode	Rankings and data exploration	Star-growth and trending views
Recurring local monitors	Yes	No	Not the same local monitor workflow	No equivalent local research-monitor workflow
Stored reports and longitudinal comparison	Yes	No	Historical analytics are available	Star history is the core focus

The comparison describes each product's main workflow. These tools solve overlapping but different problems.

Configuration

Environment variables

Variable	Description
`PORT`	Backend port. Defaults to `5002`
`HOST`	Backend bind address. Defaults to `127.0.0.1` for local use
`CONTAINER_RUNTIME`	Optional `podman` or `docker` preference for the database scripts
`GITHUB_RESEARCH_DATA_DIR`	Optional directory for the locally saved Web Data API key; defaults to the backend's `.local` directory
`PUBLIC_API_BASE_URL`	API URL used by the frontend
`PUBLIC_FRONTEND_URL`	Frontend URL used by the application
`MONGO_PORT`	Host port mapped to the MongoDB container
`MONGODB_URI`	MongoDB connection string
`DECODO_WEB_DATA_API_KEY`	Optional Decodo Web Data API key, without the `Bearer` prefix. Alternatively save it in Settings. The old `DECODO_AUTH_TOKEN` is supported as a fallback
`DECODO_SCRAPER_ENDPOINT`	Decodo scraping endpoint. The included default is normally sufficient
`GITHUB_METADATA_ENRICH_LIMIT`	Optional maximum repository-page metadata enrichment requests per run (default: 6; maximum: 12; 0 disables)
`LLM_PROVIDER`	LLM provider: `claude`, `openai`, or `gemini`
`LLM_MODEL`	Optional model override
`ANTHROPIC_API_KEY`	Anthropic API key
`OPENAI_API_KEY`	OpenAI API key
`GEMINI_API_KEY`	Google Gemini API key

The Settings page lets users select their LLM provider/model and enter or replace the Decodo Web Data API key. An interface-entered key takes precedence over the environment variable. It is stored in a local, Git-ignored backend file (apps/backend/.local/web-data-api-key when using bun dev) with restrictive filesystem permissions, not in MongoDB. API responses return only a configured indicator and source; they never return the secret. Clearing the saved key restores the environment-variable fallback. LLM provider keys remain environment variables.

Security note: The application defaults to a loopback-only backend (HOST=127.0.0.1). The settings API is not an authenticated multi-user secrets manager. For remote deployments, put it behind an authenticated reverse proxy and secure the host and credential file. Back up the local credential file separately if required.

Research filters

Mode	Setting	Options / behavior
Trending	Timeframe	Daily, weekly, monthly
Trending	Programming language	Optional
Trending	Spoken language	Optional GitHub spoken-language code
Emerging	Created within	1 day, 7 days, 30 days
Emerging	Programming language	Optional
Emerging	Minimum stars	Numeric threshold
Emerging	Minimum forks	Optional numeric threshold
Emerging	Exclude forks	Enabled by default
Keyword	Keyword	Required
Keyword	Recent repository window	Any time, 7 days, 30 days, 90 days
Keyword	Programming language	Optional
Keyword	Minimum stars	Numeric threshold

Research methodology

GitHub Research separates deterministic repository metrics from LLM interpretation.

Deterministic facts stay authoritative. Repository counts, language counts, star counts, period-star counts, and source URLs come from parsed GitHub data rather than from the LLM.
Themes require supporting evidence. The analysis prompt normally requires at least two current repositories before promoting a pattern into a theme. A single unusual project can still appear under notable repositories.
Unknown names are not silently defined. If multiple repositories mention an unfamiliar product, acronym, model, or concept, the report can identify the observable cluster but should not infer what the term means unless the repository metadata establishes it.
Emerging mode does not invent star velocity. GitHub repository-search results provide current star counts, not period-specific star gains, so Emerging reports do not claim a top period mover.
Keyword windows are supplemental. A recent window adds recently created repositories to the current keyword landscape instead of treating every result as newly created.
Historical claims use matching evidence. Repository deltas are compared only when the same repository and metric are available in earlier snapshots. Windows under six hours are explicitly treated as diagnostic.

API endpoints

Method	Path	Description
`POST`	`/research`	Run a GitHub research request
`POST`	`/research/stream`	Run research with NDJSON progress events
`GET`	`/queries`	List saved research history
`GET`	`/queries/:id`	Retrieve one stored report
`DELETE`	`/queries/:id`	Delete a history entry
`GET`	`/monitors`	List recurring monitors
`POST`	`/monitors`	Create a recurring monitor
`PATCH`	`/monitors/:id`	Update, pause, or resume a monitor
`POST`	`/monitors/:id/run`	Run a saved monitor immediately
`DELETE`	`/monitors/:id`	Delete a monitor
`GET`	`/settings`	Return provider/model selection and API-key presence status
`PATCH`	`/settings`	Update provider or model preference

FAQ

What counts as a trending GitHub repository in this tool?

Trending mode uses the repositories currently returned by GitHub Trending for the selected daily, weekly, or monthly window and optional language filters. GitHub Research analyzes that retrieved set and the period-star values GitHub exposes. It does not independently define or reconstruct GitHub's Trending ranking algorithm.

How is GitHub Research different from GitHub's own Trending page?

GitHub Trending provides a ranked view of repositories attracting attention for a selected period. GitHub Research can analyze that set, but it also adds Emerging and Keyword modes, LLM-synthesized themes, notable-repository explanations, stored reports, recurring monitors, historical comparisons, and Markdown or JSON export.

How does the keyword research mode work?

Keyword mode runs a current GitHub repository search for the supplied term and analyzes repository metadata such as descriptions, topics, languages, and stars. If you select a 7-day, 30-day, or 90-day recent window, the tool also retrieves repositories created during that period and adds them to the broader current landscape.

Does GitHub Research use the GitHub API or Decodo Web Data API?

The current research flow uses Decodo Web Data API, not the GitHub REST API, to retrieve public GitHub pages. The backend then parses the returned HTML into normalized repository snapshots before passing structured data, deterministic facts, and relevant historical snapshots to the selected LLM provider.

What are GitHub's API rate limits, and how does this tool handle them?

GitHub documents a primary REST API limit of 60 requests per hour for unauthenticated requests and 5,000 requests per hour for most authenticated users, with separate or stricter limits for some resources such as search. GitHub Research does not use the REST API as its research data source, so those REST API request quotas are not the collection mechanism used by this project. See GitHub's rate-limit documentation.

Do I need a Decodo account?

Yes. GitHub Research retrieves its GitHub source pages through Decodo Web Data API, so you need a Decodo account and a valid Web Data API key. Set DECODO_WEB_DATA_API_KEY in .env or enter the key on the Settings page. The earlier DECODO_AUTH_TOKEN is still accepted for compatibility. The Settings API never returns the key itself.

Is GitHub Research affiliated with GitHub, Inc.?

No. GitHub Research is an independent open-source project maintained by Decodo. It researches publicly available GitHub repository data and links reports back to GitHub source pages, but it is not affiliated with, sponsored by, or endorsed by GitHub, Inc.

Can I self-host GitHub Research?

Yes. GitHub Research is designed to run locally or on an always-on host with Bun, MongoDB, and the required API credentials. Recurring monitors only execute while the backend and database are running, so scheduled monitoring requires the application to remain online between runs.

Tech stack

Layer	Technology
Frontend	React 19, TanStack Router, TanStack Query, Tailwind CSS v4, Radix UI, Rsbuild
Backend	NestJS 11, MongoDB, Mongoose
GitHub retrieval	Decodo Web Data API
Parsing	Cheerio and deterministic GitHub parsers
LLMs	Anthropic Claude, OpenAI GPT, Google Gemini
Runtime	Bun
Local services	MongoDB, Docker or Podman with Compose

Project structure

.github/
  images/              # README screenshots and repository assets

apps/
  backend/src/features/
    decodo/             # Decodo Web Data API integration
    github/             # GitHub URL builders and HTML parsers
    research/           # Research execution, analysis, and reporting
    monitors/           # Saved schedules and recurring execution
    queries/            # History and stored snapshots
    llm/                # Anthropic, OpenAI, and Gemini abstraction layer
    settings/           # Provider/model configuration

  frontend/src/features/
    tracker/            # Research form, progress flow, and report UI
    monitors/           # Recurring monitor controls
    queries/            # History and stored reports
    settings/           # Runtime provider/model settings

  shared/src/           # Shared TypeScript types

docs/
  ARCHITECTURE.md

Scripts

Command	Description
`bun dev`	Build shared types and start frontend/backend development servers
`bun run build`	Build all application packages
`bun start`	Start the production backend and frontend preview server
`bun lint`	Run linting across frontend and backend
`bun format`	Format frontend and backend source files
`bun db:up`	Start MongoDB using an available Docker or Podman Compose provider
`bun db:down`	Stop the local database container

Related repositories

For Web Data API targets, parameters, and request behavior, see the Decodo Web Data API documentation.

License

MIT – see LICENSE.

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
apps		apps
docs		docs
scripts		scripts
.env.example		.env.example
.gitignore		.gitignore
.prettierrc		.prettierrc
AGENTS.md		AGENTS.md
LICENSE		LICENSE
README.md		README.md
bun.lock		bun.lock
docker-compose.yml		docker-compose.yml
package.json		package.json
tsconfig.json		tsconfig.json

Latest commit

History

Folders and files

Repository files navigation