📡 Telegram Parser API

An async FastAPI service that scrapes messages from public and private Telegram channels — with optional LLM structuring.

Quick Start · How It Works · Endpoints · Configuration

---

🚀 What is Telegram Parser API?

Telegram Parser API is an asynchronous HTTP service for extracting messages from Telegram channels and groups. Backed by Telethon (the MTProto client library), it can read public channels by username or link and private ones the authorized account can join by invite link. Every message is exported as JSON with author, date, view count, and a direct link to the post. Optional endpoints filter the results and run them through an LLM (via Together AI) to turn raw posts into structured records.

⚡ Quick Start

Docker Compose:

git clone https://github.com/simeonkolchin/telegram-parser-api.git
cd telegram-parser-api
cp .env.example .env      # add your Telegram API credentials
docker compose up --build
# → API on http://localhost:7008  ·  interactive docs at /docs

Local (Python 3.11+):

pip install -r requirements.txt
cp .env.example .env
uvicorn app.main:app --host 0.0.0.0 --port 7008

On first run Telethon prompts for a login code and writes a telethon_parser_session.session file. Keep it private — it grants full access to the account. It is gitignored and never baked into the image.

🏗️ How It Works

flowchart LR
    C[HTTP client] --> API[FastAPI routers]
    API --> P[TelethonParser / Searcher]
    P --> TG[(Telegram · MTProto)]
    API --> F[Message filters]
    F --> LLM[Together AI<br/>Llama-3.3-70B]
    API --> R[JSON response]

Loading

A request hits a router, which drives a TelethonParser (or channel searcher) against Telegram over MTProto. Raw messages are optionally passed through a filter stage (dedupe, keywords, length, recency, …) and, for the llm routes, through a Together AI model that extracts structured fields from each post. The result is returned as JSON.

✨ Key Features

• 🌐 Public parsing — by username, @username, or https://t.me/... link
• 🔒 Private parsing — by invite link (t.me/+..., joinchat/...)
• 🔎 Channel search — parallel keyword search across Telegram
• 🧹 Filtering — dedupe, keywords, min length, views, recency, drop-empty
• 🧠 LLM structuring — turn posts into typed JSON via Together AI (optional)
• ⚡ Fully async — concurrent message fetching with bounded parallelism
• 📖 Self-documenting — OpenAPI docs at /docs, spec exported on startup

🔌 Endpoints

Method	Path	Description
POST	/public-parser/parse	Parse a public channel/group
POST	/public-parser/parse/filtered	Parse + filter a public channel
POST	/public-parser/parse/llm/filtered	Parse + filter + LLM structuring
POST	/private-parser/parse	Parse a private channel by invite link
POST	/private-parser/parse/llm/filtered	Private parse + filter + LLM
POST	/channels/search	Search channels/groups by keywords
GET	/health	Health check
GET	/	Redirect to /docs

Example:

curl -X POST http://localhost:7008/public-parser/parse \
  -H "Content-Type: application/json" \
  -d '{"link": "https://t.me/durov"}'

⚙️ Configuration

Configuration is via .env (see .env.example):

Variable	Description
API_ID	Telegram API ID from my.telegram.org
API_HASH	Telegram API hash
TOGETHER_API_KEYS	Comma-separated Together AI keys (for LLM endpoints)

🛡️ Security

• Credentials live only in .env (gitignored); .env.example ships placeholders.
• The Telethon *.session file is gitignored and must never be committed — it authenticates as the associated Telegram account.
• Respect Telegram's Terms of Service and the target channels' rules when scraping.

See SECURITY.md for how to report a vulnerability.

📚 Documentation

• Getting Started
• Architecture

🗺️ Roadmap

• Persistent storage of parsed messages (PostgreSQL scaffolding exists)
• Rate-limit / backoff controls per channel
• Pluggable LLM providers beyond Together AI
• Auth on the API surface

🧑‍💻 Contributing

Contributions welcome — see CONTRIBUTING.md.

📄 License

MIT © Simeon Kolchin