⚽ FootyDraft

FootyDraft is a Telegram‑first bot that uses AI to help you run a high quality, on‑trend football Twitter (X) account. It continuously monitors news sources, fetches live match statistics, generates context‑aware tweet drafts (news, stats, and insights) using a large language model, and delivers them to you via a Telegram bot for manual review, copying, and optional Xquik posting. Over time, it learns from your engagement metrics to refine its writing style – all while keeping you in full control.

---

✨ Features

1. Real‑time news ingestion (RSS, Reddit, Google News, API‑Football) - 100% free
   • Fast transfer news lane – dedicated high‑frequency polling (every 15 minutes) of six fast‑updating RSS feeds for near‑real‑time transfer gossip
   • Post‑match & half‑time stat packs – possession, xG, shots, passes automatically generated for every finished match
   • Live in‑game stat snapshots – triggered by the first goal of each half, showing key match stats in real time
   • Weekly leaderboards – top scorers & assists for the World Cup and any future league (via football‑data.org, free tier)
   • Nerdy stats of the week – automatic detection of unusual match patterns (xG overperformance, possession without result, shot barrages, passing extremes, xG shutouts)
   • Strong freshness filter – all news items older than 12 hours are discarded, preventing yesterday’s stories from reappearing
2. Interactive Streamlit Dashboard – view drafts, performance analytics, source health, rules, and backups in one place
3. Hybrid tweet drafting with three personas (pundit, fan, analyst)
4. Three variants per normal event, one per live match event
5. Telegram bot for reviewing, copying, and tracking drafts
6. Manual feedback loop - enter tweet engagement metrics without the X API
7. Optional Xquik posting - publish approved drafts only when XQUIK_POSTING_ENABLED=1
8. Automatic database backups – compressed backup on every boot and nightly at 3 AM UTC (Dropbox primary, Telegram secondary)
9. Real‑time error tracking – Sentry integration to monitor crashes and bugs
10. Built‑in uptime monitor – /uptime command shows how long the bot has been running (WAT)
11. Gated database restore – /restore command with one‑time code, backup file picker, and automatic staging
12. Safe remote restart – /restart with two‑step confirmation, no computer needed
13. Weekly analytics that suggest style improvements based on your best‑performing tweets
14. Deduplication & age filters – 12‑hour age cutoff for RSS, Google News, and Reddit; content‑hash dedup prevents repeated drafts, with special date‑stamping for weekly recurring drafts
15. Deployable 24/7 on Fly.io (or any Docker‑compatible platform)

---

🧠 How It Works

1. News Fetchers pull headlines from BBC, Sky Sports, ESPN, The Guardian, Daily Mail, Reddit, Google News, and live match data from API‑Football. A dedicated fast lane polls six transfer‑focused RSS feeds every 15 minutes. Weekly leaderboard data is sourced from football‑data.org.
2. Event Classifier tags each story (goal, transfer, stats, debate, meme, etc.) using keyword rules.
3. Deduplication Engine ensures the same story doesn’t generate repeated drafts within 24 hours.
4. Prompt Builder combines your account’s persona, the selected mode (pundit/fan/analyst), active style rules, and your top‑performing tweets as few‑shot examples.
5. LLM Client (Groq with Mistral fallback) generates tweet drafts.
6. Telegram Bot shows pending drafts in a queue with inline “Copy” buttons, optional /postx publishing, and instant live‑event drafts.
7. Analytics Engine runs weekly, identifies patterns in your engagement data, and suggests natural‑language rules you can approve or reject.
8. Stats Engine – after every full‑time whistle, match statistics are stored in the database. A weekly job analyses the accumulated data and creates a ‘Nerdy Stats of the Week’ draft highlighting unusual patterns.
9. Leaderboard Engine – twice a week, top scorers and assist leaders are fetched from football‑data.org and turned into a combined draft. Inactive leagues are automatically skipped.
10. Backup Engine – automatically creates a gzipped database backup on every machine restart and daily at 3 AM UTC, uploading to Dropbox (primary) and Telegram (secondary).
11. Monitoring – the /uptime command shows the bot’s start time and elapsed uptime; Sentry integration tracks errors in real time.
12. Recovery – the /restore command uses a gated one‑time code to list the last 5 backups from Dropbox, then stages the selected backup; a /restart reboot applies it automatically.

---

🏗 Architecture

footydraft/
├── config/
│   ├── settings.py          # Environment variables & constants
│   └── personas.yaml        # Persona definitions (pundit, fan, analyst)
├── core/                    # Engine (UI‑agnostic)
│   ├── ingestion/           # News fetchers
│   │   ├── base.py          # Abstract BaseFetcher + NewsItem
│   │   ├── rss_fetcher.py   # BBC, Sky, ESPN, Guardian, etc.
│   │   ├── reddit_fetcher.py
│   │   ├── google_news_fetcher.py
│   │   ├── api_football_fetcher.py
│   │   ├── espn_fetcher.py  # ESPN hidden API (CL, Europa, cups)
│   │   └── monitor.py       # Source health monitoring
│   ├── classification/
│   │   ├── event_tagger.py  # Keyword‑based tagger
│   │   └── dedup.py         # Content‑hash deduplication
│   ├── generation/
│   │   ├── prompt_builder.py
│   │   ├── llm_client.py    # Groq (primary), Mistral (fallback)
│   │   └── queue_manager.py # Draft creation & daily caps
│   ├── publishing/          # Optional Xquik posting
│   │   ├── __init__.py
│   │   └── xquik.py
│   ├── analytics/
│   │   └── engine.py        # Weekly rule suggestions
│   ├── backup.py            # Dropbox & Telegram backup (gzipped)
│   ├── database.py          # SQLAlchemy init & session
│   ├── models.py            # ORM models (Draft, Tweet, Rule, etc.)
│   └── scheduler.py         # Main loop
├── bot/
│   ├── handlers.py          # All command handlers
│   ├── keyboard.py          # Inline keyboards (Copy, pagination)
│   └── main.py              # Bot entry point + live‑draft push
├── dashboard/               # Streamlit dashboard
│   ├── app.py               # Main dashboard entry point
│   ├── ui_components.py     # Shared UI (CSS, sidebar, navigation)
│   ├── utils.py             # Shared utilities (Dropbox, database helpers)
│   ├── requirements.txt     # Dashboard dependencies
│   ├── Dockerfile.dashboard # Dockerfile for dashboard service
│   ├── static/              # Static assets (favicon, etc.)
│   └── pages/               # Dashboard pages
│       ├── Backup_Browser.py
│       ├── Database_Stats.py
│       ├── Drafts.py
│       ├── Live_Check.py
│       ├── Performance.py
│       ├── Rule_Manager.py
│       └── Source_Health.py
├── data/                    # SQLite database (auto‑created)
├── scripts/                 # Diagnostic & utility scripts
│   ├── diagnostic_tags.py   # Check event classifier tag distribution
│   ├── diagnostic.py        # Check API‑Football live data & classification
│   └── direct_api_check.py  # Compare live=all vs date endpoints
├── tests/                   # Unit & integration tests
├── .env.example             # Environment template
├── Dockerfile               # Containerised deployment
├── fly.toml                 # Fly.io configuration
├── requirements.txt
├── start.sh                 # Startup script (scheduler + bot)
└── README.md

---

🚀 Quick Start

Prerequisites

• Python 3.11+
• Groq API key
• Telegram bot token (from @BotFather)
• API‑Football key for live match data

1. Clone & set up environment

git clone https://github.com/MikkyPrestige/footydraft.git
cd footydraft
python -m venv venv
source venv/bin/activate   # or venv\Scripts\activate on Windows
pip install -r requirements.txt
cp .env.example .env
# Now edit .env with your real API keys

2. Initialise the database

python -c "from core.database import init_db; init_db()"

3. Run the scheduler & bot locally

Terminal 1 – Core scheduler (fetches news, generates drafts):

python -m core.scheduler

Terminal 2 – Telegram bot:

python -m bot.main

4. Open Telegram & try commands

• /start – welcome message and command list
• /queue – view pending drafts (all, normal, or live) with Copy buttons & pagination
• /queue normal – only normal pending drafts
• /queue live – only live-match drafts
• /drafts – browse all drafts (all, pending, held, posted) with pagination and copy buttons
• /drafts pending – only pending normal drafts
• /drafts held – only held drafts
• drafts posted – only posted drafts
• /leaderboard – fetch latest top scorers & assists leaderboard
• /nerdystats – generate nerdy stats of the week draft
• /hold <draft_id> – quarantine a pending draft
• /release <draft_id> – return a held draft to the queue
• /posted <draft_id> – mark a draft as manually posted & link a tweet
• /postx <draft_id> [variant] – post an approved draft via Xquik (requires XQUIK_POSTING_ENABLED=1)
• /metrics <tweet_ref> <likes> <retweets> <replies> <impressions> – enter engagement numbers
• /stats – top & bottom tweets by likes (default)
• /impressions – top & bottom tweets by impressions
• /tweets – list all posted tweets with refs
• /rules – manage style rules (accept/reject auto‑suggestions)
• /addrule <text> – add a manual style rule
• /source_status – check health of news sources
• /livecheck – force check for live matches
• /clearqueue – delete all pending drafts (or only normal / live)
• /clearqueue normal – delete only normal pending drafts
• /clearqueue live – delete only live pending drafts
• /uptime – bot uptime and start time (WAT)
• /restore – gated database restore from Dropbox backups
• /restart – Restart the bot (two-step confirmation)

5. (Optional) Run the Streamlit Dashboard

# Install dashboard dependencies
pip install -r dashboard/requirements.txt

# Run the dashboard
streamlit run dashboard/app.py

The dashboard will be available at http://localhost:8501

6. (Optional) Run both bot and dashboard with Docker Compose

Start both services

docker compose up -d

Start only the dashboard

docker compose up -d dashboard

Start only the bot

docker compose up -d bot

The dashboard will be available at http://localhost:8501

---

☁️ Deploying to Fly.io (24/7)

The project includes a Dockerfile and fly.toml for one‑click deployment to Fly.io. It runs both the scheduler and the Telegram bot in a single small VM (~$1.94/month).

1. Install the Fly CLI: curl -L https://fly.io/install.sh | sh
2. Log in: flyctl auth login
3. Create the app & volume (run inside the project folder):

   flyctl apps create footydraft --yes
   flyctl volumes create agent_data --region iad --size 1 -a footydraft
   

4. Set secrets (replace with your values): flyctl secrets set OPENAI_API_KEY="sk-..." TELEGRAM_BOT_TOKEN="123:abc" ... -a footydraft
5. Deploy: flyctl deploy -a footydraft
6. Check logs: flyctl logs -a footydraft

Your bot will now run 24/7 without needing your computer.

---

📊 Deploying the Dashboard to Streamlit Cloud (24/7)

The dashboard can be deployed to Streamlit Cloud for 24/7 access.

1. Push your code to GitHub.
2. Go to Streamlit Cloud and sign in with GitHub.
3. Click "New app" and select your repository.
4. Set the main file path to: dashboard/app.py
5. In Settings → Secrets, add the following environment variables:

DROPBOX_APP_KEY = "your_key"
DRO