Our Methodology
Overview
AI Crypto News uses a multi-stage pipeline to collect, process, and analyze cryptocurrency news. Our system runs continuously, fetching new articles every 5 minutes, classifying them every 10 minutes, and generating entity summaries every 30 minutes.
1. News Collection
We aggregate RSS feeds from 19+ cryptocurrency news publications. Our collector:
- Fetches articles from all sources in parallel
- Deduplicates content using URL matching
- Enriches articles with Open Graph images when missing
- Stores articles in Redis cache for fast retrieval
Update Frequency: Every 5 minutes
2. Sentiment Classification
Each article is analyzed by our AI model (Llama 3.2) to determine market sentiment:
Bullish
News that suggests positive price action, adoption growth, favorable regulations, successful upgrades, or institutional interest.
Bearish
News indicating potential negative impact: hacks, regulatory crackdowns, project failures, market manipulation, or negative macroeconomic factors.
Neutral
Informational content without clear market direction: educational articles, technical updates, or balanced market analysis.
Important
High-impact news regardless of sentiment: major announcements, breaking news, regulatory decisions, or significant market events.
3. Importance Scoring
Our AI assigns importance scores from 1-10 based on potential market impact:
- 1-3 (Low): Minor updates, routine news, educational content
- 4-6 (Medium): Noteworthy developments, partnership announcements, technical milestones
- 7-9 (High): Significant market events, major protocol upgrades, regulatory news
- 10 (Critical): Market-moving events, security incidents, major institutional moves
4. Entity Extraction
We automatically identify and tag entities mentioned in articles:
- Cryptocurrencies: 84+ symbols (BTC, ETH, SOL, etc.) and full names
- ETFs: Bitcoin and Ethereum ETF tickers (IBIT, FBTC, ARKB, etc.)
- Key Terms: airdrop, listing, mainnet, halving, regulation, hack, etc.
- Protocols & Exchanges: Major DeFi protocols and centralized exchanges
5. Entity Summaries
For major cryptocurrencies (BTC, ETH, SOL, XRP, BNB, ADA, DOGE, TRX, XLM, LINK), we generate AI-powered market analysis summaries:
- Paragraph 1: Current developments and main news themes
- Paragraph 2: Market sentiment and trading implications
- Paragraph 3: Upcoming catalysts and market outlook
Update Frequency: Every 30 minutes, based on the latest 10 articles per entity
6. Caching Strategy
We use a multi-layer caching system for optimal performance:
- In-Memory Cache: Fastest access for frequently requested data
- Redis Cache: Persistent storage with configurable TTL
- Stale-While-Revalidate: Serve cached data while refreshing in background
This ensures you always get fast responses while maintaining data freshness.
AI Model Details
We use locally-hosted Ollama with Llama 3.2 models for privacy and speed:
- Classification: Llama 3.2 3B parameter model (higher accuracy)
- Summarization: Llama 3.2 1B parameter model (faster generation)
- Temperature: 0.3 (focused, consistent outputs)
- Processing: All AI inference runs locally - no data sent to external APIs
Limitations
While our AI system strives for accuracy, please be aware of these limitations:
- AI sentiment analysis may misinterpret sarcasm, nuance, or complex narratives
- Importance scores are algorithmic estimates, not guarantees of market impact
- Entity extraction may miss context-dependent references
- News aggregation depends on source RSS feed availability and formatting
- Market conditions can change faster than our update intervals
Always verify important information with original sources and conduct your own research.