UFO Research Automation System Setup Guide
UFO Research Automation System Setup Guide
Overview
This system provides automated collection of UFO/UAP research from multiple sources and integrates with Bluesky for social media posting. The bot continuously monitors news, academic papers, MUFON cases, and social discussions.
Features
- Automated Data Collection: 8+ news sources, academic papers, MUFON cases
- Social Media Integration: Bluesky posting with AT Protocol
- Intelligent Content Generation: Daily summaries, breaking news alerts, research highlights
- Database Management: SQLite database for all collected research
- Scheduling System: Automated posting schedule with customizable timing
- Credibility Assessment: Source reliability scoring system
- Daily Summary: 9:00 AM - Research statistics and top stories
- Data Collection: Every 4 hours - Collects new articles, papers, cases
- Breaking News: Hourly - High-credibility alerts
- Weekly Analysis: Sundays 6:00 PM - Trend analysis and patterns
- Research Highlights: As discovered - Important papers and findings
Installation
1. Prerequisites
```bash
Python 3.8+ required
python --version
Install pip if not available
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
python get-pip.py
```
2. Install Dependencies
```bash
Navigate to UFO files directory
cd /Users/jeffmiddleton/Desktop/ufofiles
Install required packages
pip install -r requirements.txt
For enhanced NLP features (optional)
python -m spacy download en_core_web_sm
```
3. Database Setup
```bash
The database will be created automatically on first run
Location: ufo_research.db in the same directory
```
Configuration
1. Bluesky Account Setup
```python
In BLUESKY_UFO_BOT_CONFIG.py, update these variables:
BLUESKY_HANDLE = "your_handle.bsky.social"
BLUESKY_PASSWORD = "your_app_password" # Generate app password in Bluesky settings
```
2. Environment Variables (Recommended)
Create a `.env` file:
```bash
.env file
BLUESKY_HANDLE=your_handle.bsky.social
BLUESKY_PASSWORD=your_app_password
DATABASE_PATH=ufo_research.db
LOG_LEVEL=INFO
```
3. Customization Options
```python
In UFO_RESEARCH_AUTOMATION_SYSTEM.py
Modify news sources
self.news_sources = {
"Your Custom Source": "https://example.com/rss",
# Add or remove sources as needed
}
Adjust credibility scores
def _assess_source_credibility(self, source: str) -> int:
credibility_scores = {
"Your Source": 8, # 1-10 scale
}
Modify posting schedule in BLUESKY_UFO_BOT_CONFIG.py
schedule.every().day.at("09:00").do(self.post_daily_summary) # Change time
schedule.every(2).hours.do(self.run_collection_cycle) # Change frequency
```
Usage
1. Manual Testing
```bash
Test data collection only
python UFO_RESEARCH_AUTOMATION_SYSTEM.py
Test Bluesky authentication
python -c "from BLUESKY_UFO_BOT_CONFIG import BlueSkyATProto; bot = BlueSkyATProto('handle', 'password'); print(bot.create_session())"
```
2. Run Full Bot
```bash
Start automated bot (runs continuously)
python BLUESKY_UFO_BOT_CONFIG.py
```
3. Background Operation
```bash
Run as background process
nohup python BLUESKY_UFO_BOT_CONFIG.py > bot.log 2>&1 &
Check if running
ps aux | grep python
Stop background process
kill [process_id]
```
Bot Behavior
Posting Schedule
Content Types
```
🛸 DAILY UFO RESEARCH DIGEST
📊 15 articles from 8 sources
⭐ Avg credibility: 7.2/10
🔥 TOP STORY: Pentagon releases new UAP data...
#UFO #UAP #Research
🚨 BREAKING UFO NEWS
New military encounter documented...
Source: The Debrief
Credibility: 9/10
#Breaking #UFO #UAP
📚 NEW UFO RESEARCH
Advanced Propulsion Systems Analysis...
Authors: Dr. Smith, Dr. Jones
Abstract: This paper examines...
#Research #Science #UFO
```
Database Structure
```sql
-- News articles with credibility scoring
news_articles (id, title, url, source, published_date, summary, credibility_score, tags)
-- Academic research papers
research_papers (id, title, authors, abstract, url, source, published_date, tags)
-- MUFON case reports
mufon_cases (id, case_number, date_occurred, location, shape, duration, summary)
-- Social media discussions
social_posts (id, platform, author, title, content, url, score)
-- Bot posting history
bluesky_posts (id, content, source_type, posted_at, engagement_score)
```
Monitoring and Maintenance
1. Log Files
```bash
View current logs
tail -f ufo_bot.log
Search for errors
grep ERROR ufo_bot.log
Monitor database growth
sqlite3 ufo_research.db "SELECT COUNT(*) FROM news_articles;"
```
2. Database Management
```bash
Backup database
cp ufo_research.db ufo_research_backup_$(date +%Y%m%d).db
Check database size
du -h ufo_research.db
Clean old data (optional)
sqlite3 ufo_research.db "DELETE FROM news_articles WHERE created_at < datetime('now', '-30 days');"
```
3. Performance Tuning
```python
Adjust collection limits in UFO_RESEARCH_AUTOMATION_SYSTEM.py
for entry in feed.entries[:10]: # Reduce from 10 to 5 for faster collection
Modify rate limiting
time.sleep(2) # Increase delay between requests if needed
Adjust database cleanup frequency
Add to scheduler: weekly database maintenance
```
Troubleshooting
Common Issues
#### Authentication Errors
```bash
Error: Bluesky authentication failed
Solution: Check app password, ensure it's not your main password
Generate app password in Bluesky settings > App Passwords
```
#### Rate Limiting
```bash
Error: Too many requests
Solution: Increase delays between requests
time.sleep(5) # Increase from 2 seconds
```
#### Database Locked
```bash
Error: Database is locked
Solution: Ensure only one bot instance running
ps aux | grep python # Check for multiple instances
```
#### Memory Usage
```bash
Monitor memory usage
top -p $(pgrep -f UFO_RESEARCH)
Reduce memory usage by limiting collection size
for entry in feed.entries[:5]: # Reduce collection size
```
Debug Mode
```python
Enable debug logging
logging.basicConfig(level=logging.DEBUG)
Test individual components
collector = UFOResearchCollector()
articles = collector.collect_news_articles()
print(f"Collected {len(articles)} articles")
```
Advanced Features
1. Custom Content Filters
```python
Add keyword filtering
def _filter_relevant_content(self, text: str) -> bool:
keywords = ['ufo', 'uap', 'disclosure', 'pentagon', 'military']
return any(keyword in text.lower() for keyword in keywords)
```
2. Enhanced Analytics
```python
Add trending topic analysis
def analyze_trending_topics(self) -> Dict:
# Analyze tag frequency over time
# Identify emerging topics
# Generate trend reports
```
3. Multi-Platform Support
```python
Add Twitter/X integration
Add Mastodon integration
Add Discord webhook integration
```
4. API Integrations
```python
Add direct MUFON API integration
Add academic database APIs
Add government data feeds
```
Security Considerations
1. API Keys and Passwords
```bash
Never commit credentials to version control
echo ".env" >> .gitignore
echo "*.log" >> .gitignore
Use environment variables
export BLUESKY_PASSWORD="your_secure_password"
```
2. Rate Limiting
```python
Respect server rate limits
time.sleep(2) # Minimum delay between requests
```
3. Error Handling
```python
Graceful error handling
try:
result = risky_operation()
except Exception as e:
logger.error(f"Operation failed: {e}")
continue # Don't crash, continue processing
```
Scaling and Extension
1. Multiple Bots
```python
Run specialized bots for different topics
UFO news bot
Academic research bot
Breaking news alert bot
```
2. Data Export
```python
Export research data
def export_database_to_json():
# Convert SQLite to JSON for analysis
# Generate research reports
# Create data visualizations
```
3. Web Interface
```python
Add Flask web interface
Dashboard for bot status
Manual posting interface
Research database browser
```
This automation system provides a comprehensive foundation for UFO research collection and social media engagement. The modular design allows for easy customization and extension based on your specific research needs.