Skip to main content

Audit Logs

Track all activity and changes within your organization for compliance and security.

Overview​

Audit logs provide a complete record of:

  • Query executions
  • Settings modifications
  • Team member actions
  • API key changes
  • Data exports

Feature Status​

⚠️ Backend Complete - UI Pending

The audit logging infrastructure is fully implemented and tracking all user actions. However, the UI for viewing and filtering audit logs is not yet available. You can access audit logs via the API or database queries.

Planned Capabilities​

Activity Tracking​

All actions will be logged:

Query Activity

  • Who executed queries
  • When queries ran
  • Which providers used
  • Success/failure status
  • Cost per execution

Settings Changes

  • API key additions/deletions
  • Budget modifications
  • Organization profile updates
  • Member permission changes

Access Events

  • User logins
  • Failed login attempts
  • Password changes
  • API access attempts

Audit Log Viewer​

Dedicated audit log interface with:

Filtering

  • By user/team member
  • By action type
  • By date range
  • By resource (e.g., specific API key)

Search

  • Full-text search across logs
  • Filter by specific events
  • Complex query builder

Export

  • CSV export for compliance
  • JSON export for analysis
  • PDF reports for stakeholders

Retention & Compliance​

Data Retention

  • Configurable retention period
  • Minimum 90 days
  • Option for unlimited retention
  • Automatic archival

Compliance Features

  • SOC 2 audit trail
  • GDPR compliance (right to access)
  • HIPAA audit logging (if applicable)
  • Tamper-proof logs

Current Workarounds​

While the feature is in development, you can track activity manually:

Database Queries​

Query Supabase directly for activity:

-- Recent query executions
SELECT
  u.email as user_email,
  q.query_text,
  q.provider,
  q.created_at,
  r.total_cost
FROM llm_queries q
LEFT JOIN llm_responses r ON r.query_id = q.id
LEFT JOIN auth.users u ON u.id = q.user_id
ORDER BY q.created_at DESC
LIMIT 100;
-- API key changes (if tracked)
SELECT
  action,
  provider,
  created_at,
  created_by
FROM audit_logs
WHERE resource_type = 'api_key'
ORDER BY created_at DESC;

Provider Dashboards​

Check LLM provider dashboards for:

  • API key usage
  • Request counts
  • Cost tracking
  • Error logs

Anthropic: console.anthropic.com → Usage OpenAI: platform.openai.com → Usage Google: aistudio.google.com → Activity Perplexity: docs.perplexity.ai → Usage

Manual Logging​

For critical actions, maintain a manual log:

DateUserActionDetails
2025-01-15admin@company.comAdded API keyOpenAI GPT-4
2025-01-14team@company.comExecuted 50 queriesClaude Sonnet
2025-01-12admin@company.comUpdated budget$500/mo

Use Cases​

Security Auditing​

Goal: Detect unauthorized access or suspicious activity

Current Approach:

  1. Review Supabase auth logs
  2. Check query history for unusual patterns
  3. Monitor provider dashboards for unexpected usage
  4. Set up email alerts for budget overages

Compliance Reporting​

Goal: Demonstrate proper access controls and data handling

Current Approach:

  1. Export query history from Supabase
  2. Document team member access levels
  3. Screenshot API key management
  4. Maintain manual change log for critical actions

Cost Attribution​

Goal: Track which users/projects are incurring costs

Current Approach:

-- Cost by user
SELECT
  u.email,
  SUM(r.total_cost) as total_cost,
  COUNT(*) as query_count
FROM llm_responses r
JOIN llm_queries q ON q.id = r.query_id
LEFT JOIN auth.users u ON u.id = q.user_id
GROUP BY u.email
ORDER BY total_cost DESC;

Future Development​

Full audit logging will include:

Event Types​

Comprehensive event tracking:

Authentication

  • ✅ User login
  • ✅ User logout
  • ✅ Password reset
  • ✅ MFA enabled/disabled
  • ✅ API token created

Authorization

  • ✅ Permission granted
  • ✅ Permission revoked
  • ✅ Role changed
  • ✅ Access denied

Data Access

  • ✅ Query executed
  • ✅ Report generated
  • ✅ Data exported
  • ✅ Settings viewed

Configuration

  • ✅ API key added
  • ✅ API key deleted
  • ✅ Budget changed
  • ✅ Integration configured

Advanced Features​

Real-time Alerts

  • Notify on suspicious activity
  • Alert on policy violations
  • Budget threshold warnings
  • Failed access attempts

Anomaly Detection

  • Unusual query volumes
  • Unexpected providers used
  • Off-hours activity
  • Geographic anomalies

Compliance Reports

  • Pre-built compliance templates
  • Scheduled report delivery
  • Automated attestation
  • Evidence collection

Security & Privacy​

Data Protection​

Audit logs will be:

  • Encrypted at rest: AES-256 encryption
  • Immutable: Cannot be modified after creation
  • Tamper-proof: Cryptographic signatures
  • Backed up: Regular automated backups

Access Control​

Who can view audit logs:

  • Admins: Full access to all logs
  • Members: Can view own activity only
  • Viewers: No access to audit logs
  • Auditors: Read-only access (future role)

Privacy Considerations​

Audit logs include:

  • ✅ User identifiers (email, ID)
  • ✅ Action timestamps
  • ✅ Resource affected
  • ✅ IP addresses (optional)
  • ❌ Query content (unless explicitly enabled)
  • ❌ Response content (privacy)

Best Practices​

Review Regularly​

Schedule periodic audit reviews:

  • Weekly: Check recent high-cost actions
  • Monthly: Full activity review
  • Quarterly: Compliance audit
  • Annually: Comprehensive security review

Set Up Alerts​

Configure notifications for:

  • Failed login attempts (3+ in 1 hour)
  • API key changes
  • Budget exceeded
  • Large batch executions (100+ queries)
  • Off-hours access

Document Policies​

Maintain written policies for:

  • Who can access audit logs
  • How long logs are retained
  • What triggers an investigation
  • Incident response procedures

Export for Compliance​

Regular exports for compliance:

  • Monthly cost reports
  • Quarterly access reviews
  • Annual compliance reports
  • Incident investigations

Contributing​

Want to help build audit logging?

  1. Share your compliance requirements
  2. Contribute sample queries
  3. Review privacy/security approach
  4. Test beta features when available

Next Steps​

While the feature is in development:

  1. Set up manual logging for critical actions
  2. Configure provider dashboard alerts
  3. Document your compliance requirements
  4. Query Supabase directly for activity reports