Tunnel Traffic Analytics¶

Full description of how visit statistics for tunnel sites (slug.dix.su) are collected, stored, and displayed.

For Users¶

Plan Availability¶

Plan	Analytics	Traffic quota
SIMPLE	No	Usage / limit only
VIP	No	Usage / limit only
PRO	Yes — full detail	+ display
PERS	Yes — full detail	+ display
GLAVA	Yes	Unlimited

Data appears from the moment you upgrade to PRO/PERS — no retroactive history for previous days.

What Is Shown¶

Daily breakdown (Analytics tab, date selector):

Metric	Description
Traffic (bytes)	Inbound and outbound volume for the day
Requests	Total HTTP request count (including bots and static assets)
Unique visitors	Estimated unique people (deduplication by IP+UA)
Top pages	Most visited paths on your tunnel site
Traffic sources	UTM parameters + referrer domains
Geography	Visitors' countries and cities

Monthly summary (Usage tab): total bytes and requests for the current month, no per-day breakdown — available on all plans.

Unique Visitors¶

Uniqueness is determined by SHA-256(daily_salt | IP | User-Agent). The salt rotates daily and is never persisted — meaning the raw IP cannot be reconstructed from the database even with direct access. This is the same approach used by Plausible Analytics.

This is an approximate metric: multiple people behind the same NAT (corporate network, mobile carrier) may be counted as one visitor.

Bot Filtering¶

Request type	Traffic (bytes/requests)	Page visits	Unique	Geo
Real users	✅	✅	✅	✅
Bots (Googlebot, Yandex, Facebook, etc.)	✅	❌	❌	❌
Static assets (css/js/img/fonts)	✅	❌	❌	❌

Bot traffic is included in bytes and request counts — they generate real load and consume the monthly quota.

UTM Tags and Traffic Sources¶

Supported parameters: utm_source, utm_medium, utm_campaign, utm_content, utm_term, ref.

The domain from the Referer header is automatically recorded as referrer_domain — capturing organic visits from search engines and social networks without UTM tags. Internal referrals (same tunnel host) are excluded.

Retention and Export¶

Data is retained for 90 days, then automatically deleted (daily at 00:10 UTC).
Excel export: custom date range up to 365 days back. Includes all 4 metrics (traffic, paths, parameters, geo).

What Is NOT Collected¶

Request/response body content
Authorization headers, cookies, tokens
In E2E mode — nothing at all (traffic is encrypted to the device; the server sees only ciphertext)

For Technical Specialists¶

Architecture: Hot Path via ETS¶

Direct PostgreSQL INSERTs on every request would be a bottleneck at thousands of requests per second. Instead:

HTTP request to slug.dix.su
    ↓
proxy_controller.ex
    ↓
MetricsCollector.record_request/6   ← atomic ETS increment, nanoseconds
    ↓ (every 60 sec)
flush_to_db/0
    ↓
PostgreSQL UPSERT × 4 tables

MetricsCollector is a GenServer with four in-RAM ETS tables:

ETS table	Purpose
`:tunnel_metrics`	Traffic/path/geo/UTM counters for the current flush interval
`:tunnel_unique_fingerprints`	Visitor deduplication — hashes for today and yesterday
`:tunnel_visitor_salt`	Daily salts (only today and yesterday retained)
`:public_role_cache`	User role cache (TTL 10 min) — avoids DB lookup on every request

Flush is best effort: if PostgreSQL is unavailable, up to 60s of accumulated data is lost. The proxy does not degrade.

PostgreSQL Schema¶

Four tables, all with composite primary keys and ON CONFLICT DO UPDATE:

-- Daily traffic summary
tunnel_traffic_daily (
    slug TEXT, date DATE,
    bytes_in BIGINT, bytes_out BIGINT,
    request_count BIGINT, unique_visitors BIGINT,
    PRIMARY KEY (slug, date)
)

-- Top paths per day
tunnel_path_visits_daily (
    slug TEXT, date DATE, path TEXT,
    visit_count BIGINT,
    PRIMARY KEY (slug, date, path)
)

-- UTM parameters and referrer_domain
tunnel_param_visits_daily (
    slug TEXT, date DATE, param_name TEXT, param_value TEXT,
    visit_count BIGINT,
    PRIMARY KEY (slug, date, param_name, param_value)
)

-- Geo visits
tunnel_geo_visits_daily (
    slug TEXT, date DATE, country TEXT, city TEXT,
    visit_count BIGINT,
    PRIMARY KEY (slug, date, country, city)
)

Migrations: V29__Tunnel_analytics.sql, V30__Tunnel_unique_visitors.sql, V31__Tunnel_geo_visits.sql.

Geolocation¶

Country and city are resolved from the IP address via the offline MaxMind GeoLite2 database (DixuProxy.GeoIp). No external API calls in the hot path. If the country cannot be resolved (localhost, VPN, database not loaded), no geo row is written.

Unique Visitors: Implementation¶

# MetricsCollector.ex — deduplication
defp track_unique_visitor(slug, date, ip, user_agent) do
  salt = daily_salt(date)
  fingerprint = :crypto.hash(:sha256, [salt, "|", ip, "|", user_agent])
  if :ets.insert_new(@uniq_table, {{slug, date, fingerprint}, true}) do
    bump({:unique, slug, date}, 1)
  end
end

Salt rotation: on first access for a date, strong_rand_bytes(16) is generated and stored in @salt_table. Fingerprints and salts older than yesterday are cleaned up every 30 flush cycles (~30 min).

API Endpoints¶

All endpoints are authenticated (session-based), accessible only by the tunnel owner.

Method	Path	Minimum plan	Description
GET	`/api/analytics/my?date=YYYY-MM-DD&page=N&size=N`	PRO	Daily analytics with paginated top paths
GET	`/api/analytics/monthly`	All	Cumulative traffic for the current month
GET	`/api/billing/traffic`	All	Quota usage (bytes) with limit and reset date
GET	`/api/analytics/my/export?from=YYYY-MM-DD&to=YYYY-MM-DD`	PRO	Download Excel (up to 365 days)

Sample /api/analytics/my response (PRO+):

{
  "bytesIn": 1234567,
  "bytesOut": 9876543,
  "requestCount": 4200,
  "uniqueVisitors": 37,
  "topPaths": {
    "rows": [{"path": "/", "visitCount": 120}],
    "total": 42
  },
  "topParams": [
    {"paramName": "utm_source", "paramValue": "telegram", "visitCount": 55},
    {"paramName": "referrer_domain", "paramValue": "google.com", "visitCount": 18}
  ],
  "topGeo": [
    {"country": "Russia", "city": "Moscow", "visitCount": 30}
  ]
}

Admin Access¶

Via AdminListsController:

Path	Description
`GET /admin/analytics/list?date=&page=&size=`	Paginated tunnel list with daily metrics
`GET /admin/analytics/top-paths?slug=&date=`	Top paths for a specific tunnel
`GET /admin/analytics/geo?slug=&date=`	Geo statistics for a specific tunnel

Available only to GLAVA-role sessions (Spring Security session).

Data Cleanup¶

AnalyticsCleanupScheduler — Spring @Scheduled(cron = "0 10 0 * * *"):

DELETE FROM tunnel_traffic_daily     WHERE date < CURRENT_DATE - INTERVAL '90 days';
DELETE FROM tunnel_path_visits_daily WHERE date < CURRENT_DATE - INTERVAL '90 days';
DELETE FROM tunnel_param_visits_daily WHERE date < CURRENT_DATE - INTERVAL '90 days';
DELETE FROM tunnel_geo_visits_daily  WHERE date < CURRENT_DATE - INTERVAL '90 days';

E2E Tunnels and Analytics¶

For E2E tunnels (e2e-<token>.dix.su), MetricsCollector.record_request is not called — the proxy acts as a blind TCP relay. Instead, e2e_traffic_counter.ex records bytes only (in/out) in a separate table for billing purposes. No paths, geo, or UTM for E2E.

Traffic Limits and Warnings¶

MetricsCollector.check_public_limit/1 is called before every tunnel request:

Sums ETS counters for the current month + cached DB data (TTL 5 min)
At ≥ 80% quota: sends an email warning (at most once per day)
At ≥ 100%: freezes the tunnel until the 1st of the next month

Scaling (Future)¶

Plan from SCALING.md:

Read replica for AnalyticsController — analytics SELECTs skip the master
Table partitioning for tunnel_path_visits_daily / tunnel_geo_visits_daily by date — cheap DROP PARTITION instead of DELETE
Event streaming (Kafka/Kinesis) — replace direct UPSERTs in dixu_proxy, buffer for traffic spikes