Page MenuHomePhabricator
Paste P37702

json-webrequests-stats example output
ActivePublic

Authored by Volans on Nov 2 2022, 12:30 PM.
Tags
None
Referenced Files
F35696575: json-webrequests-stats example output
Nov 2 2022, 1:36 PM
F35696301: json-logs-stats example output
Nov 2 2022, 12:30 PM
Subscribers
None
$ tail -n 100000 /srv/weblog/webrequest/sampled-1000.json | json-webrequests-stats -n 3
= Stats from 100000 records between 2022-11-02T10:53:44Z and 2022-11-02T11:05:40Z
=================================================================================
= Stats for upload CDN from 31844 records
-> top N requests per cache_status
----------
21488 hit-front
5347 hit-local
3225 int-front
-> top N requests per http_status
----------
27792 200
2816 429
564 304
-> top N requests per hostname
----------
1606 cp3059.esams.wmnet
1593 cp3061.esams.wmnet
1586 cp3063.esams.wmnet
-> top N requests per ip
----------
273 XXX.XXX.XXX.XXX
121 YYY.YYY.YYY.YYY
91 ZZZ.ZZZ.ZZZ.ZZZ
-> top N requests per uri_host
----------
31844 upload.wikimedia.org
-> top N requests per uri_path
----------
107 /wikipedia/en/thumb/f/ff/BBC_News.svg/1200px-BBC_News.svg.png.webp
86 /wikipedia/commons/4/4d/Icon_pdf_file.png
63 /wikipedia/commons/d/d4/Button_hide.png
-> top N requests per uri_query
----------
31770
3 ?resize=100x100
2 ?download
-> top N requests per referer
----------
6976 https://en.wikipedia.org/
5124 -
2614 https://en.m.wikipedia.org/
-> top N requests per user_agent
----------
4651 Mozilla/5.0 [...SNIP...] Chrome/106.0.0.0 [...SNIP...]
2648 Java/11.0.16
2357 Mozilla/5.0 [...SNIP...] Chrome/107.0.0.0 [...SNIP...]
-> top N sum(response_size) per ip [MBs]
----------
40.00 XXX.XXX.XXX.XXX
22.10 YYY.YYY.YYY.YYY
21.10 ZZZ.ZZZ.ZZZ.ZZZ
-> top N sum(time_firstbyte) per ip [seconds]
----------
17.33 XXX.XXX.XXX.XXX
10.13 YYY.YYY.YYY.YYY
5.23 ZZZ.ZZZ.ZZZ.ZZZ
-> top N sum(response_size) per uri_host [MBs]
----------
1763.82 upload.wikimedia.org
-> top N sum(time_firstbyte) per uri_host [seconds]
----------
472.34 upload.wikimedia.org
-> top N sum(response_size) per uri_path [MBs]
----------
40.00 /wikipedia/commons/e/e1/EH1357844_Christ_Church_06.jpg
22.10 /wikipedia/commons/8/85/Illustrirtes_Konversations-Lexikon_Bd._5_1876_%28131652903%29.jpg
20.98 /wikipedia/commons/9/93/Treborth_Street%2C_Liverpool.jpg
-> top N sum(time_firstbyte) per uri_path [seconds]
----------
10.13 /wikipedia/commons/thumb/a/a6/Nokia_3100_blue_front_(cutout_transparent_bg).png
5.76 /wikipedia/commons/thumb/6/6e/Met_DP-17302-014.jpg/1280px-Met_DP-17302-014.jpg
5.23 /wikipedia/commons/thumb/d/d1/Pool_and_trees_%283017057306%29.jpg/360px-Pool_and_trees_%283017057306%29.jpg
-> top N sum(response_size) per referer [MBs]
----------
632.94 -
422.39 https://www.google.com/
172.45 https://en.wikipedia.org/
-> top N sum(time_firstbyte) per referer [seconds]
----------
101.15 -
74.08 https://en.wikipedia.org/
53.62 https://commons.wikimedia.org/
-> top N sum(response_size) per user_agent [MBs]
----------
401.99 LicenseBot/1.0 (https://amazon.com/) license-bot/1.0
170.99 Mozilla/5.0 [...SNIP...] Chrome/106.0.0.0 [...SNIP...]
79.41 Mozilla/5.0 [...SNIP...] Chrome/107.0.0.0 [...SNIP...]
-> top N sum(time_firstbyte) per user_agent [seconds]
----------
65.98 Mozilla/5.0 [...SNIP...] Chrome/106.0.0.0 [...SNIP...]
40.09 Mozilla/5.0 [...SNIP...] Chrome/107.0.0.0 [...SNIP...]
24.95 Mozilla/5.0 [...SNIP...] Firefox/106.0
=================================================================================
= Stats for text CDN from 68156 records
-> top N requests per cache_status
----------
46899 hit-front
8955 pass
6550 int-front
-> top N requests per http_status
----------
50821 200
5918 304
5522 301
-> top N requests per hostname
----------
3796 cp3052.esams.wmnet
3721 cp3062.esams.wmnet
3663 cp3058.esams.wmnet
-> top N requests per ip
----------
561 XXX.XXX.XXX.XXX
284 YYY.YYY.YYY.YYY
247 ZZZ.ZZZ.ZZZ.ZZZ
-> top N requests per uri_host
----------
20500 en.wikipedia.org
7094 en.m.wikipedia.org
3522 wikimedia.org
-> top N requests per uri_path
----------
20162 /w/load.php
8480 /w/api.php
4909 /
-> top N requests per uri_query
----------
28885
2526 ?hasty=true
1149 ?action=webapp-manifest
-> top N requests per referer
----------
18951 -
1667 https://en.wikipedia.org/
1489 https://www.google.com/
-> top N requests per user_agent
----------
6846 Mozilla/5.0 [...SNIP...] Chrome/106.0.0.0 [...SNIP...]
3524 Mozilla/5.0 [...SNIP...] Chrome/107.0.0.0 [...SNIP...]
2669 Mozilla/5.0 [...SNIP...] Chrome/107.0.0.0 [...SNIP...]
-> top N sum(response_size) per ip [MBs]
----------
14.52 XXX.XXX.XXX.XXX
10.15 YYY.YYY.YYY.YYY
7.39 ZZZ.ZZZ.ZZZ.ZZZ
-> top N sum(time_firstbyte) per ip [seconds]
----------
16.51 XXX.XXX.XXX.XXX
16.47 YYY.YYY.YYY.YYY
11.13 ZZZ.ZZZ.ZZZ.ZZZ
-> top N sum(response_size) per uri_host [MBs]
----------
142.76 en.wikipedia.org
111.09 en.m.wikipedia.org
23.61 ja.m.wikipedia.org
-> top N sum(time_firstbyte) per uri_host [seconds]
----------
384.84 en.wikipedia.org
311.18 intake-analytics.wikimedia.org
164.17 commons.wikimedia.org
-> top N sum(response_size) per uri_path [MBs]
----------
271.92 /w/load.php
23.18 /w/index.php
20.61 /w/api.php
-> top N sum(time_firstbyte) per uri_path [seconds]
----------
755.44 /w/api.php
318.34 /v1/events
181.76 /w/index.php
-> top N sum(response_size) per referer [MBs]
----------
201.73 -
47.39 https://www.google.com/
3.57 https://www.wikidata.org/w/index.php?title=Q30&action=history
-> top N sum(time_firstbyte) per referer [seconds]
----------
1029.94 -
87.85 https://en.wikipedia.org/
52.62 https://www.google.com/
-> top N sum(response_size) per user_agent [MBs]
----------
54.95 Mozilla/5.0 [...SNIP...] Chrome/106.0.0.0 [...SNIP...]
29.11 Mozilla/5.0 [...SNIP...] Chrome/107.0.0.0 [...SNIP...]
25.07 curl/7.79.1
-> top N sum(time_firstbyte) per user_agent [seconds]
----------
195.31 Mozilla/5.0 [...SNIP...] Chrome/106.0.0.0 [...SNIP...]
91.68 Mozilla/5.0 [...SNIP...] Chrome/107.0.0.0 [...SNIP...]
77.84 Mozilla/5.0 [...SNIP...] Firefox/106.0
= Stats from 100000 records between 2022-11-02T10:53:44Z and 2022-11-02T11:05:40Z

Event Timeline

Volans updated the paste's language from shell to text.
Volans changed the visibility from "Custom Policy" to "Public (No Login Required)".Nov 2 2022, 12:49 PM
Volans removed a project: WMF-NDA.
Volans changed the title of this paste from json-logs-stats example output to json-webrequests-stats example output.Nov 2 2022, 1:36 PM
Volans edited the content of this paste. (Show Details)