Page Menu
Home
Phabricator
Search
Configure Global Search
Log In
Files
F34486213
Script to get word count of all wikis from Toolforge CirrusSearch elasticsearch replicas
No One
Actions
View File
Edit File
Delete File
View Transforms
Subscribe
Mute Notifications
Award Token
Flag For Later
Authored By
LucasWerkmeister
Jun 7 2021, 9:21 PM
2021-06-07 21:21:49 (UTC+0)
Size
943 B
Referenced Files
None
Subscribers
None
Script to get word count of all wikis from Toolforge CirrusSearch elasticsearch replicas
View Options
#!/usr/bin/env bash
for
port in
8243
8443
8643
;
do
curl -s https://cloudelastic.wikimedia.org:
"
$port
"
/_aliases
|
jq -r
'
del(.[".ltrstore"]) |
del(.[".tasks"]) |
to_entries |
.[] |
.value.aliases |
keys |
.[1]
'
|
grep -F _content
|
while
IFS
=
read
-r index
;
do
words
=
$(
curl
\
-s
\
-XGET
\
-H
'Content-Type: application/json'
\
-d
'{"query":{"bool":{"filter":[{"terms":{"namespace":[0]}}]}},"aggs":{"word_count":{"sum":{"field":"text.word_count"}}},"stats":["sum_word_count"]}'
\
https://cloudelastic.wikimedia.org:
"
$port
"
/
"
$index
"
/_search
|
jq -r .aggregations.word_count.value
)
;
printf
'%25s %15d\n'
"
${
index
%_content
}
"
"
$words
"
;
done
;
done
File Metadata
Details
Attached
Mime Type
text/plain; charset=utf-8
Storage Engine
blob
Storage Format
Raw Data
Storage Handle
9093529
Default Alt Text
Script to get word count of all wikis from Toolforge CirrusSearch elasticsearch replicas (943 B)
Attached To
Mode
P16317 Script to get word count of all wikis from Toolforge CirrusSearch elasticsearch replicas
Attached
Detach File
Event Timeline
Log In to Comment