Page MenuHomePhabricator

Get puppet runs into logstash
Closed, ResolvedPublic

Details

Other Assignee
jbond
Reference
fl4
ReferenceSource BranchDest BranchAuthorTitle
repos/cloud/toolforge/toolforge-deploy!203bump_maintain-kubeusersmainproject_1317_bot_df3177307bed93c3f34e421e26c86e38maintain-kubeusers: bump to 0.0.122-20240221143623-84a40909
repos/cloud/toolforge/maintain-kubeusers!14fix_permissionsmaindcarouser: fix bad permission for skel-copied files
repos/cloud/toolforge/toolforge-deploy!201bump_maintain-kubeusersmainproject_1317_bot_df3177307bed93c3f34e421e26c86e38maintain-kubeusers: bump to 0.0.121-20240221131329-d6f9cd11
repos/releng/scap!217master-I7240ab57c5461e5805562bfcf42a29b2d3680b83masterdancycanary_checks: Improve format of Waiting for canary traffic message
repos/cloud/toolforge/maintain-kubeusers!13copy_skelmaindcarouser: copy stuff from /etc/skel if there's anything
repos/releng/scap!215master-I3745ed1b83962bc72c9e0874060cc9efbf61dea4masterdancyMention how long we're waiting for canary traffic
repos/phabricator/deployment!32T165192verboseProjCreationPermErrorwmf/stableaklapperProvide link when user without permissions tries to create project
repos/mediawiki/services/mathoid!3T137787mainphysikerweltUpdate mathjax-node and texvcjs
repos/releng/scap!130master-c5d7masterdancyCall git submodule sync before git submodule update
repos/phabricator/deployment!6dannys712-wmf/stable-patch-89495wmf/stabledannys712Phabricator: add override for the browser time zone conflict message
repos/releng/release!10tarball-echo-plusmasterjforresterAdd Echo, LoginNotfiy, and Thanks to the default MediaWiki bundle
Show related patches Customize query in GitLab

Revisions and Commits

Related Objects

StatusSubtypeAssignedTask
OpenFeatureNone
Resolvedbd808

Event Timeline

There are a very large number of changes, so older changes are hidden. Show Older Changes
emailbot mentioned this in Unknown Object (Task).Dec 18 2015, 6:37 PM
This comment was removed by Legoktm.

Change 274054 had a related patch set uploaded (by Xqt):
Remove imagetransfer.py from T001/T003 testing (print function)

https://gerrit.wikimedia.org/r/274054

This comment was removed by Legoktm.

Change 291859 had a related patch set uploaded (by Xqt):
Remove casechecker .py from "T001/T003" testing (print function)

https://gerrit.wikimedia.org/r/291859

Change 274054 abandoned by Xqt:
Remove casechecker .py from T001/T003 testing (print function)

https://gerrit.wikimedia.org/r/274054

Change 303408 had a related patch set uploaded (by Xqt):
Remove casechecker.py from T001/T003 testing (print function)

https://gerrit.wikimedia.org/r/303408

Change 291859 abandoned by Xqt:
Remove casechecker .py from "T001/T003" testing (print function)

Reason:
duplicate of I1d23ab55e

https://gerrit.wikimedia.org/r/291859

Change 303408 merged by jenkins-bot:
Remove casechecker.py from ignoring T001/T003 (print function)

https://gerrit.wikimedia.org/r/303408

Change 274054 restored by Xqt:
Remove casechecker .py from T001/T003 testing (print function)

https://gerrit.wikimedia.org/r/274054

Change 308753 had a related patch set (by Paladox) published:
Add support for searching gerrit using bug:T1

https://gerrit.wikimedia.org/r/308753

Change 308753 merged by Dzahn:
Add support for searching gerrit using bug:T1

https://gerrit.wikimedia.org/r/308753

Change 330972 had a related patch set (by Paladox) published:
Test: Do not merge

https://gerrit.wikimedia.org/r/330972

Change 330972 abandoned by Paladox:
Test: Do not merge

https://gerrit.wikimedia.org/r/330972

Change 274054 merged by jenkins-bot:
Remove imagetransfer.py from T001/T003 testing

https://gerrit.wikimedia.org/r/274054

Change 407865 had a related patch set uploaded (by Paladox; owner: Paladox):
[All-Projects@refs/meta/config] Modify access rules+test+test

https://gerrit.wikimedia.org/r/407865

Change 434719 had a related patch set uploaded (by Herron; owner: Herron):
[operations/puppet@production] puppet-agent: remove --show_diff from scheduled puppet-run script

https://gerrit.wikimedia.org/r/434719

This is strange. Even the first task is still unresolved. Can someone please guide me how we go about in the phabricator about backlog clearing?

Aklapper changed the task status from Stalled to Open.Nov 3 2020, 10:38 AM
In T1#1541803, @bd808 wrote:

We have the technology to implement this, but WMF TechOps has valid concerns about sensitive data that may be contained in the puppet reports themselves. As one example, file diffs can show passwords that are generally protected via file permissions from reading by all users with shell access to the host.

Five years later, is that still the take / situation? (If it is, feel free to update the task status again.)

Change 734961 had a related patch set uploaded (by Jbond; author: John Bond):

[operations/puppet@production] puppetmaster: enable logstash reports

https://gerrit.wikimedia.org/r/734961

Change 734961 merged by Jbond:

[operations/puppet@production] puppetmaster: enable logstash reports

https://gerrit.wikimedia.org/r/734961

Change 736233 had a related patch set uploaded (by Jbond; author: jbond):

[operations/puppet@production] P:rsyslog: ship puppetmaster logs to kafka

https://gerrit.wikimedia.org/r/736233

Change 736233 merged by Jbond:

[operations/puppet@production] P:rsyslog: ship puppetmaster logs to kafka

https://gerrit.wikimedia.org/r/736233

jbond closed this task as Resolved.EditedNov 4 2021, 8:51 PM
jbond assigned this task to bd808.
jbond updated Other Assignee, added: jbond.

after the last change above, we now have puppet report data going into logstash. I haven't played with it much and im a elastic noob but the following is an example showing when a hosts where running a specific puppet git sha1 https://logstash.wikimedia.org/goto/473ad9da22eb1255d4168658f42a7ed5

As said i have not played with the data much but here is an example of what data gets sent on each run.

In T1#6599767, @Volans wrote:

@Aklapper TL;DR yes, as it depends on T213902

i think this comment referees to puppet diff's however the implementation doesn't add the diffs and im not sure if we would want them. As such i think we can finally close this task down and will boldly do so

Change 875401 had a related patch set uploaded (by Jbond; author: John Bond):

[operations/puppet@production] phabricator: update pattern to support old tickets like T1

https://gerrit.wikimedia.org/r/875401

Change 875401 merged by Jbond:

[operations/puppet@production] phabricator: update pattern to support old tickets like T1

https://gerrit.wikimedia.org/r/875401

This comment was removed by brennen.
This comment was removed by brennen.