Page MenuHomePhabricator
Paste P13393

eet.py
ActivePublic
Actions

Authored by CDanis on Nov 24 2020, 3:16 PM.
Tags
None
Referenced Files
F34428884: eet.py
Apr 26 2021, 7:27 PM
F33924038: eet.py
Nov 24 2020, 3:44 PM
F33924027: eet.py
Nov 24 2020, 3:27 PM
F33924016: eet.py
Nov 24 2020, 3:16 PM
Subscribers
None
#!/usr/bin/env python3
"""eet: tee in reverse
Consumes from a set of input files in a line-buffered way and emits their
combination on stdout.
Inputs are handled non-blockingly, so they may be pipes.
This is most useful with shell process substitution, for example:
eet.py <(kafkacat ...) <(kafkacat ...)
"""
__author__ = 'Chris Danis'
__version__ = '0.0.1'
__copyright__ = """
Copyright © 2020 Chris Danis & the Wikimedia Foundation
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file
except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the
License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND,
either express or implied. See the License for the specific language governing permissions
and limitations under the License.
"""
import argparse
import select
import os
import sys
from collections import defaultdict
parser = argparse.ArgumentParser()
parser.add_argument('files', nargs='+', type=argparse.FileType('rb', 0))
args = parser.parse_args()
for f in args.files:
os.set_blocking(f.fileno(), False)
opened_files = set(args.files)
bufs = defaultdict(bytes)
while opened_files:
readable, _, _ = select.select(opened_files, [], [], 1)
for f in readable:
fno = f.fileno()
b = f.read()
if not b:
opened_files.remove(f)
if b.endswith(b'\n'):
if bufs[fno]:
sys.stdout.buffer.write(bufs[fno])
del bufs[fno]
sys.stdout.buffer.write(b)
else:
bufs[fno] += b
s = bufs[fno].rsplit(b'\n', maxsplit=1)
if len(s) == 2:
sys.stdout.buffer.write(s[0])
sys.stdout.buffer.write(b'\n')
bufs[fno] = s[1]
Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL