new_replace_algorithm.patch
Benjavalero (Benjamín Valero)
Actions

Authored By

	Benjavalero
	Dec 19 2014, 11:53 PM

Size

2 KB

Referenced Files

None

Subscribers

None

new_replace_algorithm.patch
View Options

	From a026a5f851a48044f7c9c342c0788b12db7bff32 Mon Sep 17 00:00:00 2001
	From: =?UTF-8?q?Benjam=C3=ADn=20Valero=20Espinosa?= <benjavalero@gmail.com>
	Date: Sat, 20 Dec 2014 00:31:17 +0100
	Subject: [PATCH] New algorithm to replace text with exceptions. For long text
	pages, it can run several times faster than the old one.

	---
	pywikibot/textlib.py \| 30 ++++++++++++++++--------------
	1 file changed, 16 insertions(+), 14 deletions(-)

	diff --git a/pywikibot/textlib.py b/pywikibot/textlib.py
	index d37f654..5961b3d 100644
	--- a/pywikibot/textlib.py
	+++ b/pywikibot/textlib.py
	@@ -216,26 +216,28 @@ def replaceExcept(text, old, new, exceptions, caseInsensitive=False,
	inside[count] = item
	index = 0
	markerpos = len(text)
	+
	+ # Pre-calculate all the exception matches
	+ exceptionMatches = []
	+ for dontTouchR in dontTouchRegexes:
	+ for exceptionMatch in dontTouchR.finditer(text):
	+ exceptionMatches.append(exceptionMatch)
	+
	while True:
	match = old.search(text, index)
	if not match:
	# nothing left to replace
	break

	- # check which exception will occur next.
	- nextExceptionMatch = None
	- for dontTouchR in dontTouchRegexes:
	- excMatch = dontTouchR.search(text, index)
	- if excMatch and (
	- nextExceptionMatch is None or
	- excMatch.start() < nextExceptionMatch.start()):
	- nextExceptionMatch = excMatch
	-
	- if nextExceptionMatch is not None \
	- and nextExceptionMatch.start() <= match.start():
	- # an HTML comment or text in nowiki tags stands before the next
	- # valid match. Skip.
	- index = nextExceptionMatch.end()
	+ # Check if the match is included in any exception
	+ matchInException = False
	+ for excMatch in exceptionMatches:
	+ if (excMatch.start() <= match.start() and excMatch.end() >= match.end()):
	+ matchInException = True
	+ break
	+
	+ if matchInException:
	+ index = match.end()
	else:
	# We found a valid match. Replace it.
	if callable(new):
	--
	1.8.5.2 (Apple Git-48)

File Metadata

Mime Type: text/x-diff
Storage Engine: blob
Storage Format: Raw Data
Storage Handle: 21569
Default Alt Text: new_replace_algorithm.patch (2 KB)

new_replace_algorithm.patchBenjavalero (Benjamín Valero)Actions

new_replace_algorithm.patchView Options

File Metadata

Event Timeline

new_replace_algorithm.patch
Benjavalero (Benjamín Valero)
Actions

new_replace_algorithm.patch
View Options