conrad.irwin wrote:

I suggest a middle way:

a) add a field to the math table for texvc version (can be done with the BIN -> HEX change) but don't change the input hash at all. The version only needs to be updated when behaviour of a command changes, as error messages aren't cached.

b) change the output hash only if the output PNG may have changed (i.e. add a helper function changed_on() to texvc, like the tex_use_ams() stuff).

This avoids filling the disk with lots of duplicate images, and some easy analysis of the maths table will allow for further garbage collection when necessary.

It may be necessary to insert some retro-active changed_on()s, or to just invalidate all images once, to fix bugs currently there. (Or provide users with a method they can use to purge broken math images)

(In reply to comment #1)

I suggest a middle way:

a) add a field to the math table for texvc version (can be done with the BIN ->
HEX change) but don't change the input hash at all. The version only needs to
be updated when behaviour of a command changes, as error messages aren't
cached.

b) change the output hash only if the output PNG may have changed (i.e. add a
helper function changed_on() to texvc, like the tex_use_ams() stuff).

This avoids filling the disk with lots of duplicate images, and some easy
analysis of the maths table will allow for further garbage collection when
necessary.

Hmm... so the logic on parsing <math> would go roughly:

• calculate the input hash
• fetch 'math' table record
  • if no record, run texvc and save the new info into record
  • if record lists old version, run texvc and save the new info into record
  • if record lists current version, do nothing
• return the HTML/MathML/img from the record, depending on output format

After each texvc upgrade, this would cause a re-run of texvc for each unique <math>...</math> contents as they're encountered in wiki page parsing.

If the tweak for output hash is clever enough, this would save new versions of actually affected math bits -- cache-safe due to the new filename -- while non-affected math bits would save over the old file but not look any different, so no caching issues there.

Sounds pretty good to me!

Do we know how to adjust the output hashing only when particular commands are in use?

conrad.irwin wrote:

get per-command hash changes with texvc

(In reply to comment #2)

• calculate the input hash
• fetch 'math' table record
  • if no record, run texvc and save the new info into record
  • if record lists old version, run texvc and save the new info into record
  • if record lists current version, do nothing
• return the HTML/MathML/img from the record, depending on output format

I was originally planning to leave the old rows in the table, so that a maintenance script would be able to pick up old versions of re-rendered files and delete them when they are superceded. It may not be worth the cost of dobuling the size of the math table - I'll leave that as Wikimedia's call.

Sounds pretty good to me!

Do we know how to adjust the output hashing only when particular commands are
in use?

Patch attached :).

Attached:

sumanah wrote:

I'm sorry for the delay in response, Conrad. We're working on reducing our backlog of unreviewed commits and patches, since there's been such a wait. :-( Thanks for the patch. If you have time in the next couple of weeks, it would be great if you could check to make sure your patch still cleanly applies to MediaWiki as it is in our Subversion trunk. I'll try to get a reviewer soon!

Thanks.

physik wrote:

Rerendering can be forced with ?action=purge&mathpurge=true