Page MenuHomePhabricator

Texlive on CI Trusty slaves lacks Ancient Greek causing Math test fail for php55
Closed, ResolvedPublic

Description

might be due to missing of greek latex packages
https://integration.wikimedia.org/ci/job/mwext-testextension-php55/329/consoleFull

1) MathCoverageTest::testCoverage with data set #424 ('\\Coppa\\coppa\\varcoppa', '<img class="mwe-math-fallback-image-inline tex" alt="\\Coppa\\coppa\\varcoppa" src="8308ee5003aa36112414cad8ef874f85.png" />')
Failed to render \Coppa\coppa\varcoppa
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'<img class="mwe-math-fallback-image-inline tex" alt="\Coppa\coppa\varcoppa" src="8308ee5003aa36112414cad8ef874f85.png" />'
+'<strong class='error texerror'>Failed to parse (PNG conversion failed; check for correct installation of latex and dvipng (or dvips + gs + convert)): \Coppa\coppa\varcoppa</strong>
+'

extensions/Math/tests/MathCoverageTest.php:78
tests/phpunit/MediaWikiTestCase.php:137

2) MathCoverageTest::testCoverage with data set #425 ('\\Digamma', '<img class="mwe-math-fallback-image-inline tex" alt="\\Digamma" src="5cfd6e5df6c87798542dca2e22c1e7cb.png" />')
Failed to render \Digamma
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'<img class="mwe-math-fallback-image-inline tex" alt="\Digamma" src="5cfd6e5df6c87798542dca2e22c1e7cb.png" />'
+'<strong class='error texerror'>Failed to parse (PNG conversion failed; check for correct installation of latex and dvipng (or dvips + gs + convert)): \Digamma</strong>
+'

extensions/Math/tests/MathCoverageTest.php:78
tests/phpunit/MediaWikiTestCase.php:137

3) MathCoverageTest::testCoverage with data set #426 ('\\Koppa\\koppa', '<img class="mwe-math-fallback-image-inline tex" alt="\\Koppa\\koppa" src="52593a0cdac178d165985ac014788b97.png" />')
Failed to render \Koppa\koppa
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'<img class="mwe-math-fallback-image-inline tex" alt="\Koppa\koppa" src="52593a0cdac178d165985ac014788b97.png" />'
+'<strong class='error texerror'>Failed to parse (PNG conversion failed; check for correct installation of latex and dvipng (or dvips + gs + convert)): \Koppa\koppa</strong>
+'

extensions/Math/tests/MathCoverageTest.php:78
tests/phpunit/MediaWikiTestCase.php:137

4) MathCoverageTest::testCoverage with data set #427 ('\\Sampi\\sampi', '<img class="mwe-math-fallback-image-inline tex" alt="\\Sampi\\sampi" src="e9dabb19e4c27bf23d3c2a3629474562.png" />')
Failed to render \Sampi\sampi
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'<img class="mwe-math-fallback-image-inline tex" alt="\Sampi\sampi" src="e9dabb19e4c27bf23d3c2a3629474562.png" />'
+'<strong class='error texerror'>Failed to parse (PNG conversion failed; check for correct installation of latex and dvipng (or dvips + gs + convert)): \Sampi\sampi</strong>
+'

extensions/Math/tests/MathCoverageTest.php:78
tests/phpunit/MediaWikiTestCase.php:137

5) MathCoverageTest::testCoverage with data set #428 ('\\Stigma\\stigma\\varstigma', '<img class="mwe-math-fallback-image-inline tex" alt="\\Stigma\\stigma\\varstigma" src="7b9233276816994a33a5e968202cef6e.png" />')
Failed to render \Stigma\stigma\varstigma
Failed asserting that two strings are equal.
--- Expected
+++ Actual
@@ @@
-'<img class="mwe-math-fallback-image-inline tex" alt="\Stigma\stigma\varstigma" src="7b9233276816994a33a5e968202cef6e.png" />'
+'<strong class='error texerror'>Failed to parse (PNG conversion failed; check for correct installation of latex and dvipng (or dvips + gs + convert)): \Stigma\stigma\varstigma</strong>
+'

extensions/Math/tests/MathCoverageTest.php:78
tests/phpunit/MediaWikiTestCase.php:137

Details

Related Gerrit Patches:
operations/puppet : productionmediawiki: add texlive-generic-extra
mediawiki/extensions/Math : masterwfDebugLog texvc returns/file output check

Event Timeline

Physikerwelt raised the priority of this task from to High.
Physikerwelt updated the task description. (Show Details)
Physikerwelt added a subscriber: Physikerwelt.
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 10 2016, 6:06 AM
Physikerwelt renamed this task from Math test fail fpr php55 to Math test fail for php55.Feb 10 2016, 3:29 PM
Physikerwelt set Security to None.

as discussed on more hints lead to the absense of https://www.ctan.org/tex-archive/macros/latex/contrib/teubner
<physikerwelt> jzerebecki do you by chance know if the php55 test are executed on different hardaware or at least with different packages installed https://phabricator.wikimedia.org/T126422 indicates that the greek latex package teubner might be missing
<jzerebecki> physikerwelt: yes those run on the next ubuntu lts release compared to php53

MaxSem added a subscriber: MaxSem.Feb 10 2016, 8:48 PM

That thing is contained in texlive-lang-greek which is present in production but not CI, apparently. From a cursory glance in Puppet, it should be there via texlive-lang-all - can a CI root check?

CI slaves are provisioned by puppet using role::ci::slaves::labs which includes mediawiki::packages. That should chip the same stuff the Wikimedia MediaWiki application servers are using.

I have no idea how texlive is provisioned in puppet though. One would want to look at how it is done in puppet and eventually have the relevant puppet class to be added to contint::packages which include a bunch of other package classes.

The tree for CI is roughly:

role::ci::slave::labs
---> contint::packages
    ---> mediawiki::packages
         ---> lot of mess
    ---> other package classes
hashar lowered the priority of this task from High to Medium.Feb 10 2016, 8:51 PM

Looking for texlive-lang-greek

Precise:

integration-slave-precise-1012.integration.eqiad.wmflabs:
    texlive-lang-greek:
      Installed: 2009-3
      Candidate: 2009-3
      Version table:
     *** 2009-3 0
            500 http://nova.clouds.archive.ubuntu.com/ubuntu/ precise/main amd64 Packages
            100 /var/lib/dpkg/status

Trusty:

integration-slave-trusty-1014.integration.eqiad.wmflabs:
    texlive-lang-greek:
      Installed: 2013.20140215-1
      Candidate: 2013.20140215-1
      Version table:
     *** 2013.20140215-1 0
            500 http://nova.clouds.archive.ubuntu.com/ubuntu/ trusty/universe amd64 Packages
            100 /var/lib/dpkg/status
JanZerebecki added a comment.EditedFeb 10 2016, 9:01 PM
This comment has been deleted.

Could it be because maybe math is incompatible with 2013 version and is compatible with 2009 version which means math needs a compatibility update to support 2013.

on vagrant we have

'ocaml-native-compilers',
'texlive',
'texlive-bibtex-extra',
'texlive-font-utils',
'texlive-fonts-extra',
'texlive-lang-croatian',
'texlive-lang-cyrillic',
'texlive-lang-czechslovak',
'texlive-lang-danish',
'texlive-lang-dutch',
'texlive-lang-finnish',
'texlive-lang-french',
'texlive-lang-german',
'texlive-lang-greek',
'texlive-lang-hungarian',
'texlive-lang-italian',
'texlive-lang-latin',
'texlive-lang-mongolian',
'texlive-lang-norwegian',
'texlive-lang-other',
'texlive-lang-polish',
'texlive-lang-portuguese',
'texlive-lang-spanish',
'texlive-lang-swedish',
'texlive-lang-vietnamese',
'texlive-latex-extra',
'texlive-math-extra',
'texlive-pictures',
'texlive-pstricks',
'texlive-publishers',
'texlive-generic-extra'
integration-slave-trusty-1012:~$ dpkg -s 'ocaml-native-compilers' 'texlive' 'texlive-bibtex-extra' 'texlive-font-utils' 'texlive-fonts-extra' 'texlive-lang-croatian' 'texlive-lang-cyrillic' 'texlive-lang-czechslovak' 'texlive-lang-danish' 'texlive-lang-dutch' 'texlive-lang-finnish' 'texlive-lang-french' 'texlive-lang-german' 'texlive-lang-greek' 'texlive-lang-hungarian' 'texlive-lang-italian' 'texlive-lang-latin' 'texlive-lang-mongolian' 'texlive-lang-norwegian' 'texlive-lang-other' 'texlive-lang-polish' 'texlive-lang-portuguese' 'texlive-lang-spanish' 'texlive-lang-swedish' 'texlive-lang-vietnamese' 'texlive-latex-extra' 'texlive-math-extra' 'texlive-pictures' 'texlive-pstricks' 'texlive-publishers' 'texlive-generic-extra'|egrep '^(Package|Status|Version)'                                                                                                                                    
Package: ocaml-native-compilers
Status: install ok installed
Version: 4.01.0-3ubuntu3
Package: texlive
Status: install ok installed
Version: 2013.20140215-1
Package: texlive-bibtex-extra
Status: install ok installed
Version: 2013.20140215-2
Package: texlive-font-utils
Status: install ok installed
Version: 2013.20140215-2
Package: texlive-fonts-extra
Status: install ok installed
Version: 2013.20140215-2
dpkg-query: package 'texlive-lang-croatian' is not installed and no information is available
Package: texlive-lang-cyrillic
Status: install ok installed
Version: 2013.20140215-1
Package: texlive-lang-czechslovak
Status: install ok installed
Version: 2013.20140215-1
dpkg-query: package 'texlive-lang-danish' is not installed and no information is available
dpkg-query: package 'texlive-lang-dutch' is not installed and no information is available
dpkg-query: package 'texlive-lang-finnish' is not installed and no information is available
Package: texlive-lang-french
Status: install ok installed
Version: 2013.20140215-1
Package: texlive-lang-german
Status: install ok installed
Version: 2013.20140215-1
Package: texlive-lang-greek
Status: install ok installed
Version: 2013.20140215-1
dpkg-query: package 'texlive-lang-hungarian' is not installed and no information is available
Package: texlive-lang-italian
Status: install ok installed
Version: 2013.20140215-1
dpkg-query: package 'texlive-lang-latin' is not installed and no information is available
dpkg-query: package 'texlive-lang-mongolian' is not installed and no information is available
dpkg-query: package 'texlive-lang-norwegian' is not installed and no information is available
Package: texlive-lang-other
Status: install ok installed
Version: 2013.20140215-1
Package: texlive-lang-polish
Status: install ok installed
Version: 2013.20140215-1
Package: texlive-lang-portuguese
Status: install ok installed
Version: 2013.20140215-1
Package: texlive-lang-spanish
Status: install ok installed
Version: 2013.20140215-1
dpkg-query: package 'texlive-lang-swedish' is not installed and no information is available
dpkg-query: package 'texlive-lang-vietnamese' is not installed and no information is available
Package: texlive-latex-extra
Status: install ok installed
Version: 2013.20140215-2
Package: texlive-math-extra
Status: install ok installed
Version: 2013.20140215-2
Package: texlive-pictures
Status: install ok installed
Version: 2013.20140215-1
Package: texlive-pstricks
Status: install ok installed
Version: 2013.20140215-2
Package: texlive-publishers
Status: install ok installed
Version: 2013.20140215-2
dpkg-query: package 'texlive-generic-extra' is not installed and no information is available
Use dpkg --info (= dpkg-deb --info) to examine archive files,
and dpkg --contents (= dpkg-deb --contents) to list their contents.

It looks like texlive-generic-extra is not installed on trusty. Could that package be causing the issue.

Also it looks like these

'texlive-math-extra',
'texlive-pictures',
'texlive-pstricks',
'texlive-publishers',
'texlive-generic-extra'

Aren't installed on trusty either. They should be installed. It looks like the picture one is related to the errors.

What base image is the vagrant using where this works?

The mediawiki-math-texvc package is installed on all slaves with version 2:1.0+git20140526-1. Confirmed via:

salt -v '*slave*' pkg.version mediawiki-math-texvc

It contains the texvc binary generated by the old OCAML based renderer.

hashar updated the task description. (Show Details)Feb 11 2016, 1:14 PM

Edited task detail and copy pasted the failures.

For \Digamma there is T29754 which got filled when we had Ubuntu Hardy. It references teubner which is found on Trusty:

$ apt-cache search teubner
libstatistics-test-sequence-perl - Perl module that tests correlation of random numbers
texlive-lang-greek - TeX Live: Greek
texlive-publishers - TeX Live: Publisher styles, theses, etc.

The texlive-lang-greek package is installed on all CI slaves. With the version being different:

Precise2009-3
Trusty2013.20140215-1
integration-slave-trusty-1012:~$ dpkg -s 'ocaml-native-compilers' 'texlive' 'texlive-bibtex-extra' 'texlive-font-utils' 'texlive-fonts-extra' 'texlive-lang-croatian' 'texlive-lang-cyrillic' 'texlive-lang-czechslovak' 'texlive-lang-danish' 'texlive-lang-dutch' 'texlive-lang-finnish' 'texlive-lang-french' 'texlive-lang-german' 'texlive-lang-greek' 'texlive-lang-hungarian' 'texlive-lang-italian' 'texlive-lang-latin' 'texlive-lang-mongolian' 'texlive-lang-norwegian' 'texlive-lang-other' 'texlive-lang-polish' 'texlive-lang-portuguese' 'texlive-lang-spanish' 'texlive-lang-swedish' 'texlive-lang-vietnamese' 'texlive-latex-extra' 'texlive-math-extra' 'texlive-pictures' 'texlive-pstricks' 'texlive-publishers' 'texlive-generic-extra'|egrep '^(Package|Status|Version)'                                                                                                                                    
Package: ocaml-native-compilers
Status: install ok installed
Version: 4.01.0-3ubuntu3
Package: texlive
Status: install ok installed
Version: 2013.20140215-1
Package: texlive-bibtex-extra
Status: install ok installed
Version: 2013.20140215-2
Package: texlive-font-utils
Status: install ok installed
Version: 2013.20140215-2
Package: texlive-fonts-extra
Status: install ok installed
Version: 2013.20140215-2
dpkg-query: package 'texlive-lang-croatian' is not installed and no information is available
Package: texlive-lang-cyrillic
Status: install ok installed
Version: 2013.20140215-1
Package: texlive-lang-czechslovak
Status: install ok installed
Version: 2013.20140215-1
dpkg-query: package 'texlive-lang-danish' is not installed and no information is available
dpkg-query: package 'texlive-lang-dutch' is not installed and no information is available
dpkg-query: package 'texlive-lang-finnish' is not installed and no information is available
Package: texlive-lang-french
Status: install ok installed
Version: 2013.20140215-1
Package: texlive-lang-german
Status: install ok installed
Version: 2013.20140215-1
Package: texlive-lang-greek
Status: install ok installed
Version: 2013.20140215-1
dpkg-query: package 'texlive-lang-hungarian' is not installed and no information is available
Package: texlive-lang-italian
Status: install ok installed
Version: 2013.20140215-1
dpkg-query: package 'texlive-lang-latin' is not installed and no information is available
dpkg-query: package 'texlive-lang-mongolian' is not installed and no information is available
dpkg-query: package 'texlive-lang-norwegian' is not installed and no information is available
Package: texlive-lang-other
Status: install ok installed
Version: 2013.20140215-1
Package: texlive-lang-polish
Status: install ok installed
Version: 2013.20140215-1
Package: texlive-lang-portuguese
Status: install ok installed
Version: 2013.20140215-1
Package: texlive-lang-spanish
Status: install ok installed
Version: 2013.20140215-1
dpkg-query: package 'texlive-lang-swedish' is not installed and no information is available
dpkg-query: package 'texlive-lang-vietnamese' is not installed and no information is available
Package: texlive-latex-extra
Status: install ok installed
Version: 2013.20140215-2
Package: texlive-math-extra
Status: install ok installed
Version: 2013.20140215-2
Package: texlive-pictures
Status: install ok installed
Version: 2013.20140215-1
Package: texlive-pstricks
Status: install ok installed
Version: 2013.20140215-2
Package: texlive-publishers
Status: install ok installed
Version: 2013.20140215-2
dpkg-query: package 'texlive-generic-extra' is not installed and no information is available
Use dpkg --info (= dpkg-deb --info) to examine archive files,
and dpkg --contents (= dpkg-deb --contents) to list their contents.

@hashar I'm not sure why it fails then. But it seems not all the packages are installed seems they are missing

texlive-math-extra',
'texlive-pictures',
'texlive-pstricks',
'texlive-publishers',
'texlive-generic-extra'

The old patch for texvc: https://www.mediawiki.org/wiki/Special:Code/MediaWiki/97014 that bring back a previous change https://www.mediawiki.org/wiki/Special:Code/MediaWiki/87298.

Maybe teubner has to be explicitly enabled/privileged somehow.

hashar renamed this task from Math test fail for php55 to Texlive on CI Trusty slaves lacks Ancient Greek causing Math test fail for php55.Feb 12 2016, 11:09 AM

It is definitely an issue with the way Texlive is installed / set up on the CI Trusty slaves. I am wondering how that behave on Wikimedia Production (if it is still using tex).

Would be nice to have some command to reproduce it easily from the CLI on the instances.

Maybe there are some lookup order issue or the texlive package providing the archaic greek characters needs to be enabled somehow.

Regardless, not much I or CI folks can do unfortunately :-( I have no knowledge about Tex myself.

At least tex seems to be able to generate something:

---
[Math] Start rendering $\Sampi\sampi$ in mode png
[Math] TeX: /mwext-testextension-php55@3/src/extensions/Math/math/texvc
               '/mnt/home/jenkins-deploy/tmpfs/jenkins-1'
               '/mnt/home/jenkins-deploy/tmpfs/jenkins-1'
               '\Sampi\sampi'
               'UTF-8'
               'transparent'
[Math] Executing '/mwext-testextension-php55@3/src/extensions/Math/math/texvc
               '/mnt/home/jenkins-deploy/tmpfs/jenkins-1'
               '/mnt/home/jenkins-deploy/tmpfs/jenkins-1'
               '\Sampi\sampi'
               'UTF-8'
               'transparent''.
659.8613 572.2M  wfShellExec: /bin/bash '/mwext-testextension-php55@3/src/includes/limit.sh'
'/mwext-testextension-php55@3/src/extensions/Math/math/texvc '\''/mnt/home/jenkins-deploy/tmpfs/jenkins-1'\'' '\''/mnt/home/jenkins-deploy/tmpfs/jenkins-1'\'' '\''\Sampi\sampi'\'' '\''UTF-8'\'' '\''transparent'\''' 'MW_INCLUDE_STDERR=;MW_CPU_LIMIT=180; MW_CGROUP='\'''\''; MW_MEM_LIMIT=307200; MW_FILE_SIZE_LIMIT=102400; MW_WALL_CLOCK_LIMIT=180; MW_USE_LOG_PIPE=yes'
[Math] TeX output:
 +e9dabb19e4c27bf23d3c2a3629474562-

Giving up.

Paladox added a comment.EditedFeb 12 2016, 1:15 PM

@hashar Maybe if we look at differences between Precise and Trusty we may spot what is missing or what is miss behaving.

Too bad. Yet another reason to make progress on https://phabricator.wikimedia.org/T74240

Physikerwelt added a comment.EditedFeb 12 2016, 1:53 PM

e9dabb19e4c27bf23d3c2a3629474562 that should be the name of the tmp file in
/mnt/home/jenkins-deploy/tmpfs/jenkins-1
there should be tex file and finally a png file
How are the permissions set for those directory?

Change 270288 had a related patch set uploaded (by Hashar):
wfDebugLog texvc returns/file output check

https://gerrit.wikimedia.org/r/270288

e9dabb19e4c27bf23d3c2a3629474562 that should be the name of the tmp file in
/mnt/home/jenkins-deploy/tmpfs/jenkins-1
there should be tex file and finally a png file
How are the permissions set for those directory?

The /mnt/home/jenkins-deploy/tmpfs/jenkins-1 is the temporary directory set via LocalSettings. It is world writable and emptied on build completion.

[MathTexvc] HASHAR file does not exist /mnt/home/jenkins-deploy/tmpfs/jenkins-1/8308ee5003aa36112414cad8ef874f85.png wfTempDir(): /mnt/home/jenkins-deploy/tmpfs/jenkins-1
[Math] Start rendering $\Digamma$ in mode png
[Math] TeX: /mnt/jenkins-workspace/workspace/mwext-testextension-hhvm/src/extensions/Math/math/texvc '/mnt/home/jenkins-deploy/tmpfs/jenkins-1' '/mnt/home/jenkins-deploy/tmpfs/jenkins-1' '\Digamma' 'UTF-8' 'transparent'
[Math] Executing '/mnt/jenkins-workspace/workspace/mwext-testextension-hhvm/src/extensions/Math/math/texvc '/mnt/home/jenkins-deploy/tmpfs/jenkins-1' '/mnt/home/jenkins-deploy/tmpfs/jenkins-1' '\Digamma' 'UTF-8' 'transparent''.
364.9591 637.9M  wfShellExec: /bin/bash '/mnt/jenkins-workspace/workspace/mwext-testextension-hhvm/src/includes/limit.sh' '/mnt/jenkins-workspace/workspace/mwext-testextension-hhvm/src/extensions/Math/math/texvc '\''/mnt/home/jenkins-deploy/tmpfs/jenkins-1'\'' '\''/mnt/home/jenkins-deploy/tmpfs/jenkins-1'\'' '\''\Digamma'\'' '\''UTF-8'\'' '\''transparent'\''' 'MW_INCLUDE_STDERR=;MW_CPU_LIMIT=180; MW_CGROUP='\'''\''; MW_MEM_LIMIT=307200; MW_FILE_SIZE_LIMIT=102400; MW_WALL_CLOCK_LIMIT=180; MW_USE_LOG_PIPE=yes'
[Math] TeX output:
 l5cfd6e5df6c87798542dca2e22c1e7cb&#988;-

Note the hash mismatch:

ParserTest5cfd6e5df6c87798542dca2e22c1e7cb.png
Expect8308ee5003aa36112414cad8ef874f85.png
texvcl5cfd6e5df6c87798542dca2e22c1e7cb&988;-

So the expected hash does not match what texvc generate (that one hasn't changed). The way the hash is determined via getHash() is somehow changed with Zend 5.5 :-(

Change 270288 abandoned by Hashar:
wfDebugLog texvc returns/file output check

Reason:
was merely to debug log the math .png files

https://gerrit.wikimedia.org/r/270288

@hashar: You tested \Digamma which expects 5cfd6e5df6c87798542dca2e22c1e7cb.
8308ee5003aa36112414cad8ef874f85 is for \Coppa\coppa\varcoppa

Eekkkk. So log output is proper:

[Math] Start rendering $\Digamma$ in mode png
[Math] TeX: /mwext-testextension-hhvm/src/extensions/Math/math/texvc '/mnt/home/jenkins-deploy/tmpfs/jenkins-1' '/mnt/home/jenkins-deploy/tmpfs/jenkins-1' '\Digamma' 'UTF-8' 'transparent'
[Math] Executing '/mwext-testextension-hhvm/src/extensions/Math/math/texvc '/mnt/home/jenkins-deploy/tmpfs/jenkins-1' '/mnt/home/jenkins-deploy/tmpfs/jenkins-1' '\Digamma' 'UTF-8' 'transparent''.
364.9591 637.9M  wfShellExec: /bin/bash '/mwext-testextension-hhvm/src/includes/limit.sh' '/mwext-testextension-hhvm/src/extensions/Math/math/texvc '\''/mnt/home/jenkins-deploy/tmpfs/jenkins-1'\'' '\''/mnt/home/jenkins-deploy/tmpfs/jenkins-1'\'' '\''\Digamma'\'' '\''UTF-8'\'' '\''transparent'\''' 'MW_INCLUDE_STDERR=;MW_CPU_LIMIT=180; MW_CGROUP='\'''\''; MW_MEM_LIMIT=307200; MW_FILE_SIZE_LIMIT=102400; MW_WALL_CLOCK_LIMIT=180; MW_USE_LOG_PIPE=yes'
[Math] TeX output:
 l5cfd6e5df6c87798542dca2e22c1e7cb&#988;-
---
[MathTexvc] HASHAR file does not exist /mnt/home/jenkins-deploy/tmpfs/jenkins-1/5cfd6e5df6c87798542dca2e22c1e7cb.png wfTempDir(): /mnt/home/jenkins-deploy/tmpfs/jenkins-1

The texvc command seems to indicate it writes down to /mnt/home/jenkins-deploy/tmpfs/jenkins-1

For some php53 build:

[Math] Start rendering $\Digamma$ in mode png
[Math] TeX: /mnt/jenkins-workspace/workspace/mwext-testextension-php53/src/extensions/Math/math/texvc '/mnt/home/jenkins-deploy/tmpfs/jenkins-0' '/mnt/home/jenkins-deploy/tmpfs/jenkins-0' '\Digamma' 'UTF-8' 'transparent'
[Math] Executing '/mnt/jenkins-workspace/workspace/mwext-testextension-php53/src/extensions/Math/math/texvc '/mnt/home/jenkins-deploy/tmpfs/jenkins-0' '/mnt/home/jenkins-deploy/tmpfs/jenkins-0' '\Digamma' 'UTF-8' 'transparent''.
1075.9199 911.8M  wfShellExec: /bin/bash '/mnt/jenkins-workspace/workspace/mwext-testextension-php53/src/includes/limit.sh' '/mnt/jenkins-workspace/workspace/mwext-testextension-php53/src/extensions/Math/math/texvc '\''/mnt/home/jenkins-deploy/tmpfs/jenkins-0'\'' '\''/mnt/home/jenkins-deploy/tmpfs/jenkins-0'\'' '\''\Digamma'\'' '\''UTF-8'\'' '\''transparent'\''' 'MW_INCLUDE_STDERR=;MW_CPU_LIMIT=180; MW_CGROUP='\'''\''; MW_MEM_LIMIT=307200; MW_FILE_SIZE_LIMIT=102400; MW_WALL_CLOCK_LIMIT=180; MW_USE_LOG_PIPE=yes'
[Math] TeX output:
 l5cfd6e5df6c87798542dca2e22c1e7cb&#988;
---
[Math] TeX: getHashPath, hash is: 5cfd6e5df6c87798542dca2e22c1e7cb, path is: mwstore://math-backend/math-render/5/c/f

So not a hash issue. Some how the file is not written to the temporary directory :-((

The only difference I noticed in the output that it's /mnt/home/jenkins-deploy/tmpfs/jenkins-0 for pho53 and /mnt/home/jenkins-deploy/tmpfs/jenkins-1 for php55

The only difference I noticed in the output that it's /mnt/home/jenkins-deploy/tmpfs/jenkins-0 for pho53 and /mnt/home/jenkins-deploy/tmpfs/jenkins-1 for php55

Yup the last digit is the executor number on the slave. They can run four jobs in parallel from 0 to 4 and that is a way to vary the tmpfs between build running concurrently. Since the tmp dir is garbage collected at the end of the build, if it wasn't shard we would end up with a race condition with a build completion deleting temp files from another build.

@hashar is there any way for us to disable that test temporarily I mean disable the test that fails not the actual test. Like we did with TimedMediaHandler we disabled one of the test because it wasent running correctly.

How would we do that.

And @Physikerwelt should we disable that test temporarily.

@hashar, can you try to run the latex command that is actually run by the texvc?

Create a tex file

\nonstopmode
\documentclass[12pt]{article}
\usepackage[greek]{babel}
\usepackage{teubner}
\pagestyle{empty}
\begin{document}
$$
\Digamma
$$
\end{document}

latex filename.tex

More details are availible from render.ml in the math subdirectory of the math extension.

@Paladox the more tests we have the better;-) But all the files latex etc will be gone with T74240

@hashar for testing it's actually better to leave out the \nonstopmode

hashar@integration-slave-trusty-1001:~$ latex T126422.tex 
This is pdfTeX, Version 3.1415926-2.5-1.40.14 (TeX Live 2013/Debian)
 restricted \write18 enabled.
entering extended mode
(./T126422.tex
LaTeX2e <2011/06/27>
Babel <3.9h> and hyphenation patterns for 78 languages loaded.

(/usr/share/texlive/texmf-dist/tex/latex/base/article.cls
Document Class: article 2007/10/19 v1.4h Standard LaTeX document class
(/usr/share/texlive/texmf-dist/tex/latex/base/size12.clo))
(/usr/share/texlive/texmf-dist/tex/generic/babel/babel.sty
(/usr/share/texlive/texmf-dist/tex/generic/babel-greek/greek.ldf
(/usr/share/texlive/texmf-dist/tex/generic/babel/babel.def)
(/usr/share/texlive/texmf-dist/tex/latex/greek-fontenc/lgrenc.def
(/usr/share/texlive/texmf-dist/tex/latex/greek-fontenc/greek-fontenc.def))))
(/usr/share/texlive/texmf-dist/tex/latex/teubner/teubner.sty

! LaTeX Error: File `iftex.sty' not found.

Type X to quit or <RETURN> to proceed,
or enter new name. (Default extension: sty)

Enter file name: 
! Emergency stop.
<read *> 
         
l.50 \let
         \ifPDF\ifPDFTeX^^M
No pages of output.
Transcript written on T126422.log.
hashar@integration-slave-trusty-1001:~$

! LaTeX Error: File iftex.sty' not found.`

eeeeew


If I remove \usepackage{teubner} and \Digama it pass.

Removing just \usepackage{teubner} that yields:
LaTeX Warning: Command \textDigamma invalid in math mode on input line 7.

That points to OCG commit https://gerrit.wikimedia.org/r/#/c/107585/

Add missing texlive-generic-extra dependency.

This is listed in the .travis.yml config (for Ubuntu 12) but it's also
needed for Ubuntu 13.  It provides `iftex.sty`.

Looks like that package should be added in modules/mediawiki/manifests/packages/tex.pp

That file is in texlive-generic-extra, so maybe the solution is to fix the puppet code to install all the packages that my paste above say are missing.

Change 270322 had a related patch set uploaded (by Hashar):
mediawiki: add texlive-generic-extra

https://gerrit.wikimedia.org/r/270322

Mentioned in SAL [2016-02-12T17:32:35Z] <hashar> adding texlive-generic-extra on CI slaves by cherry picking https://gerrit.wikimedia.org/r/#/c/270322/ - T126422

$ latex T126422.tex 
This is pdfTeX, Version 3.1415926-2.5-1.40.14 (TeX Live 2013/Debian)
 restricted \write18 enabled.
entering extended mode
(./T126422.tex
LaTeX2e <2011/06/27>
Babel <3.9h> and hyphenation patterns for 78 languages loaded.

(/usr/share/texlive/texmf-dist/tex/latex/base/article.cls
Document Class: article 2007/10/19 v1.4h Standard LaTeX document class
(/usr/share/texlive/texmf-dist/tex/latex/base/size12.clo))
(/usr/share/texlive/texmf-dist/tex/generic/babel/babel.sty
(/usr/share/texlive/texmf-dist/tex/generic/babel-greek/greek.ldf
(/usr/share/texlive/texmf-dist/tex/generic/babel/babel.def)
(/usr/share/texlive/texmf-dist/tex/latex/greek-fontenc/lgrenc.def
(/usr/share/texlive/texmf-dist/tex/latex/greek-fontenc/greek-fontenc.def))))
(/usr/share/texlive/texmf-dist/tex/latex/teubner/teubner.sty
(/usr/share/texlive/texmf-dist/tex/generic/iftex/iftex.sty)
(/usr/share/texlive/texmf-dist/tex/latex/graphics/graphicx.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/keyval.sty)
(/usr/share/texlive/texmf-dist/tex/latex/graphics/graphics.sty
(/usr/share/texlive/texmf-dist/tex/latex/graphics/trig.sty)
(/usr/share/texlive/texmf-dist/tex/latex/latexconfig/graphics.cfg)
(/usr/share/texlive/texmf-dist/tex/latex/graphics/dvips.def)))
(/usr/share/texlive/texmf-dist/tex/latex/base/ifthen.sty)
(/usr/share/texlive/texmf-dist/tex/latex/base/exscale.sty)
(/usr/share/texlive/texmf-dist/tex/latex/tools/trace.sty)) (./T126422.aux
(/usr/share/texlive/texmf-dist/tex/latex/cbfonts-fd/lgrcmr.fd))

LaTeX Warning: Command \F invalid in math mode on input line 8.

[1] (./T126422.aux) )
(see the transcript file for additional information)
Output written on T126422.dvi (1 page, 152 bytes).
Transcript written on T126422.log.

Mentioned in SAL [2016-02-12T17:36:29Z] <hashar> salt -v '*slave-trusty*' cmd.run 'apt-get -y install texlive-generic-extra' # T126422

Should be good now ?

hashar closed this task as Resolved.Feb 12 2016, 8:44 PM

I have 'recheck' a few changes and the tests pass. The root cause is that Ubuntu Trusty requires texlive-generic-extra to be installed which was previously missing.

Change 270322 merged by Dzahn:
mediawiki: add texlive-generic-extra

https://gerrit.wikimedia.org/r/270322