Page MenuHomePhabricator
Create Task
Maniphest T177481

Falied startup tests for Pronlex
Closed, ResolvedPublic
Actions

Description

Pronlex stopped working and when I restart it, I get test failures.

P6080 Pronlex test failures
12017/10/05 11:28:59 lexserver: started
22017/10/05 11:28:59 demo_setup: creating demo database ...
32017/10/05 11:28:59 demo_setup: deleting demo db: /home/tts-agent/pronlex/db_files/demodb.db
42017/10/05 11:28:59 demo_setup: test database completed
52017/10/05 11:28:59 lexserver: creating test server on port :8799
62017/10/05 11:28:59 lexserver: loading dbs from folder /home/tts-agent/pronlex/db_files/
72017/10/05 11:28:59 lexserver: connecting to Sqlite3 db /home/tts-agent/pronlex/db_files/ar_ar_tst_lex.db
82017/10/05 11:28:59 lexserver: skipping file: '/home/tts-agent/pronlex/db_files/ar_ar_tst_lex.db-shm'
92017/10/05 11:28:59 lexserver: skipping file: '/home/tts-agent/pronlex/db_files/ar_ar_tst_lex.db-wal'
102017/10/05 11:28:59 lexserver: connecting to Sqlite3 db /home/tts-agent/pronlex/db_files/demodb.db
112017/10/05 11:28:59 lexserver: skipping file: '/home/tts-agent/pronlex/db_files/demodb.db-shm'
122017/10/05 11:28:59 lexserver: connecting to Sqlite3 db /home/tts-agent/pronlex/db_files/en_am_cmu_lex.db
132017/10/05 11:28:59 lexserver: skipping file: '/home/tts-agent/pronlex/db_files/en_am_cmu_lex.db-shm'
142017/10/05 11:28:59 lexserver: skipping file: '/home/tts-agent/pronlex/db_files/en_am_cmu_lex.db-wal'
152017/10/05 11:28:59 lexserver: connecting to Sqlite3 db /home/tts-agent/pronlex/db_files/no_nob_nst_lex.db
162017/10/05 11:28:59 lexserver: skipping file: '/home/tts-agent/pronlex/db_files/no_nob_nst_lex.db-shm'
172017/10/05 11:28:59 lexserver: skipping file: '/home/tts-agent/pronlex/db_files/no_nob_nst_lex.db-wal'
182017/10/05 11:28:59 lexserver: connecting to Sqlite3 db /home/tts-agent/pronlex/db_files/sv_se_nst_lex.db
192017/10/05 11:28:59 lexserver: skipping file: '/home/tts-agent/pronlex/db_files/sv_se_nst_lex.db-shm'
202017/10/05 11:28:59 lexserver: skipping file: '/home/tts-agent/pronlex/db_files/sv_se_nst_lex.db-wal'
212017/10/05 11:28:59 lexserver: loaded 5 db(s)
222017/10/05 11:28:59 lexserver: loading mapper definitions from file /home/tts-agent/pronlex/symbol_sets/mappers.txt
232017/10/05 11:28:59 lexserver: initializing mapper {sv-se_ws-sampa-DEMO sv-se_sampa_mary-DEMO [{" au . l a ' a*U - l a}]}
242017/10/05 11:28:59 lexserver: initializing mapper {sv-se_sampa_mary-DEMO sv-se_ws-sampa-DEMO [{' a*U - l a " au . l a}]}
252017/10/05 11:28:59 lexserver: initializing mapper {sv-se_ws-sampa sv-se_sampa_mary [{" au . l a ' a*U - l a}]}
262017/10/05 11:28:59 lexserver: initializing mapper {sv-se_sampa_mary sv-se_ws-sampa [{' a*U - l a " au . l a}]}
272017/10/05 11:28:59 lexserver: loaded symbol sets from dir /home/tts-agent/pronlex/symbol_sets/
282017/10/05 11:28:59 lexserver: loaded converters from dir /home/tts-agent/pronlex/symbol_sets/
292017/10/05 11:28:59 lexserver: loaded validators : [en-us_ws-sampa nb-no_ws-sampa sv-se_ws-sampa sv-se_ws-sampa-DEMO]
302017/10/05 11:28:59 lexserver: server created but not started for port :8799
312017/10/05 11:28:59 init_tests: waiting for server to start ...
322017/10/05 11:29:00 init_tests: server up and running using port :8799
332017/10/05 11:29:00 init_tests: running tests
342017/10/05 11:29:00 init_tests: testing response codes for built-in example URLs
352017/10/05 11:29:00 init_tests: /lexicon/list => /lexicon/list : 500 Internal Server Error
36** FAILED TEST ** for http://localhost:8799/lexicon/list : expected response code 200, found 500
372017/10/05 11:29:00 lexiconLookup: zero params, serving lexlookup.html
382017/10/05 11:29:00 init_tests: /lexicon/lookup => /lexicon/lookup : 200 OK
392017/10/05 11:29:00 init_tests: /lexicon/info/{lexicon_name} => /lexicon/info/demodb:demolex : 200 OK
402017/10/05 11:29:00 init_tests: /lexicon/stats/{lexicon_name} => /lexicon/stats/demodb:demolex : 500 Internal Server Error
41** FAILED TEST ** for http://localhost:8799/lexicon/stats/demodb:demolex : expected response code 200, found 500
422017/10/05 11:29:00 init_tests: /lexicon/list_current_entry_statuses/{lexicon_name} => /lexicon/list_current_entry_statuses/demodb:demole (...) : 500 Internal Server Error
43** FAILED TEST ** for http://localhost:8799/lexicon/list_current_entry_statuses/demodb:demolex : expected response code 200, found 500
442017/10/05 11:29:00 init_tests: /lexicon/list_all_entry_statuses/{lexicon_name} => /lexicon/list_all_entry_statuses/demodb:demolex : 500 Internal Server Error
45** FAILED TEST ** for http://localhost:8799/lexicon/list_all_entry_statuses/demodb:demolex : expected response code 200, found 500
462017/10/05 11:29:00 init_tests: /lexicon/validation_page => /lexicon/validation_page : 200 OK
472017/10/05 11:29:00 lexserver failed to update entry : DBManager.InsertEntries failed: failed exec : disk I/O error
482017/10/05 11:29:00 init_tests: /lexicon/addentry => /lexicon/addentry?lexicon_name=demodb:demolex&entr (...) : 500 Internal Server Error
49** FAILED TEST ** for http://localhost:8799/lexicon/addentry?lexicon_name=demodb:demolex&entry={%20%20%20%20%22strn%22:%20%22flesk%22,%20%20%20%20%22language%22:%20%22sv-se%22,%20%20%20%20%22partOfSpeech%22:%20%22NN%22,%20%20%20%20%22morphology%22:%20%22SIN-PLU|IND|NOM|NEU%22,%20%20%20%20%22wordParts%22:%20%22flesk%22,%20%20%20%20%22lemma%22:%20{ %22strn%22:%20%22flesk%22, %22reading%22:%20%22%22, %22paradigm%22:%20%22s7n-övriga%20ex%20träd%22%20%20%20%20},%20%20%20%20%22transcriptions%22:%20[ { %20%20%20%20 %20%20%20%20%22strn%22:%20%22\%22%20f%20l%20E%20s%20k%22, %20%20%20%20%22language%22:%20%22sv-se%22 }%20%20%20%20]} : expected response code 200, found 500
502017/10/05 11:29:00 init_tests: /validation/validateentry => /validation/validateentry?symbolsetname=sv-se_ws-s (...) : 200 OK
512017/10/05 11:29:00 init_tests: /validation/validateentries => /validation/validateentries?symbolsetname=sv-se_ws (...) : 200 OK
522017/10/05 11:29:00 init_tests: /validation/list => /validation/list : 200 OK
532017/10/05 11:29:00 validationStatsHandler failed to retrieve validation stats : dbapi.ValidationStats failed QueryRow : disk I/O error
542017/10/05 11:29:00 init_tests: /validation/stats/{lexicon_name} => /validation/stats/demodb:demolex : 400 Bad Request
55** FAILED TEST ** for http://localhost:8799/validation/stats/demodb:demolex : expected response code 200, found 400
562017/10/05 11:29:00 init_tests: /validation/has_validator/{symbolset} => /validation/has_validator/sv-se_ws-sampa : 200 OK
572017/10/05 11:29:00 init_tests: /validation/has_validator/{symbolset} => /validation/has_validator/ar_ws-sampa : 200 OK
582017/10/05 11:29:00 init_tests: /symbolset/list => /symbolset/list : 200 OK
592017/10/05 11:29:00 init_tests: /symbolset/content/{name} => /symbolset/content/sv-se_ws-sampa-DEMO : 200 OK
602017/10/05 11:29:00 Deleted symbol set sv-se_nst-xsampa-DEMO from cache
612017/10/05 11:29:00 Loaded symbol set sv-se_nst-xsampa-DEMO into cache
622017/10/05 11:29:00 init_tests: /symbolset/reload/{name} => /symbolset/reload/sv-se_nst-xsampa-DEMO : 200 OK
632017/10/05 11:29:00 lexserver: loading mapper definitions from file /home/tts-agent/pronlex/symbol_sets/mappers.txt
642017/10/05 11:29:00 lexserver: initializing mapper {sv-se_ws-sampa-DEMO sv-se_sampa_mary-DEMO [{" au . l a ' a*U - l a}]}
652017/10/05 11:29:00 lexserver: initializing mapper {sv-se_sampa_mary-DEMO sv-se_ws-sampa-DEMO [{' a*U - l a " au . l a}]}
662017/10/05 11:29:00 lexserver: initializing mapper {sv-se_ws-sampa sv-se_sampa_mary [{" au . l a ' a*U - l a}]}
672017/10/05 11:29:00 lexserver: initializing mapper {sv-se_sampa_mary sv-se_ws-sampa [{' a*U - l a " au . l a}]}
682017/10/05 11:29:00 init_tests: /symbolset/reload => /symbolset/reload : 200 OK
692017/10/05 11:29:00 init_tests: /symbolset/upload_page => /symbolset/upload_page : 200 OK
702017/10/05 11:29:00 init_tests: /mapper/list => /mapper/list : 200 OK
712017/10/05 11:29:00 init_tests: /mapper/map/{from}/{to}/{trans} => /mapper/map/sv-se_ws-sampa-DEMO/sv-se_sampa_mary-D (...) : 200 OK
722017/10/05 11:29:00 init_tests: /mapper/maptable/{from}/{to} => /mapper/maptable/sv-se_ws-sampa-DEMO/sv-se_sampa_m (...) : 200 OK
732017/10/05 11:29:00 init_tests: /converter/convert/{converter}/{trans} => /converter/convert/enusampa_svsampa-DEMO/%22 D i s : 200 OK
742017/10/05 11:29:00 init_tests: /converter/convert/{converter}/{trans} => /converter/convert/enusampa_svsampa-DEMO/%22 D EI (...) : 200 OK
752017/10/05 11:29:00 init_tests: /converter/list => /converter/list : 200 OK
762017/10/05 11:29:00 init_tests: /converter/table/{converter} => /converter/table/enusampa_svsampa-DEMO : 200 OK
772017/10/05 11:29:00 init_tests: /admin/lex_import_page => /admin/lex_import_page : 200 OK
782017/10/05 11:29:00 init_tests: /admin/list_dbs => /admin/list_dbs : 200 OK
792017/10/05 11:29:00 init_tests: /demo/search => /demo/search : 200 OK
802017/10/05 11:29:00 init_tests: /meta/urls => /meta/urls : 200 OK
812017/10/05 11:29:00 init_tests: /meta/examples => /meta/examples : 200 OK
822017/10/05 11:29:00 init_tests: testing some URLs with content
832017/10/05 11:29:00 init_tests: testing 200 status: 1
842017/10/05 11:29:00 init_tests: lookup/entry http://localhost:8799/ipa_table.txt
852017/10/05 11:29:00 init_tests: testing entry lookup: 4
862017/10/05 11:29:00 lexserver: Failed to get entries: DBManager.LookUp failed : dbapi.LookUp failed for demodb:[demolex] : disk I/O error
872017/10/05 11:29:00 init_tests: lookup/entry http://localhost:8799/lexicon/lookup?lexicons=demodb:demolex&wordlike=h%C3%A4st__
88** FAILED TEST ** for http://localhost:8799/lexicon/lookup?lexicons=demodb:demolex&wordlike=h%C3%A4st__ : expected response code 200, found 500
892017/10/05 11:29:00 lexserver: Failed to get entries: DBManager.LookUp failed : dbapi.LookUp failed for demodb:[demolex] : disk I/O error
902017/10/05 11:29:00 init_tests: lookup/entry http://localhost:8799/lexicon/lookup?lexicons=demodb:demolex&wordpartsregexp=h%C3%A4st
91** FAILED TEST ** for http://localhost:8799/lexicon/lookup?lexicons=demodb:demolex&wordpartsregexp=h%C3%A4st : expected response code 200, found 500
922017/10/05 11:29:00 lexserver: Failed to get entries: DBManager.LookUp failed : dbapi.LookUp failed for demodb:[demolex] : disk I/O error
932017/10/05 11:29:00 init_tests: lookup/entry http://localhost:8799/lexicon/lookup?lemmas=kex&lexicons=demodb:demolex
94** FAILED TEST ** for http://localhost:8799/lexicon/lookup?lemmas=kex&lexicons=demodb:demolex : expected response code 200, found 500
952017/10/05 11:29:00 lexserver: Failed to get entries: DBManager.LookUp failed : dbapi.LookUp failed for demodb:[demolex] : disk I/O error
962017/10/05 11:29:00 init_tests: lookup/entry http://localhost:8799/lexicon/lookup?lexicons=demodb:demolex&words=dom&transcriptionlike=%25o:%25&pp=yes
97** FAILED TEST ** for http://localhost:8799/lexicon/lookup?lexicons=demodb:demolex&words=dom&transcriptionlike=%25o:%25&pp=yes : expected response code 200, found 500
982017/10/05 11:29:00 init_tests: testing json map results: 3
992017/10/05 11:29:00 init_tests: jsonMap http://localhost:8799/mapper/map/sv-se_ws-sampa-DEMO/sv-se_sampa_mary-DEMO/%22%22%20p%20O%20j%20.%20k%20@
1002017/10/05 11:29:00 init_tests: jsonMap http://localhost:8799/mapper/map/sv-se_sampa_mary-DEMO/sv-se_ws-sampa-DEMO/%22%20p%20O%20j%20-%20k%20@
1012017/10/05 11:29:00 init_tests: jsonMap http://localhost:8799/validation/validateentry?symbolsetname=sv-se_ws-sampa-DEMO&entry={%22id%22:371546,%22lexiconId%22:1,%22strn%22:%22h%C3%A4st%22,%22language%22:%22SWE%22,%22partOfSpeech%22:%22NN%20SIN|IND|NOM|UTR%22,%22wordParts%22:%22h%C3%A4st%22,%22lemma%22:{%22id%22:42815,%22strn%22:%22h%C3%A4st%22,%22reading%22:%22%22,%22paradigm%22:%22s2q-lapp%22},%22transcriptions%22:[{%22id%22:377191,%22entryId%22:371546,%22strn%22:%22\%22%20h%20E%20s%20t%22,%22language%22:%22SWE%22,%22sources%22:[]}],%22status%22:{%22id%22:371546,%22name%22:%22imported%22,%22source%22:%22nst%22,%22timestamp%22:%222016-09-06T12:54:12Z%22,%22current%22:true}}
1022017/10/05 11:29:00 init_tests: testing json list results: 4
1032017/10/05 11:29:00 init_tests: jsonList http://localhost:8799/admin/list_dbs
1042017/10/05 11:29:00 init_tests: jsonList http://localhost:8799/mapper/list
1052017/10/05 11:29:00 init_tests: jsonList http://localhost:8799/symbolset/list
1062017/10/05 11:29:00 init_tests: jsonList http://localhost:8799/validation/list
1072017/10/05 11:29:00 init_tests: testing boolean results: 2
1082017/10/05 11:29:00 init_tests: jsonMap http://localhost:8799/validation/has_validator/sv-se_ws-sampa-DEMO
1092017/10/05 11:29:00 init_tests: jsonMap http://localhost:8799/validation/has_validator/ar_ws-sampa-DEMO
1102017/10/05 11:29:00 init_tests: 45 tests completed with 10 errors!
1112017/10/05 11:29:00 lexserver: INIT TESTS FAILED
112exit status 1

Related Objects

Event Timeline

Sebastian_Berlin-WMSE created this task.Oct 5 2017, 11:39 AM
Sebastian_Berlin-WMSE mentioned this in T177483: Horizontal scroll bar initially hidden for inline Pastes.Oct 5 2017, 11:59 AM
HannaLindgren subscribed.EditedOct 5 2017, 2:59 PM

Difficult to say why. It could be something that's changed since you made the initial setup (we have made quite a lot of changes over the past few weeks).

Do you have the demo lexicon database in your wikispeech/pronlex installation folder : db_files/demodb.db?
Which version/setup are you running? docker-compose/standalone, specific release/master?

You could try to remove the local wikispeech/pronlex installation folder and run the setup again.
Other than that, maybe try a re-install with the latest version (docker-compose from master branch) and let us know if the problem still occurs? Don't forget to remove the old installation folder before re-installing.

HannaLindgren added a comment.Oct 5 2017, 9:30 PM

disk I/O error could suggest something happened to the database when the server crashed.

Sebastian_Berlin-WMSE added a comment.Oct 6 2017, 6:40 AM

I tried re-importing the lexicons, but got the same error. A fresh installation of the standalone version worked (I used the developer version before, both on the v0.2.1 tag).

HannaLindgren added a comment.Oct 6 2017, 8:54 AM

I would guess there was something wrong with the databases, for example I the application did hang or was stopped while writing to files on disk.
Should it turn out to be a consistent error we will have to look at it, but for now, I don't think there is anything we (STTS) can do about this. Sounds OK?

Sebastian_Berlin-WMSE closed this task as Resolved.Oct 6 2017, 9:02 AM
Sebastian_Berlin-WMSE claimed this task.

Agreed. I'm happy to assume that this was a one time fluke, until it happens again.

Content licensed under Creative Commons Attribution-ShareAlike (CC BY-SA) 4.0 unless otherwise noted; code licensed under GNU General Public License (GPL) 2.0 or later and other open source licenses. By using this site, you agree to the Terms of Use, Privacy Policy, and Code of Conduct. · Wikimedia Foundation · Privacy Policy · Code of Conduct · Terms of Use · Disclaimer · CC-BY-SA · GPL