Request for increasing labs' account's Vmem limitation
Closed, DeclinedPublic
Actions

Assigned To

None

Authored By

	Yamaha5
	Jul 25 2016, 6:20 PM

Description

I want to run WPcleaning's code. it uses dump file. when i run the code it shows an error about out of memory. would you please increase my acount's Vmem Limitation

the code

wget https://dumps.wikimedia.org/fawiki/latest/fawiki-latest-pages-articles.xml.bz2
wget http://site4145.mutu.sivit.org/WikiCleaner/WikipediaCleaner.jar
java -Xmx5g -cp WikipediaCleaner.jar org.wikipediacleaner.Bot fa Rezabot ******* ListCheckWiki fawiki-latest-pages-articles.xml.bz2 wiki:wikipedia:WikiProject_Check_Wikipedia/A_{0} 002

My account name is Reza

Event Timeline

Yamaha5 created this task.Jul 25 2016, 6:20 PM

Restricted Application added a subscriber: Aklapper. · View Herald TranscriptJul 25 2016, 6:20 PM

Yamaha5 renamed this task from Request for increasing lab's account's Vmem limitation to Request for increasing labs' account's Vmem limitation .Jul 25 2016, 6:20 PM

Can you change 'java -Xmx5g' to 'java -Xmx4g' and see if that works?

Also what's the jsub command you are using to submit this? Are you trying to run this on the bastions directly?

I run it directly to have a test. the dump is more than 4g so -Xmx4g doesn't work

Ah, I think the bastions in general have stricter memory limits than that, so you don't make it unusable for other people. For something that requires an entire 4g of RAM you'll have to definitely submit it to the grid to test, I'm afriad.

I tried to submit and it shows this error

jsub -once -N wpclean  -mem 5g java -Xmx4g -cp /data/project/rezabot/WikipediaCleaner.jar org.wikipediacleaner.Bot fa Rezabot ***** ListCheckWiki fawiki-latest-pages-articles.xml.bz2 wiki:wikipedia:WikiProject_Check_Wikipedia/A_{0} 002

error:

Error: Could not create the Java Virtual Machine.
Error: A fatal exception has occurred. Program will exit.
Error: Could not find or load main class cp

Right - in this case I think the grid *did* provide 5G of VMEM (we don't have upper limits on general girdengine usage, only on webservice usage), and JVM still refused to start. This requires JVM tuning, not gridengine I think.

• Phabricator_maintenance removed a subscriber: yuvipanda.Jun 7 2017, 6:41 PM

Harej edited projects, added Cloud-VPS (Quota-requests); removed Cloud-Services.Mar 5 2018, 10:07 PM

Harej edited projects, added Toolforge; removed Cloud-VPS (Quota-requests).

Closing as declined because bug was inactive for over 18 months. As Yuvi pointed out in T141288#2493221, only the webservice grid runner has a built in high water mark that has to be increased by admins. If the tool is attempting to open an entire compressed dump it is unlikely that the grid will ever have enough free ram to do that however. Without source code or further explanation of the data processing algorithms in place there's not much we can do.

Request for increasing labs' account's Vmem limitation Closed, DeclinedPublicActions

Description

Event Timeline

Request for increasing labs' account's Vmem limitation
Closed, DeclinedPublic
Actions