• We have selected one Outreachy intern that will start on December. The intern will help on the task of developing the machine readable dataset.

• The dataset are ready. We are waiting for the paper to be published to share the link in public. Currently, the datasets are available under request via email.

• We started a preliminary analysis propagation of sources across Wikis.

• We are extending the list to other languages: es, pt, ca.
• Reviewing outreaching applications that will help on creating the machine readable dataset.

• We are exploring a follow-up on this project, that based on our results, will focus on how to model the spread of disinformation.

For more details on the timeline recommendations please check Isaac's comment here: T263874#6589856

Got you. Yes, looks good, please add it in the outreachy application.

Hello everybody!

@KemmieKemy thanks for submitting. You are doing great progress.

@elukey, yes.

@Rvvalentim , please can you double check if you need any of those files?

• Paper was submitted last week.

In T263874#6534422, @diego wrote:

In T263874#6534403, @Toluwani7 wrote:

# Wikidata JSON dump we'll start processing (56 GB in size, compressed) so far too large to process the whole thing right now
!ls -shH "{WIKIDATA_DIR}{WIKIDATA_DUMP_FN}"

I'm still going through the Wikidata example, do you know what the shH option might mean? I can't find it online

Here you have: https://man7.org/linux/man-pages/man1/ls.1.html

In T263874#6534403, @Toluwani7 wrote:

# Wikidata JSON dump we'll start processing (56 GB in size, compressed) so far too large to process the whole thing right now
!ls -shH "{WIKIDATA_DIR}{WIKIDATA_DUMP_FN}"

I'm still going through the Wikidata example, do you know what the shH option might mean? I can't find it online

Hi @Lisasiziba
Please check the instructions here T263874.

• We are currently working on the paper, adding new analysis, and improvements on the model published in the first round of analysis.

• The data can be found in HDFS: /user/dsaez/topicsForAllWikipediaPages2020-08-24AllProps.csv , OR
• You can also download the dataset from: https://analytics.wikimedia.org/published/datasets/topics/
• The dataset follows the same format described here: https://figshare.com/articles/Topics_for_each_Wikipedia_Article_across_Languages/12127434

Hi @RBrounley_WMF, thanks for sharing this and for the great work you are doing. Few comments from my side:

@leila I see some overlap although this task seems to be broader than the one I'm working on. Given that I don't see much documentation nor code about this task, I prefer to not take responsibility on this.

• We are currently working on preparing a paper to be submitted at the end of October.

• The two datasets have been prepared:
  • One dataset with items that propagates across Wikipedias, removing bot activity.
  • Another dataset about external references (links) across projects.

• A recent (with all the articles existing until Aug 31th) dump have been created. During the following days I will upload it in a public repository.

Updates

• We have finished the first model.
• Report can be found here.
• I will close this task, and continue reporting the progress on this line of research here: T260566

Updates

• We have published the first round of analysis.
• Some important highlights:
  • The size of the project (ie number of articles) is not correlated with likelihood of propagate content to other projects.
  • Initial results shows correlation between cultural similarity and the likelihood of two or more projects to share similar content.
  • For long cascades (ie, articles that exists in several languages), we are able to predict with a reasonable accuracy, the new languages that will create articles about the same topic.

• We have finished to develop the first two models, one based in a DNN an another based in BoW. We are working on the report, that should be online (in meta) during the following weeks.
• We are also working in analyzing the different coverage of entities in Wikipedias across languages.
• Based on the two developed models, we are comparing with related work, and discussing about a new model or a fusion of the existing ones.

• First round of analysis and models have finished. We are working on the report, that should be online in the following weeks.

@GoranSMilovanovic looks great.

It is the sum of all sitelinks divided by the total number of articles. Do you think the formulation should be changed? Thanks.

No, I think is good like this.
What would be great is to - additionally - have the statistic only for Wipedias (or by project-type, eg: wikipedia, wicktionary, etc..)

Hi @GoranSMilovanovic ,
First of all, thanks for the work. Looks really good. My only doubt is about this sentence:

• Analyzing which Properties in Wikidata receives much changes.

• Qualitative analysis on the model results.

@leila: this task was delayed due COVID-19 related problems. That is the reason to keep it on the FY20 April-June lane, we are working to finish it at the end of August. I'm ok moving it to the FY21 if that works for you.

@elukey yes! this is working, thank you very much! It would be great if you can apply the same patch in the stat1005.

And I confirm that this error is not happening on the stat1007.

Thanks @JAllemandou I'll try that.

@elukey the pyspark --master yarn solution means to run another notebook (in another port)?

@elukey apparently no changes (I've restarted the server), I'm getting this error:

• Defined a new supervised approach, using a novel technique of negative sampling.

We have build collaborations with 4 researchers. All the paper work (MOUs) and announcements has been done. You can see more details here: https://www.mediawiki.org/wiki/Wikimedia_Research/Formal_collaborations

Additional information, this is the python version (in the driver, I guess):

• Divided problem in two steps:
  • Will the item propagate (Binary classification )?
  • Where will the item propagate?

• Working in representing Wikipedia page as set of Wikidata Items

• Collaborations with Souneil and Giovanni has been announced in @wiki-research-l

• No updates this week

• Defined a subset of pages for manual evaluation.

• Testing a binary classification model to predict if an item will keep propagating to other wikis, or not.

External researchers had solved this problem: https://zenodo.org/record/2539424

• Giovanni and Souneil has been added to the official list of formal collaborators.

• Started to design a crowdsourced task to label data.

Thanks @JAllemandou

Thanks @ashley and @JAllemandou . I've solved the problem using wmf.mediawiki_history, so no emergencies from my side. However, is that the expected behavior for wmf_raw.mediawiki_revision? And in the case that it is, for the records, could you please provide an example of the join you are suggesting, I don't see an actor table in wmf_raw:

• All legal procedures has been finalized.

• Improving the sec2sec model.

• Adding topic information to the model.

Yes @Nuria is an internship. End date August 10th.

great! gpu's for everybody, power to the people :) !!!
Thanks @elukey

Hi
These are the groups we need for @YiJuLu:

@Dzahn & @Aklapper here you have @YiJuLu 's key: https://office.wikimedia.org/wiki/User:Diego_(WMF)/internKeys

• Defined baseline models based in global and local popularity. (always predicting the most popular language transition).