Hi @Surlycyborg,

Do we have a deadline for this task? Thanks!

The only deadline is really the Application period for Outreachy, which goes until mid-October.

What does this mean: "send us a link to it by commenting on this task." ?
Thanks

Sorry, that was a bit ambiguous I guess :) I mean to make a comment on this Phabricator entry, like we're doing now, with the URL to your repository.

Yes, that's how I understood it, just wanted to be sure as I thought we are competing for the internship and sharing the code might be not the best strategy.

If someone copies code from another submission it will be fairly obvious to us :)

What about the first task, how do we submit it ? I asked in another thread, but haven't received a response.

Hi @Samwalton9 ,
Which section title takes precedence? I noticed there's different levels and was wondering about which one to use.

That is probably a question for @Miriam - I imagine we want the same level of section title that was used to train the model.

@Surlycyborg, do we include the content from the following sections:
'See also',
'References',
'External links',
'Further reading'

cc @Miriam

Those sections typically don't have a lot of usable text that is not links or references, so I think no, we can just ignore those.

In T234606#5549527, @Achillesheel02 wrote:

Hi @Samwalton9 ,
Which section title takes precedence? I noticed there's different levels and was wondering about which one to use.

Hi @Achillesheel02 great question! We should use the section title at the highest level, so that we can generalize across different articles.
E.g. if you have a sentence whose main Section is "Biography", and whose sub-section is "Early life" or "Career", we should keep "Biography" as the section title used as input for the model. Does that make sense?

Also, yes, please discard anything coming from
'See also',
'References',
'External links',
'Further reading'
As @Surlycyborg said, these sections don't have a lot of content needing citations, so we can discard them.

Hi @Miriam @Surlycyborg,
Here's my repo: https://github.com/achillesheel02/wikimedia-citation-classification-task-2.git
Thank you.

Note: mwapi requires python 3 while the citation is written in python 2

@Surlycyborg the deadline according to Outreacy is the 5th of Nov right?

@Miriam @Samwalton9 So according to this task the function should take the article title, but I assume the user may not enter the article title exactly as what it appears it Wikipedia, so I assume I am gonna use MediaWiki API to search for articles.

So then should I take the first result of the search or should I ask the user to select on the titles given I will show the result to the user.

In T234606#5553671, @Achillesheel02 wrote:

Hi @Miriam @Surlycyborg,
Here's my repo: https://github.com/achillesheel02/wikimedia-citation-classification-task-2.git
Thank you.

Thank you! I hope to take a look and maybe file an issue in the next couple of days :)

In T234606#5562230, @Ghassanmas wrote:

@Surlycyborg the deadline according to Outreacy is the 5th of Nov right?

Looks like it, thanks for the link. The mentor's documentation didn't have a date :( I'll use your link in the task description.

@Miriam @Samwalton9 So according to this task the function should take the article title, but I assume the user may not enter the article title exactly as what it appears it Wikipedia, so I assume I am gonna use MediaWiki API to search for articles.

So then should I take the first result of the search or should I ask the user to select on the titles given I will show the result to the user.

For the purposes of this task, I think we don't need to search: just output an error if the article title as given gives no results from the API. In the actual project, we will likely need to process *all* articles on Wikipedia, or at least all within a certain category - so there is no user-input title anyway.

I think the API already normalizes article titles to some extent (e.g. if you ask for the text of an article with spaces, it normalizes spaces to underscores), so we do have a little bit of freedom already in how we take input from the user for this task.

@Surlycyborg But the mediawiki API usually returns the disambiguation page or the redirect page not the exact article. which would need to send another request assuming the redirect is correct or the first result of the disambiguation is correct. But if using the search API before then we could get the article by pageid instead.

@Ghassanmas you’re right, this is also the path I followed.

In T234606#5562504, @Ghassanmas wrote:

@Surlycyborg But the mediawiki API usually returns the disambiguation page or the redirect page not the exact article. which would need to send another request assuming the redirect is correct or the first result of the disambiguation is correct. But if using the search API before then we could get the article by pageid instead.

In T234606#5562504, @Ghassanmas wrote:

@Surlycyborg But the mediawiki API usually returns the disambiguation page or the redirect page not the exact article. which would need to send another request assuming the redirect is correct or the first result of the disambiguation is correct. But if using the search API before then we could get the article by pageid instead.

Huh, interesting. To be clear, are you using the query API ? Using that API, I would have tried something like this to get the text using the default title normalization of the API.

That said, using search and grabbing the first result is perfectly fine too - and if it gives us a better user experience by accepting different spellings of articles, that's all the better :)

Hi @Surlycyborg @Miriam,

Here is my repo:
https://github.com/AikoChou/wikimedia-outreachy-2019

Thanks. :)

@Surlycyborg @Miriam @Samwalton9 I have confusion hopefully you could clear it out.

First, could you define the statement and the sentence in the scope of this project.

I see in your paper that you did the rain on random sentences. But the example on the github repo in doing the prediction on statements which could be composed of multiple sentences.

So I am not sure when parsing the article if I need to split the paragraphs by sentences or using another method.

In T234606#5573542, @Ghassanmas wrote:

@Surlycyborg @Miriam @Samwalton9 I have confusion hopefully you could clear it out.

First, could you define the statement and the sentence in the scope of this project.

I see in your paper that you did the rain on random sentences. But the example on the github repo in doing the prediction on statements which could be composed of multiple sentences.

Hi @Ghassanmas I am also confusing about the definition between statement and sentence in this project. Thanks for pointing out.

As I see in the github repo, at the beginning of the function text_to_word_list, it dealt with the multiple sentences in a statement by selecting the first sentence. (or randomly choose one, but this line was commented out)

So in my opinion, the paragraphs need to be split into sentences as the third point in the task description then classify them.

Hi there @Miriam @Surlycyborg and @Samwalton9,

Here is my repo:
https://github.com/kendallcorner/citation-needed-paper

I forked your script and added some things. It completes all the items in this task. I was also confused on how to determine what a 'statement' was, since the example input file had multiple sentences in a statement, but I just tried something. Let me know what you think.

Thanks!
Kendall

@AikoChou @Kendallcorner Thank you! I hope to have a look in the next few days (weekend is more realistic) and leave some comments as an issue on the repository :)

@Ghassanmas Yes, please split the paragraphs into sentences.

Hi @Miriam @Surlycyborg,
Here's my repo: https://github.com/hammedb197/wikimedia_citation_classification.git

Thank you.

Hi @Miriam @Surlycyborg
I was just wondering, as Outreachy participants, is there any kind of correspondence we need to give to Outreachy itself? Because technically we are supposed to ‘record’ the contributions in our Outreachy accounts.

@Achillesheel02: That question does not sound directly related to the topic of this task - can you please bring it up in an Outreachy channel (e.g. Zulip)? Thanks for your understanding.

Hey, that is my repo https://github.com/jubs12/citation_needed. :)
Thanks

Please note that the Outreachy deadline is November 5th! We've not had many people finalise their application yet so please make sure you do that if you want to be considered!

You can record your contribution and make a final application at https://www.outreachy.org/december-2019-to-march-2020-internship-round/communities/wikimedia/a-system-for-releasing-data-dumps-from-a-classifie/contributions/

In T234606#5587019, @Achillesheel02 wrote:

Hi @Miriam @Surlycyborg
I was just wondering, as Outreachy participants, is there any kind of correspondence we need to give to Outreachy itself? Because technically we are supposed to ‘record’ the contributions in our Outreachy accounts.

cc @Samwalton9

In T234606#5618796, @Samwalton9 wrote:

Please note that the Outreachy deadline is November 5th! We've not had many people finalise their application yet so please make sure you do that if you want to be considered!

You can record your contribution and make a final application at https://www.outreachy.org/december-2019-to-march-2020-internship-round/communities/wikimedia/a-system-for-releasing-data-dumps-from-a-classifie/contributions/

Hi @Miriam @Surlycyborg here is the link to my repository https://github.com/tinumide/wikimedia_task_2

Hi @Miriam @Surlycyborg and @Samwalton9 ! Here is the link to my repo https://github.com/ferculell/wikimedia-citation-needed-project
Looking forward your comments.

Thank you!

Thank you all for your submissions! I'm looking through them and they will very much be considered along with the final submission (deadline November 5th, as @Samwalton9 mentioned), even if I don't make further comments on GitHub :)

@Surlycyborg Hi, can we still submit ? I am just done with the second task.

In T234606#5629558, @iriyagupta wrote:

@Surlycyborg Hi, can we still submit ? I am just done with the second task.

Yes! You have about 24 hours from now to submit your contributions and final application on the Outreachy website!

Hi @Miriam @Surlycyborg and @Samwalton9 ! In the final application, Outreachy ask the applicants to fill out a timeline for the internship project. I feel a bit unsure about how to do it. Could you provide us with any guidances or advices about it? Thanks :)

Hi! Here is my repository https://github.com/LiviaCavalcanti/citation-needed-paper/

Final reminder! The application deadline is in ~6 hours.

You can record your contribution and make a final application at https://www.outreachy.org/december-2019-to-march-2020-internship-round/communities/wikimedia/a-system-for-releasing-data-dumps-from-a-classifie/contributions/

Hi there, this is my repo https://github.com/salwa-gebaly/detecting-unsourced-sentences.
@Samwalton9 @Miriam @Surlycyborg

I unfortunately began contributing very very late, also looking only on Wikimedia outreachy page, because fabricator website didn't work from my laptop, this prevented me from all the clarifications in this page, but here we are now.

My question now would be, could we do any additional work after the deadline?

Thanks.

Hello here is my repo, https://github.com/ghassanmas/citation-needed-paper
You can run the script by python article.py then follow the instructions.

Thanks!

@Samwalton9 @Miriam @Surlycyborg

Thanks for creating a proposal! As we are past the deadline, if you would like us to consider your proposal for review, please move it to the submitted column. Thank you!

Hi @srishakatux,
I don't know what you mean by moving it to the submitted column?
Thanks

@Samwalton9 @Miriam when are we supposed to hear back?

Hi, thank you all again for your submissions! We chose @AikoChou for this project couple of weeks ago. Sorry we never wrote back here -- I actually thought applicants would get some communication through the Outreachy platform.