Page MenuHomePhabricator

"start line" parameter should work for data retrieval from csv-files
Closed, ResolvedPublic

Description

Setup

  • MediaWiki 1.35.14 (ef29848) 2024-01-01T09:28:21
  • PHP 7.4.3-4ubuntu2.19 (apache2handler)
  • MariaDB 10.3.38-MariaDB-0ubuntu0.20.04.1
  • External Data 3.2 (d77e75a) 2022-10-04T18:06:50

Issue
A call to, e.g., #display_external_table works if you use the "end line" parameter like this:

{| class="wikitable"
! number
! name
! code
{{#display_external_table:
  source=https://example.org/w/images/9/9c/Export_slice_123.csv
 |format=CSV with header
 |end line=10
 |data= mynr=number, priref=name, pc=code
 |template=Test Template
}}
|}

The results until line 10 show.

Once you add the "start line" parameter like this:

{| class="wikitable"
! number
! name
! code
{{#display_external_table:
  source=https://example.org/w/images/9/9c/Export_slice_123.csv
 |format=CSV with header
 |start line=2
 |end line=10
 |data= mynr=number, priref=name, pc=code
 |template=Test Template
}}
|}

No data are shown.

The same happens when using the #get_web_data: parser function to get some data out of the file.

Event Timeline

Kghbln renamed this task from "start line" parameter should work for data retrieval to "start line" parameter should work for data retrieval from csv-files.Jan 17 2024, 10:20 AM
Kghbln updated the task description. (Show Details)

Change 991314 had a related patch set uploaded (by Alex Mashin; author: mashin):

[mediawiki/extensions/ExternalData@master] Improve cutout features and error handling.

https://gerrit.wikimedia.org/r/991314

With start line > 1 CSV with headers will not work. start line and end line are applied to the file before it is parsed and header line is taken into account. You can use header lines = 1 to guarantee that the headers are included, and, therefore, columns have correct names.

alex-mashin changed the task status from Open to In Progress.Jan 17 2024, 12:17 PM
alex-mashin claimed this task.

Change 991314 merged by jenkins-bot:

[mediawiki/extensions/ExternalData@master] Improve cutout features and error handling.

https://gerrit.wikimedia.org/r/991314

With start line > 1 CSV with headers will not work. start line and end line are applied to the file before it is parsed and header line is taken into account. You can use header lines = 1 to guarantee that the headers are included, and, therefore, columns have correct names.

Thanks for the info. Good to know. Hmm, this is a noob question, but how can I retrieve just the content from the lines, e.g., five to ten, disregarding the lines one to four? Looks like this is not directly possible. Probably, I have to go via the #store_external_table: parser function and query the data wherever and however I stored them.

With start line > 1 CSV with headers will not work. start line and end line are applied to the file before it is parsed and header line is taken into account. You can use header lines = 1 to guarantee that the headers are included, and, therefore, columns have correct names.

Thanks for the info. Good to know. Hmm, this is a noob question, but how can I retrieve just the content from the lines, e.g., five to ten, disregarding the lines one to four? Looks like this is not directly possible. Probably, I have to go via the #store_external_table: parser function and query the data wherever and however I stored them.

Upgrade and set header lines = 1 | start line = 5 | end line = 10, assuming that you count lines from one, and the count includes the header line.

@Kghbln, have you managed to show a fragment of a CSV-file?

I have just successfully tested the following setup:

  • MediaWiki 1.42.0-alpha (c6251dc) 06:57, 8 January 2024
  • PHP 7.4.3-4ubuntu2.19 (apache2handler)
  • ICU 66.1
  • MariaDB 10.3.38-MariaDB-0ubuntu0.20.04.1
  • External Data 3.3 (48baa1b) 17:37, 17 January 2024 // master

Thanks a lot for dealing with this!