Change Details

###Data Platform Engineering Bug Report or Data Problem Form. ####Please fill out the following ####Please ensure you set priority What kind of problem are you reporting? - [] Access related problem - [] Service related problem - [x] Data related problem #####For a data related problem: - Is this a data quality issue? Yes - What datasets and/or dashboards are affected? [[ https://datahub.wikimedia.org/dataset/urn:li:dataset:(urn:li:dataPlatform:hive,wmf.interlanguage_navigation,PROD)/Schema?is_lineage_mode=false&schemaFilter= | wmf.internlanguage_navigation ]] - What are the observed vs expected results? * Expected: counts of **//all//** daily webrequests that go from one language version of a project to another language version of the same project. * Actual: excludes mobile requests (i.e., excludes webrequests in which the referer has ".m" or other qualifier) See lines 55 and 56 in https://gerrit.wikimedia.org/g/analytics/refinery/+/fee5f29f8f1955f292532e65478bc6eaddea9846/hql/interlanguage/daily/interlanguage_navigation.hql (pasted below) ``` -- The referer host has no .m, or other qualifiers AND size(normalize_host(parse_url(referer, 'HOST')).qualifiers) = 0 ``` Solution: Is it possible to fix this to include mobile referrers going forward? And is it possible to retroactively correct the historical data, as well?

###Data Platform Engineering Bug Report or Data Problem Form. ####Please fill out the following ####Please ensure you set priority What kind of problem are you reporting? - [] Access related problem - [] Service related problem - [x] Data related problem #####For a data related problem: - Is this a data quality issue? Yes - What datasets and/or dashboards are affected? [[ https://datahub.wikimedia.org/dataset/urn:li:dataset:(urn:li:dataPlatform:hive,wmf.interlanguage_navigation,PROD)/Schema?is_lineage_mode=false&schemaFilter= | wmf.interlanguage_navigation ]] - What are the observed vs expected results? * Expected: counts of **//all//** daily webrequests that go from one language version of a project to another language version of the same project. * Actual: excludes mobile requests (i.e., excludes webrequests in which the referer has ".m" or other qualifier) See lines 55 and 56 in https://gerrit.wikimedia.org/g/analytics/refinery/+/fee5f29f8f1955f292532e65478bc6eaddea9846/hql/interlanguage/daily/interlanguage_navigation.hql (pasted below) ``` -- The referer host has no .m, or other qualifiers AND size(normalize_host(parse_url(referer, 'HOST')).qualifiers) = 0 ``` Solution: Is it possible to fix this to include mobile referrers going forward? And is it possible to retroactively correct the historical data, as well?

###Data Platform Engineering Bug Report or Data Problem Form. ####Please fill out the following ####Please ensure you set priority What kind of problem are you reporting? - [] Access related problem - [] Service related problem - [x] Data related problem #####For a data related problem: - Is this a data quality issue? Yes - What datasets and/or dashboards are affected? [[ https://datahub.wikimedia.org/dataset/urn:li:dataset:(urn:li:dataPlatform:hive,wmf.interlanguage_navigation,PROD)/Schema?is_lineage_mode=false&schemaFilter= | wmf.internlanguage_navigation ]] - What are the observed vs expected results? * Expected: counts of **//all//** daily webrequests that go from one language version of a project to another language version of the same project. * Actual: excludes mobile requests (i.e., excludes webrequests in which the referer has ".m" or other qualifier) See lines 55 and 56 in https://gerrit.wikimedia.org/g/analytics/refinery/+/fee5f29f8f1955f292532e65478bc6eaddea9846/hql/interlanguage/daily/interlanguage_navigation.hql (pasted below) ``` -- The referer host has no .m, or other qualifiers AND size(normalize_host(parse_url(referer, 'HOST')).qualifiers) = 0 ``` Solution: Is it possible to fix this to include mobile referrers going forward? And is it possible to retroactively correct the historical data, as well?