Page MenuHomePhabricator

Deploy lazy loaded images to mobile web
Closed, ResolvedPublic1 Story Points

Description

Deploy lazy loaded images to the mobile web.

Event Timeline

dr0ptp4kt created this task.Aug 8 2016, 4:08 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptAug 8 2016, 4:08 PM
dr0ptp4kt triaged this task as High priority.Aug 8 2016, 4:08 PM
MBinder_WMF set the point value for this task to 1.Aug 10 2016, 4:16 PM
MBinder_WMF moved this task from Needs Analysis to To Do on the Reading-Web-Sprint-79-Uh-oh board.
jhobs updated the task description. (Show Details)Aug 10 2016, 4:17 PM
dr0ptp4kt renamed this task from Deploy lazy loaded images to all large mobile web Wikipedias to Deploy lazy loaded images to all mobile web Wikipedias.Aug 15 2016, 4:17 PM
dr0ptp4kt updated the task description. (Show Details)
dr0ptp4kt added a subscriber: Zppix.

@BBlack when we do this (within next 2 weeks), would you be able to take a look at the load on the mobile web servers before and after the change and whether there is any positive impact?

@BBlack when we do this (within next 2 weeks), would you be able to take a look at the load on the mobile web servers before and after the change and whether there is any positive impact?

I think by this you mean, can we look for a reduction in image requests from our https://upload.wikimedia.org from mobile (*.m.wikiwhatever.org) referers? We can see this data in the sampled-1000 outputs we look at for quick estimations, yes. Analytics has this data as well if you want a more-comprehensive analysis.

@BBlack hey apologies for the vagueness - I just seem to remember a conversation around you being able to check a few things (e.g. memory usage/cpu usage/bytes shipped) post this change specific to mobile domains (e.g. exclude desktop traffic). If I can just use graphite for this please ignore my request.

@jhobs looks like you can SWAT this today.

@Jdlrobson - We can look for a reduction in total traffic from the upload caches in bytes terms in grafana and other such data, but only for all of upload as a whole easily. The data has a fair amount of hourly/daily/weekly random variance, and then requests serviced by the upload caches don't all have referrers from our sites in general (a lot of third parties link our images freely), and mobile in turn is a smaller fraction than desktop. So the effect may be visible, but it may be small and/or hard to distinguish from the noise.

Whereas in the analytics logs, we can actually look at downloads which had a referrer from our mobile sites. We can do that in terms of request-count or bytes (although I think if a long image transfer is aborted, it may still log the entire size). We (ops) have easy access to a 1/1000 sample of that data going back 2 months at any given time, and Analytics has the fuller dataset.

@jhobs looks like you can SWAT this today.

Evening SWAT is outside of my usual working hours, but I'll schedule it for tomorrow morning.

@dr0ptp4kt Per an IRC convo, can we get absolute complete confirmation that this needs to go out to all Wikipedias and can't go out to all Wikis (and reasoning)?

Change 305387 had a related patch set uploaded (by Jhobs):
Deploy lazy loaded images to all mobile web wikipedias

https://gerrit.wikimedia.org/r/305387

@jhobs, based on that IRC conversation, feel free to enable for all wikis.

dr0ptp4kt renamed this task from Deploy lazy loaded images to all mobile web Wikipedias to Deploy lazy loaded images to mobile web.Aug 17 2016, 10:32 PM
dr0ptp4kt updated the task description. (Show Details)
jhobs moved this task from Doing to Code Review on the Reading-Web-Sprint-79-Uh-oh board.

Change 305387 merged by jenkins-bot:
Deploy lazy loaded images to mobile web

https://gerrit.wikimedia.org/r/305387

jhobs reassigned this task from jhobs to dr0ptp4kt.Aug 18 2016, 3:35 PM
jhobs added a subscriber: jhobs.

For signoff!

@Jdlrobson - We can look for a reduction in total traffic from the upload caches [...] the effect may be visible, but it may be small and/or hard to distinguish from the noise.

Was just looking at raw request counts for the upload clusters as a whole, and there is a notable drop in total image requests that seems to correlate with about what we'd expect for cache expiry of older pages and replacement with lazy-loaded images. Can't confirm until we dig into other correlating stats next week, though :) Notice the right edge of this 30-day view of total requests to the upload caches:

dr0ptp4kt closed this task as Resolved.Aug 22 2016, 8:05 PM

Signing off. This seems to be taking hold.

s6-analytics-slave.eqiad.wmnet [log]> select
    -> event_isAnon,
    -> event_isHttp2,
    -> event_lazyLoadImages,
    -> left(timestamp,8) as ts,
    -> count(*)
    -> from NavigationTiming_15485142
    -> where
    -> timestamp > '20160818'
    -> and timestamp < '2016082219'
    -> and event_mobileMode is not null
    -> group by
    -> event_isAnon,
    -> event_isHttp2,
    -> event_lazyLoadImages,
    -> ts
    -> order by
    -> event_isAnon,
    -> event_isHttp2,
    -> event_lazyLoadImages,
    -> ts
    -> ;
+--------------+---------------+----------------------+----------+----------+
| event_isAnon | event_isHttp2 | event_lazyLoadImages | ts       | count(*) |
+--------------+---------------+----------------------+----------+----------+
|            0 |             0 | NULL                 | 20160818 |       45 |
|            0 |             0 | A                    | 20160818 |       41 |
|            0 |             0 | A                    | 20160819 |       74 |
|            0 |             0 | A                    | 20160820 |      101 |
|            0 |             0 | A                    | 20160821 |       82 |
|            0 |             0 | A                    | 20160822 |       46 |
|            0 |             1 | NULL                 | 20160818 |      116 |
|            0 |             1 | A                    | 20160818 |      121 |
|            0 |             1 | A                    | 20160819 |      245 |
|            0 |             1 | A                    | 20160820 |      233 |
|            0 |             1 | A                    | 20160821 |      278 |
|            0 |             1 | A                    | 20160822 |      183 |
|            1 |             0 | NULL                 | 20160818 |    29977 |
|            1 |             0 | NULL                 | 20160819 |     6204 |
|            1 |             0 | NULL                 | 20160820 |      309 |
|            1 |             0 | NULL                 | 20160821 |      132 |
|            1 |             0 | NULL                 | 20160822 |       64 |
|            1 |             0 | A                    | 20160818 |    15010 |
|            1 |             0 | A                    | 20160819 |    37950 |
|            1 |             0 | A                    | 20160820 |    46042 |
|            1 |             0 | A                    | 20160821 |    51671 |
|            1 |             0 | A                    | 20160822 |    35495 |
|            1 |             1 | NULL                 | 20160818 |    93566 |
|            1 |             1 | NULL                 | 20160819 |    18517 |
|            1 |             1 | NULL                 | 20160820 |      169 |
|            1 |             1 | NULL                 | 20160821 |       18 |
|            1 |             1 | NULL                 | 20160822 |        6 |
|            1 |             1 | A                    | 20160818 |    37152 |
|            1 |             1 | A                    | 20160819 |   112026 |
|            1 |             1 | A                    | 20160820 |   142430 |
|            1 |             1 | A                    | 20160821 |   161046 |
|            1 |             1 | A                    | 20160822 |   103169 |
+--------------+---------------+----------------------+----------+----------+