See the investigation results at T188807 and the recently updated documentation at https://wikitech.wikimedia.org/wiki/X-Analytics .

@Krinkle: see documentation and ticket liked from wikitech, while the shear majority of requests are indeed under https there are some that are not. I think a simplified way to [put it is that when we serve content we do so under https, it might very well be that the initial request (met with a 301 or other) was under http. Hopefully this makes sense. Closing ticket but reopen if you feel explanation is not sufficient.

I think we need more info here of why is this important and what you would like to see.

@ema what's the current state or plan for this in ATS?

In T196558#5726093, @Nuria wrote:

I think we need more info here of why is this important and what you would like to see.

The objective is to not send X-Analytics in HTTP response headers to end-users. This seems mostly wasted bandwidth.

If I understand correctly, this header is currently used for two things internally:

1. To transport internal state information about the request during VCL execution to to varnishlog, where it can be picked up by Hadoop etc.
2. To transport internal state information from MediaWiki to the VCL execution.

For case 1, I believe there are other means to transport this information within Varnish that don't require adding it as public property to resp.headers. For example using std.log (VCL_Log) in Varnish. Previous comments and Traffic team knows more about this.

For case 2, headers are unavoidable but that's only between the app server and Varnish frontend. When they are read by the Analytics' VCL code and merged into VCL_Log, the header can be unset after that.

In the immediate future there is not yet a schedule to remove use of Varnish frontend (nginx-tls becomes ats-frontend, and varnish-backend becomes ats-backend. But Varnish frontend for caching and web request logging remains in the middle as today, for now).

But Varnish frontend for caching and web request logging remains in the middle as today, for now).

• i think* next quarter we are starting to migrate varnish frontend to AST? In which case it seems a good time to take a second look to this workflow. cc Traffic and @ema so they can clarify whether the varnish frontend migration is happening come Q3

In T196558#5728077, @Gilles wrote:

@ema what's the current state or plan for this in ATS?

In ATS there is no direct equivalent of VCL_Log + reading things out of VSM, but we can investigate alternative routes to achieve the same goal in Q3.

In T196558#5749468, @Nuria wrote:

But Varnish frontend for caching and web request logging remains in the middle as today, for now).

• i think* next quarter we are starting to migrate varnish frontend to AST? In which case it seems a good time to take a second look to this workflow. cc Traffic and @ema so they can clarify whether the varnish frontend migration is happening come Q3

The plan for Q3 is to (begin? it's a non-negligible amount of work) the migration of all analytics outputs from varnish-fe to ats-tls.

Since the migration work is starting in Q3, it sounds like this task would be obsolete at some point in 2020, right? I guess as part of the migration it would be nice to make it a requirement to get rid of those headers being sent to end users, granted that ATS is capable of doing that.

Change 559711 had a related patch set uploaded (by Ema; owner: Ema):
[operations/puppet@production] ATS: remove X-Analytics from responses sent to users

https://gerrit.wikimedia.org/r/559711

In T196558#5753569, @Gilles wrote:

Since the migration work is starting in Q3, it sounds like this task would be obsolete at some point in 2020, right? I guess as part of the migration it would be nice to make it a requirement to get rid of those headers being sent to end users, granted that ATS is capable of doing that.

I propose we stop sending the header to clients early in Q3, removing it at the ats-tls layer: https://gerrit.wikimedia.org/r/559711. The ats-tls analytics implementation will have to to work in spite of this, obviously, making it something of an implicit requirement.

Change 559711 merged by Ema:
[operations/puppet@production] ATS: remove X-Analytics from responses sent to users

https://gerrit.wikimedia.org/r/559711

Mentioned in SAL (#wikimedia-operations) [2020-01-15T15:28:52Z] <ema> cp3064: ats-tls-restart to apply https://gerrit.wikimedia.org/r/559711 T196558

Mentioned in SAL (#wikimedia-operations) [2020-01-15T15:37:51Z] <vgutierrez> rolling restart of ats-tls instances - T196558 T242778