Page MenuHomePhabricator

Define a special range in constants.pp for the LVS hosts
Closed, DeclinedPublic

Description

Just now I was trying to make a ferm rule for a web backend (labweb1001/1002) that limited port 80 access to the lvs servers. All the examples I see just use $DOMAIN_NETWORKS which seems a bit broad.

Probably we should define something limited to the LVS servers and use that for things behind lvs.

Event Timeline

Andrew created this task.Feb 21 2018, 4:55 PM
Restricted Application added a subscriber: Aklapper. · View Herald TranscriptFeb 21 2018, 4:55 PM
mark added a subscriber: mark.Feb 21 2018, 5:42 PM

You are setting up a publicly accessible web service, right? So you should probably open up port 80 (and/or 443) to the entire world, not just LVS servers.

Traffic is "routed" via LVS (more or less), not proxied by it. The source IPs your web server(s) will see are the original source IPs from your users. Return traffic will also go straight to your users, not even passing via LVS.

This particular service is behind the misc-web varnishes. So port 80 needs to be open to those varnishes and to lvs, but nothing else. $DOMAIN_NETWORKS covers both those sets, so it's reasonable but more permissive than really necessary.

@Andrew : There is $CACHE_MISC already as a network constant / ferm macro.

Dzahn added a subscriber: Dzahn.Feb 28 2018, 3:36 PM
modules/profile/manifests/phabricator/main.pp:        srange => '$CACHE_MISC',
modules/profile/manifests/ci/firewall.pp:        srange => '$CACHE_MISC',
modules/profile/manifests/releases/mediawiki.pp:        srange => '$CACHE_MISC',
modules/profile/manifests/iegreview.pp:        srange => '$CACHE_MISC',
modules/profile/manifests/etherpad.pp:        srange => '$CACHE_MISC',
modules/profile/manifests/wikimania_scholarships.pp:        srange => '$CACHE_MISC'
...

Yes, $CACHE_MISC includes the varnishes, but not lvs. See https://gerrit.wikimedia.org/r/#/c/413194/

You will see that before that patch, I was already using CACHE_MISC. Before that patch, LVS was unable to contact my hosts and marked them both as down. After that patch LVS detected that both hosts were up and pooled them.

Dzahn added a comment.Feb 28 2018, 4:06 PM

Gotcha! Yea, we could add a new one for _just_ LVS hosts and then combine them similar to:

$deployable_networks = [
    $mw_appserver_networks,
    $analytics_networks,
BBlack added a subscriber: BBlack.Feb 28 2018, 4:09 PM

Some of this conversation is confusing! :) I take it the situation is it's a public service behind cache misc, and our internal LVS ranges are used *behind* cache_misc to route to multiple backends with healthchecking.

If the service is going to be public, shouldn't it also be directly open inside our network? Is there a reason we want it exposed via cache_misc, but inaccessible for a test query from e.g. neodymium?

(to clarify: that we haven't needed an LVS-monitoring-ips ferm rule before implies that all the other services behind internal LVS (including those behind cache_misc) just have open ports. I'm wondering why this one is special-er).

For what is worth, I don't like the idea of adding anything like that in network::constants. I don't even like the current $special_hosts construct (it has gotten out of hand) and I am the one who started it. ferm rules should not be defined using the macro way, since that is not immediately clear how it is constructed and thus difficult to reason about. The macro is only populated on the hosts, using ERB, it's uppercase and git grepping for it in our repo only reveals the uses, not the definition. Doing the mental jump from that to network::constants is not something we should be forcing ourselves to do. Instead we should be using role specific hiera lookups.

akosiaris closed this task as Declined.Apr 4 2018, 7:53 AM

I am gonna close this as declined. Feel free to reopen though.